vsock/virtio: fix potential unbounded skb queue
Subsystems (6)
PATCH 1/1 vsock/virtio: fix potential unbounded skb queue
virtio_transport_inc_rx_pkt() checks vvs->rx_bytes + len > vvs->buf_alloc.
virtio_transport_recv_enqueue() skips coalescing for packets with VIRTIO_VSOCK_SEQ_EOM.
If fed with packets with len == 0 and VIRTIO_VSOCK_SEQ_EOM, a very large number of packets can be queued because vvs->rx_bytes stays at 0.
Fix this by estimating the skb metadata size:
(Number of skbs in the queue) * SKB_TRUESIZE(0)
--- net/vmw_vsock/virtio_transport_common.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
net/vmw_vsock/virtio_transport_common.c +3 -1
@@ -447,7 +447,9 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,static bool virtio_transport_inc_rx_pkt (struct virtio_vsock_sock *vvs,u32 len) { -if (vvs->buf_used + len > vvs->buf_alloc) +u64 skb_overhead = (skb_queue_len (&vvs->rx_queue) +1 ) *SKB_TRUESIZE (0 ); + +if (skb_overhead + vvs->buf_used + len > vvs->buf_alloc)return false; vvs->rx_bytes += len;
Signature · 1 line
2.54.0.545.g6539524ca2-goog
This patch was applied to netdev/net.git (main) by Jakub Kicinski <[email protected]>:
Hello:
This patch was applied to netdev/net.git (main)
by Jakub Kicinski <[email protected]>:
Quoted 10 lines · On Thu, 30 Apr 2026 12:26:52 +0000 you wrote:
virtio_transport_inc_rx_pkt()checksvvs->rx_bytes+ len >vvs->buf_alloc.
virtio_transport_recv_enqueue()skips coalescing for packets withVIRTIO_VSOCK_SEQ_EOM.If fed with packets with len == 0 and
VIRTIO_VSOCK_SEQ_EOM, a very large number of packets can be queued becausevvs->rx_bytesstays at 0.[...]
Here is the summary with links:
- [net] vsock/virtio: fix potential unbounded skb queue
https://git.kernel.org/netdev/net/c/059b7dbd20a6
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html
Stefano Garzarella
maintainerQuoted 33 lines · Eric Dumazet
virtio_transport_inc_rx_pkt()checksvvs->rx_bytes+ len >vvs->buf_alloc.
virtio_transport_recv_enqueue()skips coalescing for packets withVIRTIO_VSOCK_SEQ_EOM.If fed with packets with len == 0 and
VIRTIO_VSOCK_SEQ_EOM, a very large number of packets can be queued becausevvs->rx_bytesstays at 0.Fix this by estimating the skb metadata size:
(Number of skbs in the queue) *
SKB_TRUESIZE(0)Fixes 077706165717 ("virtio/vsock: don't use skbuff state to account credit")Signed-off-by Eric Dumazet <[email protected]>Cc Arseniy Krasnov <[email protected]>Cc Stefan Hajnoczi <[email protected]>Cc Stefano Garzarella <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Xuan Zhuo <[email protected]>Cc "Eugenio Pérez" <[email protected]>---
net/vmw_vsock/virtio_transport_common.c| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)net/vmw_vsock/virtio_transport_common.c +0
-0
net/vmw_vsock/virtio_transport_common.c static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, · line 453
Quoted 8 lines
net/vmw_vsock/virtio_transport_common.c +3
-1@@ -447,7 +447,9 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,static bool virtio_transport_inc_rx_pkt (struct virtio_vsock_sock *vvs,u32 len) { -if (vvs->buf_used + len > vvs->buf_alloc) +u64 skb_overhead = (skb_queue_len (&vvs->rx_queue) +1 ) *SKB_TRUESIZE (0 ); + +if (skb_overhead + vvs->buf_used + len > vvs->buf_alloc)return false;
I'm not sure about this fix, I mean that maybe this is incomplete. In virtio-vsock, there is a credit mechanism between the two peers:
https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/virtio-v1.3-csd01.html#x1-4850003
This takes only the payload into account, so it’s true that this problem exists; however, perhaps we should also inform the other peer of a lower credit balance, otherwise the other peer will believe it has much more credit than it actually does, send a large payload, and then the packet will be discarded and the data lost (there are no retransmissions, etc.).
Thanks,
Stefano
Quoted 48 lines · Stefano Garzarella
Quoted 33 lines · Eric Dumazet
virtio_transport_inc_rx_pkt()checksvvs->rx_bytes+ len >vvs->buf_alloc.
virtio_transport_recv_enqueue()skips coalescing for packets withVIRTIO_VSOCK_SEQ_EOM.If fed with packets with len == 0 and
VIRTIO_VSOCK_SEQ_EOM, a very large number of packets can be queued becausevvs->rx_bytesstays at 0.Fix this by estimating the skb metadata size:
(Number of skbs in the queue) *
SKB_TRUESIZE(0)Fixes 077706165717 ("virtio/vsock: don't use skbuff state to account credit")Signed-off-by Eric Dumazet <[email protected]>Cc Arseniy Krasnov <[email protected]>Cc Stefan Hajnoczi <[email protected]>Cc Stefano Garzarella <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Xuan Zhuo <[email protected]>Cc "Eugenio Pérez" <[email protected]>---
net/vmw_vsock/virtio_transport_common.c| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)net/vmw_vsock/virtio_transport_common.c +0
-0net/vmw_vsock/virtio_transport_common.c static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, · line 453
Quoted 8 lines
net/vmw_vsock/virtio_transport_common.c +3
-1@@ -447,7 +447,9 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,static bool virtio_transport_inc_rx_pkt (struct virtio_vsock_sock *vvs,u32 len) { -if (vvs->buf_used + len > vvs->buf_alloc) +u64 skb_overhead = (skb_queue_len (&vvs->rx_queue) +1 ) *SKB_TRUESIZE (0 ); + +if (skb_overhead + vvs->buf_used + len > vvs->buf_alloc)return false;I'm not sure about this fix, I mean that maybe this is incomplete. In virtio-vsock, there is a credit mechanism between the two peers:
https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/virtio-v1.3-csd01.html#x1-4850003
Quoted 6 lines
This takes only the payload into account, so it’s true that this problem exists; however, perhaps we should also inform the other peer of a lower credit balance, otherwise the other peer will believe it has much more credit than it actually does, send a large payload, and then the packet will be discarded and the data lost (there are no retransmissions, etc.).
I dunno, perhaps revert 077706165717 ("virtio/vsock: don't use skbuff state to account credit")
and find a better fix then?
There is always a discrepancy between skb->len and skb->truesize. You will not be able to announce a 1MB window, and accept one milliion skb of 1-byte each.
This kind of contract is broken.
Stefano Garzarella
maintainerQuoted 56 lines · Eric Dumazet
Quoted 48 lines · Stefano Garzarella
Quoted 33 lines · Eric Dumazet
virtio_transport_inc_rx_pkt()checksvvs->rx_bytes+ len >vvs->buf_alloc.
virtio_transport_recv_enqueue()skips coalescing for packets withVIRTIO_VSOCK_SEQ_EOM.If fed with packets with len == 0 and
VIRTIO_VSOCK_SEQ_EOM, a very large number of packets can be queued becausevvs->rx_bytesstays at 0.Fix this by estimating the skb metadata size:
(Number of skbs in the queue) *
SKB_TRUESIZE(0)Fixes 077706165717 ("virtio/vsock: don't use skbuff state to account credit")Signed-off-by Eric Dumazet <[email protected]>Cc Arseniy Krasnov <[email protected]>Cc Stefan Hajnoczi <[email protected]>Cc Stefano Garzarella <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Xuan Zhuo <[email protected]>Cc "Eugenio Pérez" <[email protected]>---
net/vmw_vsock/virtio_transport_common.c| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)net/vmw_vsock/virtio_transport_common.c +0
-0net/vmw_vsock/virtio_transport_common.c static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, · line 453
Quoted 8 lines
net/vmw_vsock/virtio_transport_common.c +3
-1@@ -447,7 +447,9 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,static bool virtio_transport_inc_rx_pkt (struct virtio_vsock_sock *vvs,u32 len) { -if (vvs->buf_used + len > vvs->buf_alloc) +u64 skb_overhead = (skb_queue_len (&vvs->rx_queue) +1 ) *SKB_TRUESIZE (0 ); + +if (skb_overhead + vvs->buf_used + len > vvs->buf_alloc)return false;I'm not sure about this fix, I mean that maybe this is incomplete. In virtio-vsock, there is a credit mechanism between the two peers:
https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/virtio-v1.3-csd01.html#x1-4850003Quoted 6 lines
This takes only the payload into account, so it’s true that this problem exists; however, perhaps we should also inform the other peer of a lower credit balance, otherwise the other peer will believe it has much more credit than it actually does, send a large payload, and then the packet will be discarded and the data lost (there are no retransmissions, etc.).
Quoted 3 lines
I dunno, perhaps revert 077706165717 ("virtio/vsock: don't use skbuff state to account credit")
and find a better fix then?
IIRC the same issue was there before the commit fixed by that one (commit 71dc9ec9ac7d ("virtio/vsock: replace virtio_vsock_pkt with sk_buff")), so not sure about reverting it TBH.
CCing Arseniy and Bobby.
Quoted 7 lines
There is always a discrepancy between
skb->lenandskb->truesize. You will not be able to announce a 1MB window, and accept one milliion skb of 1-byte each.This kind of contract is broken.
Yep, I agree, but before we start discarding data (and losing it), IMHO we should at least inform the other peer that we're out of space.
@Stefan, @Michael, do you think we can do something in the spec to avoid this issue and in some way take into account also the metadata in the credit. I mean to avoid the 1-byte packets flooding.
Thanks,
Stefano
Quoted 78 lines · Stefano Garzarella
Quoted 56 lines · Eric Dumazet
Quoted 48 lines · Stefano Garzarella
Quoted 33 lines · Eric Dumazet
virtio_transport_inc_rx_pkt()checksvvs->rx_bytes+ len >vvs->buf_alloc.
virtio_transport_recv_enqueue()skips coalescing for packets withVIRTIO_VSOCK_SEQ_EOM.If fed with packets with len == 0 and
VIRTIO_VSOCK_SEQ_EOM, a very large number of packets can be queued becausevvs->rx_bytesstays at 0.Fix this by estimating the skb metadata size:
(Number of skbs in the queue) *
SKB_TRUESIZE(0)Fixes 077706165717 ("virtio/vsock: don't use skbuff state to account credit")Signed-off-by Eric Dumazet <[email protected]>Cc Arseniy Krasnov <[email protected]>Cc Stefan Hajnoczi <[email protected]>Cc Stefano Garzarella <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Xuan Zhuo <[email protected]>Cc "Eugenio Pérez" <[email protected]>---
net/vmw_vsock/virtio_transport_common.c| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)net/vmw_vsock/virtio_transport_common.c +0
-0net/vmw_vsock/virtio_transport_common.c static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, · line 453
Quoted 8 lines
net/vmw_vsock/virtio_transport_common.c +3
-1@@ -447,7 +447,9 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,static bool virtio_transport_inc_rx_pkt (struct virtio_vsock_sock *vvs,u32 len) { -if (vvs->buf_used + len > vvs->buf_alloc) +u64 skb_overhead = (skb_queue_len (&vvs->rx_queue) +1 ) *SKB_TRUESIZE (0 ); + +if (skb_overhead + vvs->buf_used + len > vvs->buf_alloc)return false;I'm not sure about this fix, I mean that maybe this is incomplete. In virtio-vsock, there is a credit mechanism between the two peers:
https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/virtio-v1.3-csd01.html#x1-4850003Quoted 6 lines
This takes only the payload into account, so it’s true that this problem exists; however, perhaps we should also inform the other peer of a lower credit balance, otherwise the other peer will believe it has much more credit than it actually does, send a large payload, and then the packet will be discarded and the data lost (there are no retransmissions, etc.).
Quoted 3 lines
I dunno, perhaps revert 077706165717 ("virtio/vsock: don't use skbuff state to account credit")
and find a better fix then?IIRC the same issue was there before the commit fixed by that one (commit
71dc9ec9ac7d("virtio/vsock: replacevirtio_vsock_pktwithsk_buff")), so not sure about reverting it TBH.CCing Arseniy and Bobby.
Quoted 7 lines
There is always a discrepancy between
skb->lenandskb->truesize. You will not be able to announce a 1MB window, and accept one milliion skb of 1-byte each.This kind of contract is broken.
Yep, I agree, but before we start discarding data (and losing it), IMHO we should at least inform the other peer that we're out of space.
Quoted 8 lines
@Stefan, @Michael, do you think we can do something in the spec to avoid this issue and in some way take into account also the metadata in the credit. I mean to avoid the 1-byte packets flooding.
Thanks,
Stefano
Indeed the old pre-fix skb code would have the same issue.
I can't think of any way around this without extending the spec.
Best,
Bobby
Quoted 62 lines · Bobby Eshleman
Quoted 61 lines · Stefano Garzarella
Quoted 59 lines · Eric Dumazet
Quoted 48 lines · Stefano Garzarella
Quoted 33 lines · Eric Dumazet
virtio_transport_inc_rx_pkt()checksvvs->rx_bytes+ len >vvs->buf_alloc.
virtio_transport_recv_enqueue()skips coalescing for packets withVIRTIO_VSOCK_SEQ_EOM.If fed with packets with len == 0 and
VIRTIO_VSOCK_SEQ_EOM, a very large number of packets can be queued becausevvs->rx_bytesstays at 0.Fix this by estimating the skb metadata size:
(Number of skbs in the queue) *
SKB_TRUESIZE(0)Fixes 077706165717 ("virtio/vsock: don't use skbuff state to account credit")Signed-off-by Eric Dumazet <[email protected]>Cc Arseniy Krasnov <[email protected]>Cc Stefan Hajnoczi <[email protected]>Cc Stefano Garzarella <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Xuan Zhuo <[email protected]>Cc "Eugenio Pérez" <[email protected]>---
net/vmw_vsock/virtio_transport_common.c| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)net/vmw_vsock/virtio_transport_common.c +0
-0net/vmw_vsock/virtio_transport_common.c static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, · line 453
Quoted 8 lines
net/vmw_vsock/virtio_transport_common.c +3
-1@@ -447,7 +447,9 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,static bool virtio_transport_inc_rx_pkt (struct virtio_vsock_sock *vvs,u32 len) { -if (vvs->buf_used + len > vvs->buf_alloc) +u64 skb_overhead = (skb_queue_len (&vvs->rx_queue) +1 ) *SKB_TRUESIZE (0 ); + +if (skb_overhead + vvs->buf_used + len > vvs->buf_alloc)return false;I'm not sure about this fix, I mean that maybe this is incomplete. In virtio-vsock, there is a credit mechanism between the two peers:
https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/virtio-v1.3-csd01.html#x1-4850003Quoted 6 lines
This takes only the payload into account, so it’s true that this problem exists; however, perhaps we should also inform the other peer of a lower credit balance, otherwise the other peer will believe it has much more credit than it actually does, send a large payload, and then the packet will be discarded and the data lost (there are no retransmissions, etc.).
I dunno, perhaps revert 077706165717 ("virtio/vsock: don't use skbuff state to account credit")
and find a better fix then?
Quoted 5 lines
Quoted 5 lines
IIRC the same issue was there before the commit fixed by that one (commit
71dc9ec9ac7d("virtio/vsock: replacevirtio_vsock_pktwithsk_buff")), so not sure about reverting it TBH.CCing Arseniy and Bobby.
Thanks!
Quoted 16 lines
Quoted 16 lines
Quoted 8 lines
There is always a discrepancy between
skb->lenandskb->truesize. You will not be able to announce a 1MB window, and accept one milliion skb of 1-byte each.This kind of contract is broken.
Yep, I agree, but before we start discarding data (and losing it), IMHO we should at least inform the other peer that we're out of space.
@Stefan, @Michael, do you think we can do something in the spec to avoid this issue and in some way take into account also the metadata in the credit. I mean to avoid the 1-byte packets flooding.
Quoted 8 lines
Quoted 4 lines
Thanks,
StefanoIndeed the old pre-fix skb code would have the same issue.
I can't think of any way around this without extending the spec.
Hi, thanks, agree with Bobby, that accounting metadata (e.g. skb size here) was not implemented "by design" in credit logic - another side of data exchange knows nothing about that. Also the same situation was before skb implementation was added by Bobby. So looks like need to update spec may be.
Thanks!
Quoted 3 lines
Best,
Bobby
Stefano Garzarella
maintainerQuoted 94 lines · Arseniy Krasnov
Quoted 62 lines · Bobby Eshleman
Quoted 61 lines · Stefano Garzarella
Quoted 59 lines · Eric Dumazet
Quoted 48 lines · Stefano Garzarella
Quoted 33 lines · Eric Dumazet
virtio_transport_inc_rx_pkt()checksvvs->rx_bytes+ len >vvs->buf_alloc.
virtio_transport_recv_enqueue()skips coalescing for packets withVIRTIO_VSOCK_SEQ_EOM.If fed with packets with len == 0 and
VIRTIO_VSOCK_SEQ_EOM, a very large number of packets can be queued becausevvs->rx_bytesstays at 0.Fix this by estimating the skb metadata size:
(Number of skbs in the queue) *
SKB_TRUESIZE(0)Fixes 077706165717 ("virtio/vsock: don't use skbuff state to account credit")Signed-off-by Eric Dumazet <[email protected]>Cc Arseniy Krasnov <[email protected]>Cc Stefan Hajnoczi <[email protected]>Cc Stefano Garzarella <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Xuan Zhuo <[email protected]>Cc "Eugenio Pérez" <[email protected]>---
net/vmw_vsock/virtio_transport_common.c| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)net/vmw_vsock/virtio_transport_common.c +0
-0net/vmw_vsock/virtio_transport_common.c static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, · line 453
Quoted 8 lines
net/vmw_vsock/virtio_transport_common.c +3
-1@@ -447,7 +447,9 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,static bool virtio_transport_inc_rx_pkt (struct virtio_vsock_sock *vvs,u32 len) { -if (vvs->buf_used + len > vvs->buf_alloc) +u64 skb_overhead = (skb_queue_len (&vvs->rx_queue) +1 ) *SKB_TRUESIZE (0 ); + +if (skb_overhead + vvs->buf_used + len > vvs->buf_alloc)return false;I'm not sure about this fix, I mean that maybe this is incomplete. In virtio-vsock, there is a credit mechanism between the two peers:
https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/virtio-v1.3-csd01.html#x1-4850003Quoted 6 lines
This takes only the payload into account, so it’s true that this problem exists; however, perhaps we should also inform the other peer of a lower credit balance, otherwise the other peer will believe it has much more credit than it actually does, send a large payload, and then the packet will be discarded and the data lost (there are no retransmissions, etc.).
I dunno, perhaps revert 077706165717 ("virtio/vsock: don't use skbuff state to account credit")
and find a better fix then?Quoted 5 lines
Quoted 5 lines
IIRC the same issue was there before the commit fixed by that one (commit
71dc9ec9ac7d("virtio/vsock: replacevirtio_vsock_pktwithsk_buff")), so not sure about reverting it TBH.CCing Arseniy and Bobby.
Thanks!
Quoted 21 lines
Quoted 20 lines
Quoted 8 lines
There is always a discrepancy between
skb->lenandskb->truesize. You will not be able to announce a 1MB window, and accept one milliion skb of 1-byte each.This kind of contract is broken.
Yep, I agree, but before we start discarding data (and losing it), IMHO we should at least inform the other peer that we're out of space.
@Stefan, @Michael, do you think we can do something in the spec to avoid this issue and in some way take into account also the metadata in the credit. I mean to avoid the 1-byte packets flooding.
Thanks,
Stefano
Quoted 8 lines
Quoted 3 lines
Indeed the old pre-fix skb code would have the same issue.
I can't think of any way around this without extending the spec.
Hi, thanks, agree with Bobby, that accounting metadata (e.g. skb size here) was not implemented "by design" in credit logic - another side of data exchange knows nothing about that. Also the same situation was before skb implementation was added by Bobby. So looks like need to update spec may be.
Even if we change the specifications, we still need to work with older devices, so we should find a solution for this as well.
My main concern is data loss, so I'm considering the following options:
1. Notify the other peer of a smaller buf_alloc from the start, leave some room for overhead, and when it's running out, notify them that buf_alloc = 0. This way, the peer realizes it can’t send anything else.
2. Or update buf_alloc each time by removing the overhead, similar to what’s currently done in virtio_transport_inc_rx_pkt(), but also do it in virtio_transport_inc_tx_pkt().
As I said, IMO this patch alone is incomplete; we need to communicate with the peer somehow regarding space. I don’t think including the overhead in fwd_cnt is spec compliant, since the other peer has no idea how much overhead is needed, but reducing buf_alloc should be okay, even though I’m concerned about packets in flight.
As a quick fix, I think option 2 might be the easiest; I’ll run some tests and send over a patch.
But in the long run, I think we absolutely need to improve memory management in vsock, perhaps by avoiding custom solutions.
Thanks,
Stefano
Michael S. Tsirkin
maintainerQuoted 78 lines · Stefano Garzarella
Quoted 56 lines · Eric Dumazet
Quoted 48 lines · Stefano Garzarella
Quoted 33 lines · Eric Dumazet
virtio_transport_inc_rx_pkt()checksvvs->rx_bytes+ len >vvs->buf_alloc.
virtio_transport_recv_enqueue()skips coalescing for packets withVIRTIO_VSOCK_SEQ_EOM.If fed with packets with len == 0 and
VIRTIO_VSOCK_SEQ_EOM, a very large number of packets can be queued becausevvs->rx_bytesstays at 0.Fix this by estimating the skb metadata size:
(Number of skbs in the queue) *
SKB_TRUESIZE(0)Fixes 077706165717 ("virtio/vsock: don't use skbuff state to account credit")Signed-off-by Eric Dumazet <[email protected]>Cc Arseniy Krasnov <[email protected]>Cc Stefan Hajnoczi <[email protected]>Cc Stefano Garzarella <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Xuan Zhuo <[email protected]>Cc "Eugenio Pérez" <[email protected]>---
net/vmw_vsock/virtio_transport_common.c| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)net/vmw_vsock/virtio_transport_common.c +0
-0net/vmw_vsock/virtio_transport_common.c static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, · line 453
Quoted 8 lines
net/vmw_vsock/virtio_transport_common.c +3
-1@@ -447,7 +447,9 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,static bool virtio_transport_inc_rx_pkt (struct virtio_vsock_sock *vvs,u32 len) { -if (vvs->buf_used + len > vvs->buf_alloc) +u64 skb_overhead = (skb_queue_len (&vvs->rx_queue) +1 ) *SKB_TRUESIZE (0 ); + +if (skb_overhead + vvs->buf_used + len > vvs->buf_alloc)return false;I'm not sure about this fix, I mean that maybe this is incomplete. In virtio-vsock, there is a credit mechanism between the two peers:
https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/virtio-v1.3-csd01.html#x1-4850003Quoted 6 lines
This takes only the payload into account, so it’s true that this problem exists; however, perhaps we should also inform the other peer of a lower credit balance, otherwise the other peer will believe it has much more credit than it actually does, send a large payload, and then the packet will be discarded and the data lost (there are no retransmissions, etc.).
Quoted 3 lines
I dunno, perhaps revert 077706165717 ("virtio/vsock: don't use skbuff state to account credit")
and find a better fix then?IIRC the same issue was there before the commit fixed by that one (commit
71dc9ec9ac7d("virtio/vsock: replacevirtio_vsock_pktwithsk_buff")), so not sure about reverting it TBH.CCing Arseniy and Bobby.
Quoted 7 lines
There is always a discrepancy between
skb->lenandskb->truesize. You will not be able to announce a 1MB window, and accept one milliion skb of 1-byte each.This kind of contract is broken.
Yep, I agree, but before we start discarding data (and losing it), IMHO we should at least inform the other peer that we're out of space.
Quoted 6 lines
@Stefan, @Michael, do you think we can do something in the spec to avoid this issue and in some way take into account also the metadata in the credit. I mean to avoid the 1-byte packets flooding.
Thanks,
Stefano
Why do we need the metadata? Just don't keep it around if you begin running low on memory.
Signature · 1 line
MST
Stefano Garzarella
maintainerQuoted 83 lines · Michael S. Tsirkin
Quoted 82 lines · Stefano Garzarella
Quoted 56 lines · Eric Dumazet
Quoted 48 lines · Stefano Garzarella
Quoted 33 lines · Eric Dumazet
virtio_transport_inc_rx_pkt()checksvvs->rx_bytes+ len >vvs->buf_alloc.
virtio_transport_recv_enqueue()skips coalescing for packets withVIRTIO_VSOCK_SEQ_EOM.If fed with packets with len == 0 and
VIRTIO_VSOCK_SEQ_EOM, a very large number of packets can be queued becausevvs->rx_bytesstays at 0.Fix this by estimating the skb metadata size:
(Number of skbs in the queue) *
SKB_TRUESIZE(0)Fixes 077706165717 ("virtio/vsock: don't use skbuff state to account credit")Signed-off-by Eric Dumazet <[email protected]>Cc Arseniy Krasnov <[email protected]>Cc Stefan Hajnoczi <[email protected]>Cc Stefano Garzarella <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Xuan Zhuo <[email protected]>Cc "Eugenio Pérez" <[email protected]>---
net/vmw_vsock/virtio_transport_common.c| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)net/vmw_vsock/virtio_transport_common.c +0
-0net/vmw_vsock/virtio_transport_common.c static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, · line 453
Quoted 8 lines
net/vmw_vsock/virtio_transport_common.c +3
-1@@ -447,7 +447,9 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,static bool virtio_transport_inc_rx_pkt (struct virtio_vsock_sock *vvs,u32 len) { -if (vvs->buf_used + len > vvs->buf_alloc) +u64 skb_overhead = (skb_queue_len (&vvs->rx_queue) +1 ) *SKB_TRUESIZE (0 ); + +if (skb_overhead + vvs->buf_used + len > vvs->buf_alloc)return false;I'm not sure about this fix, I mean that maybe this is incomplete. In virtio-vsock, there is a credit mechanism between the two peers:
https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/virtio-v1.3-csd01.html#x1-4850003Quoted 6 lines
This takes only the payload into account, so it’s true that this problem exists; however, perhaps we should also inform the other peer of a lower credit balance, otherwise the other peer will believe it has much more credit than it actually does, send a large payload, and then the packet will be discarded and the data lost (there are no retransmissions, etc.).
Quoted 3 lines
I dunno, perhaps revert 077706165717 ("virtio/vsock: don't use skbuff state to account credit")
and find a better fix then?IIRC the same issue was there before the commit fixed by that one (commit
71dc9ec9ac7d("virtio/vsock: replacevirtio_vsock_pktwithsk_buff")), so not sure about reverting it TBH.CCing Arseniy and Bobby.
Quoted 7 lines
There is always a discrepancy between
skb->lenandskb->truesize. You will not be able to announce a 1MB window, and accept one milliion skb of 1-byte each.This kind of contract is broken.
Yep, I agree, but before we start discarding data (and losing it), IMHO we should at least inform the other peer that we're out of space.
@Stefan, @Michael, do you think we can do something in the spec to avoid this issue and in some way take into account also the metadata in the credit. I mean to avoid the 1-byte packets flooding.
Quoted 5 lines
Quoted 2 lines
Thanks,
StefanoWhy do we need the metadata? Just don't keep it around if you begin running low on memory.
I don't think removing the skuffs will be easy; we added them for ebpf, zero-copy, and seqpacket as well. For now, we're already doing something: merging the skuffs if they don't have EOM set.
As a quick fix, I'm thinking of reducing the buf_alloc value to account for the overhead and notifying the other peer, at least until we find a better solution.
Stefano
Michael S. Tsirkin
maintainerQuoted 84 lines · Stefano Garzarella
Quoted 83 lines · Michael S. Tsirkin
Quoted 82 lines · Stefano Garzarella
Quoted 56 lines · Eric Dumazet
Quoted 48 lines · Stefano Garzarella
Quoted 33 lines · Eric Dumazet
virtio_transport_inc_rx_pkt()checksvvs->rx_bytes+ len >vvs->buf_alloc.
virtio_transport_recv_enqueue()skips coalescing for packets withVIRTIO_VSOCK_SEQ_EOM.If fed with packets with len == 0 and
VIRTIO_VSOCK_SEQ_EOM, a very large number of packets can be queued becausevvs->rx_bytesstays at 0.Fix this by estimating the skb metadata size:
(Number of skbs in the queue) *
SKB_TRUESIZE(0)Fixes 077706165717 ("virtio/vsock: don't use skbuff state to account credit")Signed-off-by Eric Dumazet <[email protected]>Cc Arseniy Krasnov <[email protected]>Cc Stefan Hajnoczi <[email protected]>Cc Stefano Garzarella <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Xuan Zhuo <[email protected]>Cc "Eugenio Pérez" <[email protected]>---
net/vmw_vsock/virtio_transport_common.c| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)net/vmw_vsock/virtio_transport_common.c +0
-0net/vmw_vsock/virtio_transport_common.c static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, · line 453
Quoted 8 lines
net/vmw_vsock/virtio_transport_common.c +3
-1@@ -447,7 +447,9 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,static bool virtio_transport_inc_rx_pkt (struct virtio_vsock_sock *vvs,u32 len) { -if (vvs->buf_used + len > vvs->buf_alloc) +u64 skb_overhead = (skb_queue_len (&vvs->rx_queue) +1 ) *SKB_TRUESIZE (0 ); + +if (skb_overhead + vvs->buf_used + len > vvs->buf_alloc)return false;I'm not sure about this fix, I mean that maybe this is incomplete. In virtio-vsock, there is a credit mechanism between the two peers:
https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/virtio-v1.3-csd01.html#x1-4850003Quoted 6 lines
This takes only the payload into account, so it’s true that this problem exists; however, perhaps we should also inform the other peer of a lower credit balance, otherwise the other peer will believe it has much more credit than it actually does, send a large payload, and then the packet will be discarded and the data lost (there are no retransmissions, etc.).
Quoted 3 lines
I dunno, perhaps revert 077706165717 ("virtio/vsock: don't use skbuff state to account credit")
and find a better fix then?IIRC the same issue was there before the commit fixed by that one (commit
71dc9ec9ac7d("virtio/vsock: replacevirtio_vsock_pktwithsk_buff")), so not sure about reverting it TBH.CCing Arseniy and Bobby.
Quoted 7 lines
There is always a discrepancy between
skb->lenandskb->truesize. You will not be able to announce a 1MB window, and accept one milliion skb of 1-byte each.This kind of contract is broken.
Yep, I agree, but before we start discarding data (and losing it), IMHO we should at least inform the other peer that we're out of space.
@Stefan, @Michael, do you think we can do something in the spec to avoid this issue and in some way take into account also the metadata in the credit. I mean to avoid the 1-byte packets flooding.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Thanks,
StefanoWhy do we need the metadata? Just don't keep it around if you begin running low on memory.
I don't think removing the skuffs will be easy; we added them for ebpf, zero-copy, and seqpacket as well.
You do not need to remove them completely.
Quoted 2 lines
For now, we're already doing something:
merging the skuffs if they don't have EOM set.
Right that's good. You could go further and merge with EOM too if you stick the info about message boundaries somewhere else.
Quoted 5 lines
As a quick fix, I'm thinking of reducing the
buf_allocvalue to account for the overhead and notifying the other peer, at least until we find a better solution.Stefano
well if you want to support pathological cases such as 1 byte messages that would mean like 100x reduction no?
Stefano Garzarella
maintainerQuoted 94 lines · Michael S. Tsirkin
Quoted 84 lines · Stefano Garzarella
Quoted 83 lines · Michael S. Tsirkin
Quoted 82 lines · Stefano Garzarella
Quoted 56 lines · Eric Dumazet
Quoted 48 lines · Stefano Garzarella
Quoted 33 lines · Eric Dumazet
virtio_transport_inc_rx_pkt()checksvvs->rx_bytes+ len >vvs->buf_alloc.
virtio_transport_recv_enqueue()skips coalescing for packets withVIRTIO_VSOCK_SEQ_EOM.If fed with packets with len == 0 and
VIRTIO_VSOCK_SEQ_EOM, a very large number of packets can be queued becausevvs->rx_bytesstays at 0.Fix this by estimating the skb metadata size:
(Number of skbs in the queue) *
SKB_TRUESIZE(0)Fixes 077706165717 ("virtio/vsock: don't use skbuff state to account credit")Signed-off-by Eric Dumazet <[email protected]>Cc Arseniy Krasnov <[email protected]>Cc Stefan Hajnoczi <[email protected]>Cc Stefano Garzarella <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Xuan Zhuo <[email protected]>Cc "Eugenio Pérez" <[email protected]>---
net/vmw_vsock/virtio_transport_common.c| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)net/vmw_vsock/virtio_transport_common.c +0
-0net/vmw_vsock/virtio_transport_common.c static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, · line 453
Quoted 8 lines
net/vmw_vsock/virtio_transport_common.c +3
-1@@ -447,7 +447,9 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,static bool virtio_transport_inc_rx_pkt (struct virtio_vsock_sock *vvs,u32 len) { -if (vvs->buf_used + len > vvs->buf_alloc) +u64 skb_overhead = (skb_queue_len (&vvs->rx_queue) +1 ) *SKB_TRUESIZE (0 ); + +if (skb_overhead + vvs->buf_used + len > vvs->buf_alloc)return false;I'm not sure about this fix, I mean that maybe this is incomplete. In virtio-vsock, there is a credit mechanism between the two peers:
https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/virtio-v1.3-csd01.html#x1-4850003Quoted 6 lines
This takes only the payload into account, so it’s true that this problem exists; however, perhaps we should also inform the other peer of a lower credit balance, otherwise the other peer will believe it has much more credit than it actually does, send a large payload, and then the packet will be discarded and the data lost (there are no retransmissions, etc.).
Quoted 3 lines
I dunno, perhaps revert 077706165717 ("virtio/vsock: don't use skbuff state to account credit")
and find a better fix then?IIRC the same issue was there before the commit fixed by that one (commit
71dc9ec9ac7d("virtio/vsock: replacevirtio_vsock_pktwithsk_buff")), so not sure about reverting it TBH.CCing Arseniy and Bobby.
Quoted 7 lines
There is always a discrepancy between
skb->lenandskb->truesize. You will not be able to announce a 1MB window, and accept one milliion skb of 1-byte each.This kind of contract is broken.
Yep, I agree, but before we start discarding data (and losing it), IMHO we should at least inform the other peer that we're out of space.
@Stefan, @Michael, do you think we can do something in the spec to avoid this issue and in some way take into account also the metadata in the credit. I mean to avoid the 1-byte packets flooding.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Thanks,
StefanoWhy do we need the metadata? Just don't keep it around if you begin running low on memory.
I don't think removing the skuffs will be easy; we added them for ebpf, zero-copy, and seqpacket as well.
Quoted 8 lines
You do not need to remove them completely.
Quoted 2 lines
For now, we're already doing something:
merging the skuffs if they don't have EOM set.Right that's good. You could go further and merge with EOM too if you stick the info about message boundaries somewhere else.
This adds a lot of complexity IMO, but we can try.
Do you have something in mind?
Quoted 10 lines
Quoted 6 lines
As a quick fix, I'm thinking of reducing the
buf_allocvalue to account for the overhead and notifying the other peer, at least until we find a better solution.Stefano
well if you want to support pathological cases such as 1 byte messages that would mean like 100x reduction no?
Yep, but since this patch is already merged, IMHO that is better than losing data in those pathological cases.
Thanks,
Stefano
Jakub Kicinski
maintainerQuoted 5 lines · On Thu, 7 May 2026 14:59:13 +0200 Stefano Garzarella wrote:
Quoted 2 lines
well if you want to support pathological cases such as 1 byte messages that would mean like 100x reduction no?
Yep, but since this patch is already merged, IMHO that is better than losing data in those pathological cases.
We can revert if you think that the risk of regression is high.. Please LMK soon, we can do it before patch reaches Linus.
Stefano Garzarella
maintainerQuoted 10 lines · Jakub Kicinski
Quoted 5 lines · On Thu, 7 May 2026 14:59:13 +0200 Stefano Garzarella wrote:
Quoted 2 lines
well if you want to support pathological cases such as 1 byte messages that would mean like 100x reduction no?
Yep, but since this patch is already merged, IMHO that is better than losing data in those pathological cases.
We can revert if you think that the risk of regression is high.. Please LMK soon, we can do it before patch reaches Linus.
Some tests in tools/testing/vsock/vsock_test.c are failing with this patch applied.
Test 18 are failing sometime in this way (I guess because we are dropping packets):
18 - SOCK_STREAM MSG_ZEROCOPY...hash mismatch
Test 22 is failing 100% in this way:
22 - SOCK_STREAM virtio credit update + SO_RCVLOWAT...send failed: Resource temporarily unavailable
With my followup patch adding also advertisement to the other peer (still draft locally, waiting for Michael proposal) I saw 22 failing, because tests expects that can use the entire buf_alloc, but now we are reducing it. So IMO we should do like in __sock_set_rcvbuf() and double the buffer size, or at least digest an overhead equal to the buffer size set by the user via SO_VM_SOCKETS_BUFFER_SIZE (yeah, AF_VSOCK has it owns sockopt since the beginning :-().
With that approach tests are passing, but I'd like to stress a bit more that patch. I'll send it tomorrow as fixup of this patch, or if you prefer to revert, I'll send as standalone.
Thanks,
Stefano
Quoted 35 lines · Stefano Garzarella
Quoted 10 lines · Jakub Kicinski
Quoted 5 lines · On Thu, 7 May 2026 14:59:13 +0200 Stefano Garzarella wrote:
Quoted 2 lines
well if you want to support pathological cases such as 1 byte messages that would mean like 100x reduction no?
Yep, but since this patch is already merged, IMHO that is better than losing data in those pathological cases.
We can revert if you think that the risk of regression is high.. Please LMK soon, we can do it before patch reaches Linus.
Some tests in
tools/testing/vsock/vsock_test.care failing with this patch applied.Test 18 are failing sometime in this way (I guess because we are dropping packets):
18 -
SOCK_STREAMMSG_ZEROCOPY...hash mismatchTest 22 is failing 100% in this way:
22 -
SOCK_STREAMvirtio credit update + SO_RCVLOWAT...send failed:
Resource temporarily unavailableWith my followup patch adding also advertisement to the other peer (still draft locally, waiting for Michael proposal) I saw 22 failing, because tests expects that can use the entire
buf_alloc, but now we are reducing it. So IMO we should do like in__sock_set_rcvbuf()and double the buffer size, or at least digest an overhead equal to the buffer size set by the user viaSO_VM_SOCKETS_BUFFER_SIZE(yeah,AF_VSOCKhas it owns sockopt since the beginning :-().
Quoted 4 lines
With that approach tests are passing, but I'd like to stress a bit more that patch. I'll send it tomorrow as fixup of this patch, or if you prefer to revert, I'll send as standalone.
A plain revert is a big issue, now users now how to crash hypervisors.
This vulnerability allows a compromised guest (controlling virtio_vsock_hdr fields)
to continuously flood the host's vsock receive queue without triggering any memory
accounting limits or reader wakeups, resulting in unbounded host
kernel memory consumption (Host DoS via OOM).
A vulnerability where a KVM guest can crash or deadlock its host is classified as a KVM DoS.
Am I missing something?
Jakub Kicinski
maintainerQuoted 40 lines · On Thu, 7 May 2026 09:32:24 -0700 Eric Dumazet wrote:
Quoted 26 lines · Stefano Garzarella
Quoted 2 lines · Jakub Kicinski
We can revert if you think that the risk of regression is high.. Please LMK soon, we can do it before patch reaches Linus.
Some tests in
tools/testing/vsock/vsock_test.care failing with this patch applied.Test 18 are failing sometime in this way (I guess because we are dropping packets):
18 -
SOCK_STREAMMSG_ZEROCOPY...hash mismatchTest 22 is failing 100% in this way:
22 -
SOCK_STREAMvirtio credit update + SO_RCVLOWAT...send failed:
Resource temporarily unavailableWith my followup patch adding also advertisement to the other peer (still draft locally, waiting for Michael proposal) I saw 22 failing, because tests expects that can use the entire
buf_alloc, but now we are reducing it. So IMO we should do like in__sock_set_rcvbuf()and double the buffer size, or at least digest an overhead equal to the buffer size set by the user viaSO_VM_SOCKETS_BUFFER_SIZE(yeah,AF_VSOCKhas it owns sockopt since the beginning :-().Quoted 3 lines
With that approach tests are passing, but I'd like to stress a bit more that patch. I'll send it tomorrow as fixup of this patch, or if you prefer to revert, I'll send as standalone.
A plain revert is a big issue, now users now how to crash hypervisors.
This vulnerability allows a compromised guest (controlling
virtio_vsock_hdrfields)
to continuously flood the host's vsock receive queue without triggering any memory
accounting limits or reader wakeups, resulting in unbounded host
kernel memory consumption (Host DoS via OOM).
Quoted 4 lines
A vulnerability where a KVM guest can crash or deadlock its host is classified as a KVM DoS.
Am I missing something?
Alright, let's leave it.
Stefano Garzarella
maintainerQuoted 41 lines · Jakub Kicinski
Quoted 40 lines · On Thu, 7 May 2026 09:32:24 -0700 Eric Dumazet wrote:
Quoted 26 lines · Stefano Garzarella
Quoted 2 lines · Jakub Kicinski
We can revert if you think that the risk of regression is high.. Please LMK soon, we can do it before patch reaches Linus.
Some tests in
tools/testing/vsock/vsock_test.care failing with this patch applied.Test 18 are failing sometime in this way (I guess because we are dropping packets):
18 -
SOCK_STREAMMSG_ZEROCOPY...hash mismatchTest 22 is failing 100% in this way:
22 -
SOCK_STREAMvirtio credit update + SO_RCVLOWAT...send failed:
Resource temporarily unavailableWith my followup patch adding also advertisement to the other peer (still draft locally, waiting for Michael proposal) I saw 22 failing, because tests expects that can use the entire
buf_alloc, but now we are reducing it. So IMO we should do like in__sock_set_rcvbuf()and double the buffer size, or at least digest an overhead equal to the buffer size set by the user viaSO_VM_SOCKETS_BUFFER_SIZE(yeah,AF_VSOCKhas it owns sockopt since the beginning :-().Quoted 3 lines
With that approach tests are passing, but I'd like to stress a bit more that patch. I'll send it tomorrow as fixup of this patch, or if you prefer to revert, I'll send as standalone.
A plain revert is a big issue, now users now how to crash hypervisors.
This vulnerability allows a compromised guest (controlling
virtio_vsock_hdrfields)
to continuously flood the host's vsock receive queue without triggering any memory
accounting limits or reader wakeups, resulting in unbounded host
kernel memory consumption (Host DoS via OOM).
Quoted 7 lines
Quoted 4 lines
A vulnerability where a KVM guest can crash or deadlock its host is classified as a KVM DoS.
Am I missing something?
Alright, let's leave it.
I posted a potential fixup: https://lore.kernel.org/netdev/[email protected]/
Thanks,
Stefano
Michael S. Tsirkin
maintainerQuoted 101 lines · Stefano Garzarella
Quoted 100 lines · Michael S. Tsirkin
Quoted 84 lines · Stefano Garzarella
Quoted 83 lines · Michael S. Tsirkin
Quoted 82 lines · Stefano Garzarella
Quoted 56 lines · Eric Dumazet
Quoted 48 lines · Stefano Garzarella
Quoted 33 lines · Eric Dumazet
virtio_transport_inc_rx_pkt()checksvvs->rx_bytes+ len >vvs->buf_alloc.
virtio_transport_recv_enqueue()skips coalescing for packets withVIRTIO_VSOCK_SEQ_EOM.If fed with packets with len == 0 and
VIRTIO_VSOCK_SEQ_EOM, a very large number of packets can be queued becausevvs->rx_bytesstays at 0.Fix this by estimating the skb metadata size:
(Number of skbs in the queue) *
SKB_TRUESIZE(0)Fixes 077706165717 ("virtio/vsock: don't use skbuff state to account credit")Signed-off-by Eric Dumazet <[email protected]>Cc Arseniy Krasnov <[email protected]>Cc Stefan Hajnoczi <[email protected]>Cc Stefano Garzarella <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Xuan Zhuo <[email protected]>Cc "Eugenio Pérez" <[email protected]>---
net/vmw_vsock/virtio_transport_common.c| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)net/vmw_vsock/virtio_transport_common.c +0
-0net/vmw_vsock/virtio_transport_common.c static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, · line 453
Quoted 8 lines
net/vmw_vsock/virtio_transport_common.c +3
-1@@ -447,7 +447,9 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,static bool virtio_transport_inc_rx_pkt (struct virtio_vsock_sock *vvs,u32 len) { -if (vvs->buf_used + len > vvs->buf_alloc) +u64 skb_overhead = (skb_queue_len (&vvs->rx_queue) +1 ) *SKB_TRUESIZE (0 ); + +if (skb_overhead + vvs->buf_used + len > vvs->buf_alloc)return false;I'm not sure about this fix, I mean that maybe this is incomplete. In virtio-vsock, there is a credit mechanism between the two peers:
https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/virtio-v1.3-csd01.html#x1-4850003Quoted 6 lines
This takes only the payload into account, so it’s true that this problem exists; however, perhaps we should also inform the other peer of a lower credit balance, otherwise the other peer will believe it has much more credit than it actually does, send a large payload, and then the packet will be discarded and the data lost (there are no retransmissions, etc.).
Quoted 3 lines
I dunno, perhaps revert 077706165717 ("virtio/vsock: don't use skbuff state to account credit")
and find a better fix then?IIRC the same issue was there before the commit fixed by that one (commit
71dc9ec9ac7d("virtio/vsock: replacevirtio_vsock_pktwithsk_buff")), so not sure about reverting it TBH.CCing Arseniy and Bobby.
Quoted 7 lines
There is always a discrepancy between
skb->lenandskb->truesize. You will not be able to announce a 1MB window, and accept one milliion skb of 1-byte each.This kind of contract is broken.
Yep, I agree, but before we start discarding data (and losing it), IMHO we should at least inform the other peer that we're out of space.
@Stefan, @Michael, do you think we can do something in the spec to avoid this issue and in some way take into account also the metadata in the credit. I mean to avoid the 1-byte packets flooding.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Thanks,
StefanoWhy do we need the metadata? Just don't keep it around if you begin running low on memory.
I don't think removing the skuffs will be easy; we added them for ebpf, zero-copy, and seqpacket as well.
You do not need to remove them completely.
Quoted 2 lines
For now, we're already doing something:
merging the skuffs if they don't have EOM set.
Quoted 6 lines
Quoted 2 lines
Right that's good. You could go further and merge with EOM too if you stick the info about message boundaries somewhere else.
This adds a lot of complexity IMO, but we can try.
Do you have something in mind?
I'll send something shortly just to give you an idea.
Quoted 16 lines
Quoted 10 lines
Quoted 6 lines
As a quick fix, I'm thinking of reducing the
buf_allocvalue to account for the overhead and notifying the other peer, at least until we find a better solution.Stefano
well if you want to support pathological cases such as 1 byte messages that would mean like 100x reduction no?
Yep, but since this patch is already merged, IMHO that is better than losing data in those pathological cases.
Thanks,
Stefano
Michael S. Tsirkin
maintainerQuoted 101 lines · Stefano Garzarella
Quoted 100 lines · Michael S. Tsirkin
Quoted 84 lines · Stefano Garzarella
Quoted 83 lines · Michael S. Tsirkin
Quoted 82 lines · Stefano Garzarella
Quoted 56 lines · Eric Dumazet
Quoted 48 lines · Stefano Garzarella
Quoted 33 lines · Eric Dumazet
virtio_transport_inc_rx_pkt()checksvvs->rx_bytes+ len >vvs->buf_alloc.
virtio_transport_recv_enqueue()skips coalescing for packets withVIRTIO_VSOCK_SEQ_EOM.If fed with packets with len == 0 and
VIRTIO_VSOCK_SEQ_EOM, a very large number of packets can be queued becausevvs->rx_bytesstays at 0.Fix this by estimating the skb metadata size:
(Number of skbs in the queue) *
SKB_TRUESIZE(0)Fixes 077706165717 ("virtio/vsock: don't use skbuff state to account credit")Signed-off-by Eric Dumazet <[email protected]>Cc Arseniy Krasnov <[email protected]>Cc Stefan Hajnoczi <[email protected]>Cc Stefano Garzarella <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Xuan Zhuo <[email protected]>Cc "Eugenio Pérez" <[email protected]>---
net/vmw_vsock/virtio_transport_common.c| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)net/vmw_vsock/virtio_transport_common.c +0
-0net/vmw_vsock/virtio_transport_common.c static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, · line 453
Quoted 8 lines
net/vmw_vsock/virtio_transport_common.c +3
-1@@ -447,7 +447,9 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,static bool virtio_transport_inc_rx_pkt (struct virtio_vsock_sock *vvs,u32 len) { -if (vvs->buf_used + len > vvs->buf_alloc) +u64 skb_overhead = (skb_queue_len (&vvs->rx_queue) +1 ) *SKB_TRUESIZE (0 ); + +if (skb_overhead + vvs->buf_used + len > vvs->buf_alloc)return false;I'm not sure about this fix, I mean that maybe this is incomplete. In virtio-vsock, there is a credit mechanism between the two peers:
https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/virtio-v1.3-csd01.html#x1-4850003Quoted 6 lines
This takes only the payload into account, so it’s true that this problem exists; however, perhaps we should also inform the other peer of a lower credit balance, otherwise the other peer will believe it has much more credit than it actually does, send a large payload, and then the packet will be discarded and the data lost (there are no retransmissions, etc.).
Quoted 3 lines
I dunno, perhaps revert 077706165717 ("virtio/vsock: don't use skbuff state to account credit")
and find a better fix then?IIRC the same issue was there before the commit fixed by that one (commit
71dc9ec9ac7d("virtio/vsock: replacevirtio_vsock_pktwithsk_buff")), so not sure about reverting it TBH.CCing Arseniy and Bobby.
Quoted 7 lines
There is always a discrepancy between
skb->lenandskb->truesize. You will not be able to announce a 1MB window, and accept one milliion skb of 1-byte each.This kind of contract is broken.
Yep, I agree, but before we start discarding data (and losing it), IMHO we should at least inform the other peer that we're out of space.
@Stefan, @Michael, do you think we can do something in the spec to avoid this issue and in some way take into account also the metadata in the credit. I mean to avoid the 1-byte packets flooding.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Thanks,
StefanoWhy do we need the metadata? Just don't keep it around if you begin running low on memory.
I don't think removing the skuffs will be easy; we added them for ebpf, zero-copy, and seqpacket as well.
You do not need to remove them completely.
Quoted 2 lines
For now, we're already doing something:
merging the skuffs if they don't have EOM set.
Quoted 6 lines
Quoted 2 lines
Right that's good. You could go further and merge with EOM too if you stick the info about message boundaries somewhere else.
This adds a lot of complexity IMO, but we can try.
Do you have something in mind?
BER is clearly overkill but here's a POC that claude made for me, just to give u an idea. It's clearly has a ton of issues, for example I dislike how GFP_ATOMIC is handled.
Yet it seems to work fine in light testing.
-->
vsock/virtio: use DWARF ULEB128 to record EOM boundaries, enable cross-EOM skb coalescing
virtio_transport_recv_enqueue() currently refuses to coalesce an incoming skb with the previous one when the previous skb carries VIRTIO_VSOCK_SEQ_EOM. This forces one skb per seqpacket message. For workloads with many small or zero-byte messages the per-skb overhead (~960 bytes) dominates, causing unbounded memory growth.
Decouple message boundary tracking from the skb structure: store boundary offsets in a compact side buffer using DWARF ULEB128 encoding with the EOR flag folded into the low bit, then allow the data of multiple complete messages to be coalesced into a single skb.
Cross-EOM coalescing fires only when:
- both the tail skb and the incoming packet carry EOM (complete msgs)
- the incoming packet fits in the tail skb's tailroom
- no BPF psock is attached (read_skb expects one msg per skb)
On allocation failure the code falls back to separate skbs (existing behaviour). Credit accounting is unchanged; the boundary buffer is capped at PAGE_SIZE.
include/linux/virtio_vsock.h +7 -0
@@ -12,6 +12,7 @@struct virtio_vsock_skb_cb {bool reply;bool tap_delivered; +bool has_boundary_entries;u32 offset; };
@@ -167,6 +168,12 @@ struct virtio_vsock_sock {u32 buf_used;struct sk_buff_head rx_queue;u32 msg_count; + +/* ULEB128-encoded seqpacket message boundary buffer */ +u8 *boundary_buf; +u32 boundary_len; +u32 boundary_alloc; +u32 boundary_off; };struct virtio_vsock_pkt_info {
net/vmw_vsock/virtio_transport_common.c +275 -63
420 lines not on this page · raw message
Stefano Garzarella
maintainerQuoted 3 lines · Michael S. Tsirkin
Quoted 2 lines · Stefano Garzarella
Quoted 1 line · Michael S. Tsirkin
On Thu, May 07, 2026 at 11:09:47AM +0200, Stefano Garzarella wrote:
[...]
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines
For now, we're already doing something:
merging the skuffs if they don't have EOM set.Right that's good. You could go further and merge with EOM too if you stick the info about message boundaries somewhere else.
Quoted 7 lines
Quoted 3 lines
This adds a lot of complexity IMO, but we can try.
Do you have something in mind?
BER is clearly overkill but here's a POC that claude made for me, just to give u an idea. It's clearly has a ton of issues, for example I dislike how
GFP_ATOMICis handled.
Okay, I somewhat understand, but clearly this isn't net material, so for now I think the best thing to do is to merge the fixup I sent (or something similar): https://lore.kernel.org/netdev/[email protected]/
This is a major change that should be merged with more caution. Could this have too much of an impact on performance?
Thanks,
Stefano
Quoted 488 lines
Yet it seems to work fine in light testing.
-->
vsock/virtio: use DWARF ULEB128 to record EOM boundaries, enable cross-EOM skb coalescing
virtio_transport_recv_enqueue()currently refuses to coalesce an incoming skb with the previous one when the previous skb carriesVIRTIO_VSOCK_SEQ_EOM. This forces one skb per seqpacket message. For workloads with many small or zero-byte messages the per-skb overhead (~960 bytes) dominates, causing unbounded memory growth.Decouple message boundary tracking from the skb structure: store boundary offsets in a compact side buffer using DWARF ULEB128 encoding with the EOR flag folded into the low bit, then allow the data of multiple complete messages to be coalesced into a single skb.
Cross-EOM coalescing fires only when:
- both the tail skb and the incoming packet carry EOM (complete msgs)
- the incoming packet fits in the tail skb's tailroom
- no BPF psock is attached (read_skbexpects one msg per skb)On allocation failure the code falls back to separate skbs (existing behaviour). Credit accounting is unchanged; the boundary buffer is capped at
PAGE_SIZE.Signed-off-by Michael S. Tsirkin <[email protected]>Co-Authored-By Claude Opus 4.6 (1M context) <[email protected]>include/linux/virtio_vsock.h +7
-0@@ -12,6 +12,7 @@struct virtio_vsock_skb_cb {bool reply;bool tap_delivered; +bool has_boundary_entries;u32 offset; };@@ -167,6 +168,12 @@ struct virtio_vsock_sock {u32 buf_used;struct sk_buff_head rx_queue;u32 msg_count; + +/* ULEB128-encoded seqpacket message boundary buffer */ +u8 *boundary_buf; +u32 boundary_len; +u32 boundary_alloc; +u32 boundary_off; };struct virtio_vsock_pkt_info {net/vmw_vsock/virtio_transport_common.c +275
-63@@ -11,6 +11,7 @@#include <linux/sched/signal.h> #include <linux/ctype.h> #include <linux/list.h> +#include <linux/skmsg.h> #include <linux/virtio_vsock.h> #include <uapi/linux/vsockmon.h> @@ -26,6 +27,91 @@/* Threshold for detecting small packets to copy */ #define GOOD_COPY_LEN 128 +#define VSOCK_BOUNDARY_BUF_INIT 64 +#define VSOCK_BOUNDARY_BUF_MAX PAGE_SIZE + +/* ULEB128 boundary encoding: value = (msg_len << 1) | eor. +* Each byte carries 7 data bits; bit 7 is set on all but the last byte. +* Max 5 bytes for a u32 msg_len (33 bits with eor shift). +*/ +static int vsock_uleb_encode_boundary (u8 *buf,u32 msg_len,bool eor) +{ +u64 val = ((u64 )msg_len <<1 ) | eor; +int n =0 ; + +do { + buf[n] = val &0x7f ; + val >>=7 ; +if (val) + buf[n] |=0x80 ; + n++; + }while (val); + +return n; +} + +static int vsock_uleb_decode_boundary (const u8 *buf,u32 avail, +u32 *msg_len,bool *eor) +{ +u64 val =0 ; +int shift =0 ; +int n =0 ; + +do { +if (n >= avail || shift >=35 ) +return -EINVAL ; + val |= (u64)(buf[n] &0x7f ) << shift; + shift +=7 ; + }while (buf[n++] &0x80 ); + + *eor = val &1 ; + *msg_len = val >>1 ; +return n; +} + +static void vsock_boundary_buf_compact (struct virtio_vsock_sock *vvs) +{ +if (vvs->boundary_off ==0 ) +return ; + + vvs->boundary_len -= vvs->boundary_off; +memmove (vvs->boundary_buf, vvs->boundary_buf + vvs->boundary_off, + vvs->boundary_len); + vvs->boundary_off =0 ; +} + +static int vsock_boundary_buf_ensure (struct virtio_vsock_sock *vvs,u32 needed) +{ +u32 new_alloc; +u8 *new_buf; + +if (vvs->boundary_alloc >= needed) +return 0 ; + +/* Reclaim consumed space before growing */ +if (vvs->boundary_off) { + needed -= vvs->boundary_off; +vsock_boundary_buf_compact (vvs); +if (vvs->boundary_alloc >= needed) +return 0 ; + } + + new_alloc =max (needed, vvs->boundary_alloc ? vvs->boundary_alloc *2 + :VSOCK_BOUNDARY_BUF_INIT ); +if (new_alloc >VSOCK_BOUNDARY_BUF_MAX ) + new_alloc =VSOCK_BOUNDARY_BUF_MAX ; +if (new_alloc < needed) +return -ENOMEM ; + + new_buf =krealloc (vvs->boundary_buf, new_alloc,GFP_ATOMIC ); +if (!new_buf) +return -ENOMEM ; + + vvs->boundary_buf = new_buf; + vvs->boundary_alloc = new_alloc; +return 0 ; +} +static void virtio_transport_cancel_close_work (struct vsock_sock *vsk,bool cancel_timeout);static s64 virtio_transport_has_space (struct virtio_vsock_sock *vvs);@@ -682,41 +768,74 @@ virtio_transport_seqpacket_do_peek(struct vsock_sock *vsk, total =0 ; len =msg_data_left (msg); - skb_queue_walk(&vvs->rx_queue, skb) { -struct virtio_vsock_hdr *hdr; + skb =skb_peek (&vvs->rx_queue); +if (skb &&VIRTIO_VSOCK_SKB_CB (skb)->has_boundary_entries) { +u32 msg_len, offset; +size_t bytes; +bool eor; +int ret; -if (total < len) { -size_t bytes; + ret =vsock_uleb_decode_boundary ( + vvs->boundary_buf + vvs->boundary_off, + vvs->boundary_len - vvs->boundary_off, + &msg_len, &eor); +if (ret <0 ) + goto unlock; + + offset =VIRTIO_VSOCK_SKB_CB (skb)->offset; + bytes =min (len, (size_t )msg_len); + +if (bytes) {int err; - bytes = len - total; -if (bytes > skb->len) - bytes = skb->len; -spin_unlock_bh (&vvs->rx_lock); - -/* sk_lock is held by caller so no one else can dequeue. -* Unlock rx_lock since skb_copy_datagram_iter() may sleep. -*/ - err =skb_copy_datagram_iter (skb,VIRTIO_VSOCK_SKB_CB (skb)->offset, + err =skb_copy_datagram_iter (skb, offset, &msg->msg_iter, bytes);if (err)return err; -spin_lock_bh (&vvs->rx_lock); } - total += skb->len; - hdr =virtio_vsock_hdr (skb); + total = msg_len; +if (eor) + msg->msg_flags |=MSG_EOR ; + }else { + skb_queue_walk(&vvs->rx_queue, skb) { +struct virtio_vsock_hdr *hdr; -if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ) { -if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOR ) - msg->msg_flags |=MSG_EOR ; +if (total < len) { +size_t bytes; +int err; -break ; + bytes = len - total; +if (bytes > skb->len) + bytes = skb->len; + +spin_unlock_bh (&vvs->rx_lock); + + err =skb_copy_datagram_iter ( + skb, +VIRTIO_VSOCK_SKB_CB (skb)->offset, + &msg->msg_iter, bytes); +if (err) +return err; + +spin_lock_bh (&vvs->rx_lock); + } + + total += skb->len; + hdr =virtio_vsock_hdr (skb); + +if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ) { +if (le32_to_cpu (hdr->flags) & +VIRTIO_VSOCK_SEQ_EOR ) + msg->msg_flags |=MSG_EOR ; +break ; + } } } +unlock:spin_unlock_bh (&vvs->rx_lock);return total;@@ -740,57 +859,105 @@ static int virtio_transport_seqpacket_do_dequeue(struct vsock_sock *vsk, }while (!msg_ready) { -struct virtio_vsock_hdr *hdr; -size_t pkt_len; - - skb =__skb_dequeue (&vvs->rx_queue); + skb =skb_peek (&vvs->rx_queue);if (!skb)break ; - hdr =virtio_vsock_hdr (skb); - pkt_len = (size_t )le32_to_cpu (hdr->len); -if (dequeued_len >=0 ) { +if (VIRTIO_VSOCK_SKB_CB (skb)->has_boundary_entries) {size_t bytes_to_copy; +u32 msg_len, offset; +bool eor; +int ret; - bytes_to_copy =min (user_buf_len, pkt_len); + ret =vsock_uleb_decode_boundary ( + vvs->boundary_buf + vvs->boundary_off, + vvs->boundary_len - vvs->boundary_off, + &msg_len, &eor); +if (ret <0 ) +break ; + vvs->boundary_off += ret; -if (bytes_to_copy) { + offset =VIRTIO_VSOCK_SKB_CB (skb)->offset; + bytes_to_copy =min (user_buf_len, (size_t )msg_len); + +if (bytes_to_copy && dequeued_len >=0 ) {int err; -/* sk_lock is held by caller so no one else can dequeue. -* Unlock rx_lock since skb_copy_datagram_iter() may sleep. -*/ spin_unlock_bh (&vvs->rx_lock); - - err =skb_copy_datagram_iter (skb,0 , + err =skb_copy_datagram_iter (skb, offset, &msg->msg_iter, bytes_to_copy); -if (err) { -/* Copy of message failed. Rest of -* fragments will be freed without copy. -*/ - dequeued_len = err; - }else { - user_buf_len -= bytes_to_copy; - } -spin_lock_bh (&vvs->rx_lock); +if (err) + dequeued_len = err; +else + user_buf_len -= bytes_to_copy; }if (dequeued_len >=0 ) - dequeued_len += pkt_len; - } + dequeued_len += msg_len; -if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ) { +VIRTIO_VSOCK_SKB_CB (skb)->offset += msg_len; msg_ready = true; vvs->msg_count--; -if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOR ) +if (eor) msg->msg_flags |=MSG_EOR ; - } -virtio_transport_dec_rx_pkt (vvs, pkt_len, pkt_len); -kfree_skb (skb); +virtio_transport_dec_rx_pkt (vvs, msg_len, msg_len); + +if (VIRTIO_VSOCK_SKB_CB (skb)->offset >= skb->len) { +__skb_unlink (skb, &vvs->rx_queue); +kfree_skb (skb); + } + +if (vvs->boundary_off >= vvs->boundary_len /2 ) +vsock_boundary_buf_compact (vvs); + }else { +struct virtio_vsock_hdr *hdr; +size_t pkt_len; + + skb =__skb_dequeue (&vvs->rx_queue); +if (!skb) +break ; + hdr =virtio_vsock_hdr (skb); + pkt_len = (size_t )le32_to_cpu (hdr->len); + +if (dequeued_len >=0 ) { +size_t bytes_to_copy; + + bytes_to_copy =min (user_buf_len, pkt_len); + +if (bytes_to_copy) { +int err; + +spin_unlock_bh (&vvs->rx_lock); + err =skb_copy_datagram_iter ( + skb,0 , &msg->msg_iter, + bytes_to_copy); +if (err) + dequeued_len = err; +else + user_buf_len -= bytes_to_copy; +spin_lock_bh (&vvs->rx_lock); + } + +if (dequeued_len >=0 ) + dequeued_len += pkt_len; + } + +if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ) { + msg_ready = true; + vvs->msg_count--; + +if (le32_to_cpu (hdr->flags) & +VIRTIO_VSOCK_SEQ_EOR ) + msg->msg_flags |=MSG_EOR ; + } + +virtio_transport_dec_rx_pkt (vvs, pkt_len, pkt_len); +kfree_skb (skb); + } }spin_unlock_bh (&vvs->rx_lock);@@ -1132,6 +1299,7 @@ void virtio_transport_destruct(struct vsock_sock *vsk)virtio_transport_cancel_close_work (vsk, true); +kfree (vvs->boundary_buf);kfree (vvs); vsk->trans =NULL ; }@@ -1224,6 +1392,11 @@ static void virtio_transport_remove_sock(struct vsock_sock *vsk)* removing it. */ __skb_queue_purge (&vvs->rx_queue); +kfree (vvs->boundary_buf); + vvs->boundary_buf =NULL ; + vvs->boundary_len =0 ; + vvs->boundary_alloc =0 ; + vvs->boundary_off =0 ;vsock_remove_sock (vsk); }@@ -1395,23 +1568,62 @@ virtio_transport_recv_enqueue(struct vsock_sock *vsk, !skb_is_nonlinear(skb)) {struct virtio_vsock_hdr *last_hdr;struct sk_buff *last_skb; +bool last_has_eom; +bool has_eom; last_skb =skb_peek_tail (&vvs->rx_queue); last_hdr =virtio_vsock_hdr (last_skb); + last_has_eom =le32_to_cpu (last_hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ; + has_eom =le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ; -/* If there is space in the last packet queued, we copy the -* new packet in its buffer. We avoid this if the last packet -* queued has VIRTIO_VSOCK_SEQ_EOM set, because this is -* delimiter of SEQPACKET message, so 'pkt' is the first packet -* of a new message. -*/ -if (skb->len <skb_tailroom (last_skb) && - !(le32_to_cpu (last_hdr->flags) &VIRTIO_VSOCK_SEQ_EOM )) { -memcpy (skb_put (last_skb, skb->len), skb->data, skb->len); - free_pkt = true; - last_hdr->flags |= hdr->flags; -le32_add_cpu (&last_hdr->len, len); - goto out; +if (skb->len <skb_tailroom (last_skb)) { +if (!last_has_eom) { +/* Same-message coalescing (existing path) */ +memcpy (skb_put (last_skb, skb->len), + skb->data, skb->len); + free_pkt = true; + last_hdr->flags |= hdr->flags; +le32_add_cpu (&last_hdr->len, len); + goto out; + } + +/* Cross-EOM: coalesce complete messages into one skb, +* recording message boundaries in a compact BER buffer. +* Only when incoming packet also has EOM (complete msg). +*/ +if (has_eom && !sk_psock (sk_vsock (vsk))) { +bool prev_eor, cur_eor; +u8 tmp[12 ]; +int n =0 ; + + cur_eor =le32_to_cpu (hdr->flags) & +VIRTIO_VSOCK_SEQ_EOR ; + +if (!VIRTIO_VSOCK_SKB_CB (last_skb)->has_boundary_entries) { +u32 prev_len =le32_to_cpu (last_hdr->len); + + prev_eor =le32_to_cpu (last_hdr->flags) & +VIRTIO_VSOCK_SEQ_EOR ; + n +=vsock_uleb_encode_boundary ( + tmp + n, prev_len, prev_eor); + } + n +=vsock_uleb_encode_boundary ( + tmp + n, len, cur_eor); + +if (!vsock_boundary_buf_ensure ( + vvs, vvs->boundary_len + n)) { +memcpy (vvs->boundary_buf + + vvs->boundary_len, tmp, n); + vvs->boundary_len += n; +VIRTIO_VSOCK_SKB_CB (last_skb)->has_boundary_entries = true; +memcpy (skb_put (last_skb, skb->len), + skb->data, skb->len); + free_pkt = true; + last_hdr->flags |= hdr->flags; +le32_add_cpu (&last_hdr->len, len); + goto out; + } + } } }
Michael S. Tsirkin
maintainerQuoted 18 lines · Stefano Garzarella
Quoted 3 lines · Michael S. Tsirkin
Quoted 2 lines · Stefano Garzarella
Quoted 1 line · Michael S. Tsirkin
On Thu, May 07, 2026 at 11:09:47AM +0200, Stefano Garzarella wrote:
[...]
Quoted 11 lines
Quoted 10 lines
Quoted 6 lines
Quoted 2 lines
For now, we're already doing something:
merging the skuffs if they don't have EOM set.Right that's good. You could go further and merge with EOM too if you stick the info about message boundaries somewhere else.
This adds a lot of complexity IMO, but we can try.
Do you have something in mind?
Quoted 8 lines
Quoted 3 lines
BER is clearly overkill but here's a POC that claude made for me, just to give u an idea. It's clearly has a ton of issues, for example I dislike how
GFP_ATOMICis handled.Okay, I somewhat understand, but clearly this isn't net material, so for now I think the best thing to do is to merge the fixup I sent (or something similar):
https://lore.kernel.org/netdev/[email protected]/
will respond there.
Quoted 5 lines
This is a major change that should be merged with more caution. Could this have too much of an impact on performance?
Thanks,
Stefano
Upstream is so broken now, I'd not worry about perf even.
Quoted 488 lines
Quoted 488 lines
Yet it seems to work fine in light testing.
-->
vsock/virtio: use DWARF ULEB128 to record EOM boundaries, enable cross-EOM skb coalescing
virtio_transport_recv_enqueue()currently refuses to coalesce an incoming skb with the previous one when the previous skb carriesVIRTIO_VSOCK_SEQ_EOM. This forces one skb per seqpacket message. For workloads with many small or zero-byte messages the per-skb overhead (~960 bytes) dominates, causing unbounded memory growth.Decouple message boundary tracking from the skb structure: store boundary offsets in a compact side buffer using DWARF ULEB128 encoding with the EOR flag folded into the low bit, then allow the data of multiple complete messages to be coalesced into a single skb.
Cross-EOM coalescing fires only when:
- both the tail skb and the incoming packet carry EOM (complete msgs)
- the incoming packet fits in the tail skb's tailroom
- no BPF psock is attached (read_skbexpects one msg per skb)On allocation failure the code falls back to separate skbs (existing behaviour). Credit accounting is unchanged; the boundary buffer is capped at
PAGE_SIZE.Signed-off-by Michael S. Tsirkin <[email protected]>Co-Authored-By Claude Opus 4.6 (1M context) <[email protected]>include/linux/virtio_vsock.h +7
-0@@ -12,6 +12,7 @@struct virtio_vsock_skb_cb {bool reply;bool tap_delivered; +bool has_boundary_entries;u32 offset; };@@ -167,6 +168,12 @@ struct virtio_vsock_sock {u32 buf_used;struct sk_buff_head rx_queue;u32 msg_count; + +/* ULEB128-encoded seqpacket message boundary buffer */ +u8 *boundary_buf; +u32 boundary_len; +u32 boundary_alloc; +u32 boundary_off; };struct virtio_vsock_pkt_info {net/vmw_vsock/virtio_transport_common.c +275
-63@@ -11,6 +11,7 @@#include <linux/sched/signal.h> #include <linux/ctype.h> #include <linux/list.h> +#include <linux/skmsg.h> #include <linux/virtio_vsock.h> #include <uapi/linux/vsockmon.h> @@ -26,6 +27,91 @@/* Threshold for detecting small packets to copy */ #define GOOD_COPY_LEN 128 +#define VSOCK_BOUNDARY_BUF_INIT 64 +#define VSOCK_BOUNDARY_BUF_MAX PAGE_SIZE + +/* ULEB128 boundary encoding: value = (msg_len << 1) | eor. +* Each byte carries 7 data bits; bit 7 is set on all but the last byte. +* Max 5 bytes for a u32 msg_len (33 bits with eor shift). +*/ +static int vsock_uleb_encode_boundary (u8 *buf,u32 msg_len,bool eor) +{ +u64 val = ((u64 )msg_len <<1 ) | eor; +int n =0 ; + +do { + buf[n] = val &0x7f ; + val >>=7 ; +if (val) + buf[n] |=0x80 ; + n++; + }while (val); + +return n; +} + +static int vsock_uleb_decode_boundary (const u8 *buf,u32 avail, +u32 *msg_len,bool *eor) +{ +u64 val =0 ; +int shift =0 ; +int n =0 ; + +do { +if (n >= avail || shift >=35 ) +return -EINVAL ; + val |= (u64)(buf[n] &0x7f ) << shift; + shift +=7 ; + }while (buf[n++] &0x80 ); + + *eor = val &1 ; + *msg_len = val >>1 ; +return n; +} + +static void vsock_boundary_buf_compact (struct virtio_vsock_sock *vvs) +{ +if (vvs->boundary_off ==0 ) +return ; + + vvs->boundary_len -= vvs->boundary_off; +memmove (vvs->boundary_buf, vvs->boundary_buf + vvs->boundary_off, + vvs->boundary_len); + vvs->boundary_off =0 ; +} + +static int vsock_boundary_buf_ensure (struct virtio_vsock_sock *vvs,u32 needed) +{ +u32 new_alloc; +u8 *new_buf; + +if (vvs->boundary_alloc >= needed) +return 0 ; + +/* Reclaim consumed space before growing */ +if (vvs->boundary_off) { + needed -= vvs->boundary_off; +vsock_boundary_buf_compact (vvs); +if (vvs->boundary_alloc >= needed) +return 0 ; + } + + new_alloc =max (needed, vvs->boundary_alloc ? vvs->boundary_alloc *2 + :VSOCK_BOUNDARY_BUF_INIT ); +if (new_alloc >VSOCK_BOUNDARY_BUF_MAX ) + new_alloc =VSOCK_BOUNDARY_BUF_MAX ; +if (new_alloc < needed) +return -ENOMEM ; + + new_buf =krealloc (vvs->boundary_buf, new_alloc,GFP_ATOMIC ); +if (!new_buf) +return -ENOMEM ; + + vvs->boundary_buf = new_buf; + vvs->boundary_alloc = new_alloc; +return 0 ; +} +static void virtio_transport_cancel_close_work (struct vsock_sock *vsk,bool cancel_timeout);static s64 virtio_transport_has_space (struct virtio_vsock_sock *vvs);@@ -682,41 +768,74 @@ virtio_transport_seqpacket_do_peek(struct vsock_sock *vsk, total =0 ; len =msg_data_left (msg); - skb_queue_walk(&vvs->rx_queue, skb) { -struct virtio_vsock_hdr *hdr; + skb =skb_peek (&vvs->rx_queue); +if (skb &&VIRTIO_VSOCK_SKB_CB (skb)->has_boundary_entries) { +u32 msg_len, offset; +size_t bytes; +bool eor; +int ret; -if (total < len) { -size_t bytes; + ret =vsock_uleb_decode_boundary ( + vvs->boundary_buf + vvs->boundary_off, + vvs->boundary_len - vvs->boundary_off, + &msg_len, &eor); +if (ret <0 ) + goto unlock; + + offset =VIRTIO_VSOCK_SKB_CB (skb)->offset; + bytes =min (len, (size_t )msg_len); + +if (bytes) {int err; - bytes = len - total; -if (bytes > skb->len) - bytes = skb->len; -spin_unlock_bh (&vvs->rx_lock); - -/* sk_lock is held by caller so no one else can dequeue. -* Unlock rx_lock since skb_copy_datagram_iter() may sleep. -*/ - err =skb_copy_datagram_iter (skb,VIRTIO_VSOCK_SKB_CB (skb)->offset, + err =skb_copy_datagram_iter (skb, offset, &msg->msg_iter, bytes);if (err)return err; -spin_lock_bh (&vvs->rx_lock); } - total += skb->len; - hdr =virtio_vsock_hdr (skb); + total = msg_len; +if (eor) + msg->msg_flags |=MSG_EOR ; + }else { + skb_queue_walk(&vvs->rx_queue, skb) { +struct virtio_vsock_hdr *hdr; -if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ) { -if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOR ) - msg->msg_flags |=MSG_EOR ; +if (total < len) { +size_t bytes; +int err; -break ; + bytes = len - total; +if (bytes > skb->len) + bytes = skb->len; + +spin_unlock_bh (&vvs->rx_lock); + + err =skb_copy_datagram_iter ( + skb, +VIRTIO_VSOCK_SKB_CB (skb)->offset, + &msg->msg_iter, bytes); +if (err) +return err; + +spin_lock_bh (&vvs->rx_lock); + } + + total += skb->len; + hdr =virtio_vsock_hdr (skb); + +if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ) { +if (le32_to_cpu (hdr->flags) & +VIRTIO_VSOCK_SEQ_EOR ) + msg->msg_flags |=MSG_EOR ; +break ; + } } } +unlock:spin_unlock_bh (&vvs->rx_lock);return total;@@ -740,57 +859,105 @@ static int virtio_transport_seqpacket_do_dequeue(struct vsock_sock *vsk, }while (!msg_ready) { -struct virtio_vsock_hdr *hdr; -size_t pkt_len; - - skb =__skb_dequeue (&vvs->rx_queue); + skb =skb_peek (&vvs->rx_queue);if (!skb)break ; - hdr =virtio_vsock_hdr (skb); - pkt_len = (size_t )le32_to_cpu (hdr->len); -if (dequeued_len >=0 ) { +if (VIRTIO_VSOCK_SKB_CB (skb)->has_boundary_entries) {size_t bytes_to_copy; +u32 msg_len, offset; +bool eor; +int ret; - bytes_to_copy =min (user_buf_len, pkt_len); + ret =vsock_uleb_decode_boundary ( + vvs->boundary_buf + vvs->boundary_off, + vvs->boundary_len - vvs->boundary_off, + &msg_len, &eor); +if (ret <0 ) +break ; + vvs->boundary_off += ret; -if (bytes_to_copy) { + offset =VIRTIO_VSOCK_SKB_CB (skb)->offset; + bytes_to_copy =min (user_buf_len, (size_t )msg_len); + +if (bytes_to_copy && dequeued_len >=0 ) {int err; -/* sk_lock is held by caller so no one else can dequeue. -* Unlock rx_lock since skb_copy_datagram_iter() may sleep. -*/ spin_unlock_bh (&vvs->rx_lock); - - err =skb_copy_datagram_iter (skb,0 , + err =skb_copy_datagram_iter (skb, offset, &msg->msg_iter, bytes_to_copy); -if (err) { -/* Copy of message failed. Rest of -* fragments will be freed without copy. -*/ - dequeued_len = err; - }else { - user_buf_len -= bytes_to_copy; - } -spin_lock_bh (&vvs->rx_lock); +if (err) + dequeued_len = err; +else + user_buf_len -= bytes_to_copy; }if (dequeued_len >=0 ) - dequeued_len += pkt_len; - } + dequeued_len += msg_len; -if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ) { +VIRTIO_VSOCK_SKB_CB (skb)->offset += msg_len; msg_ready = true; vvs->msg_count--; -if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOR ) +if (eor) msg->msg_flags |=MSG_EOR ; - } -virtio_transport_dec_rx_pkt (vvs, pkt_len, pkt_len); -kfree_skb (skb); +virtio_transport_dec_rx_pkt (vvs, msg_len, msg_len); + +if (VIRTIO_VSOCK_SKB_CB (skb)->offset >= skb->len) { +__skb_unlink (skb, &vvs->rx_queue); +kfree_skb (skb); + } + +if (vvs->boundary_off >= vvs->boundary_len /2 ) +vsock_boundary_buf_compact (vvs); + }else { +struct virtio_vsock_hdr *hdr; +size_t pkt_len; + + skb =__skb_dequeue (&vvs->rx_queue); +if (!skb) +break ; + hdr =virtio_vsock_hdr (skb); + pkt_len = (size_t )le32_to_cpu (hdr->len); + +if (dequeued_len >=0 ) { +size_t bytes_to_copy; + + bytes_to_copy =min (user_buf_len, pkt_len); + +if (bytes_to_copy) { +int err; + +spin_unlock_bh (&vvs->rx_lock); + err =skb_copy_datagram_iter ( + skb,0 , &msg->msg_iter, + bytes_to_copy); +if (err) + dequeued_len = err; +else + user_buf_len -= bytes_to_copy; +spin_lock_bh (&vvs->rx_lock); + } + +if (dequeued_len >=0 ) + dequeued_len += pkt_len; + } + +if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ) { + msg_ready = true; + vvs->msg_count--; + +if (le32_to_cpu (hdr->flags) & +VIRTIO_VSOCK_SEQ_EOR ) + msg->msg_flags |=MSG_EOR ; + } + +virtio_transport_dec_rx_pkt (vvs, pkt_len, pkt_len); +kfree_skb (skb); + } }spin_unlock_bh (&vvs->rx_lock);@@ -1132,6 +1299,7 @@ void virtio_transport_destruct(struct vsock_sock *vsk)virtio_transport_cancel_close_work (vsk, true); +kfree (vvs->boundary_buf);kfree (vvs); vsk->trans =NULL ; }@@ -1224,6 +1392,11 @@ static void virtio_transport_remove_sock(struct vsock_sock *vsk)* removing it. */ __skb_queue_purge (&vvs->rx_queue); +kfree (vvs->boundary_buf); + vvs->boundary_buf =NULL ; + vvs->boundary_len =0 ; + vvs->boundary_alloc =0 ; + vvs->boundary_off =0 ;vsock_remove_sock (vsk); }@@ -1395,23 +1568,62 @@ virtio_transport_recv_enqueue(struct vsock_sock *vsk, !skb_is_nonlinear(skb)) {struct virtio_vsock_hdr *last_hdr;struct sk_buff *last_skb; +bool last_has_eom; +bool has_eom; last_skb =skb_peek_tail (&vvs->rx_queue); last_hdr =virtio_vsock_hdr (last_skb); + last_has_eom =le32_to_cpu (last_hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ; + has_eom =le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ; -/* If there is space in the last packet queued, we copy the -* new packet in its buffer. We avoid this if the last packet -* queued has VIRTIO_VSOCK_SEQ_EOM set, because this is -* delimiter of SEQPACKET message, so 'pkt' is the first packet -* of a new message. -*/ -if (skb->len <skb_tailroom (last_skb) && - !(le32_to_cpu (last_hdr->flags) &VIRTIO_VSOCK_SEQ_EOM )) { -memcpy (skb_put (last_skb, skb->len), skb->data, skb->len); - free_pkt = true; - last_hdr->flags |= hdr->flags; -le32_add_cpu (&last_hdr->len, len); - goto out; +if (skb->len <skb_tailroom (last_skb)) { +if (!last_has_eom) { +/* Same-message coalescing (existing path) */ +memcpy (skb_put (last_skb, skb->len), + skb->data, skb->len); + free_pkt = true; + last_hdr->flags |= hdr->flags; +le32_add_cpu (&last_hdr->len, len); + goto out; + } + +/* Cross-EOM: coalesce complete messages into one skb, +* recording message boundaries in a compact BER buffer. +* Only when incoming packet also has EOM (complete msg). +*/ +if (has_eom && !sk_psock (sk_vsock (vsk))) { +bool prev_eor, cur_eor; +u8 tmp[12 ]; +int n =0 ; + + cur_eor =le32_to_cpu (hdr->flags) & +VIRTIO_VSOCK_SEQ_EOR ; + +if (!VIRTIO_VSOCK_SKB_CB (last_skb)->has_boundary_entries) { +u32 prev_len =le32_to_cpu (last_hdr->len); + + prev_eor =le32_to_cpu (last_hdr->flags) & +VIRTIO_VSOCK_SEQ_EOR ; + n +=vsock_uleb_encode_boundary ( + tmp + n, prev_len, prev_eor); + } + n +=vsock_uleb_encode_boundary ( + tmp + n, len, cur_eor); + +if (!vsock_boundary_buf_ensure ( + vvs, vvs->boundary_len + n)) { +memcpy (vvs->boundary_buf + + vvs->boundary_len, tmp, n); + vvs->boundary_len += n; +VIRTIO_VSOCK_SKB_CB (last_skb)->has_boundary_entries = true; +memcpy (skb_put (last_skb, skb->len), + skb->data, skb->len); + free_pkt = true; + last_hdr->flags |= hdr->flags; +le32_add_cpu (&last_hdr->len, len); + goto out; + } + } } }
Michael S. Tsirkin
maintainerQuoted 16 lines · Stefano Garzarella
Quoted 3 lines · Michael S. Tsirkin
Quoted 2 lines · Stefano Garzarella
Quoted 1 line · Michael S. Tsirkin
On Thu, May 07, 2026 at 11:09:47AM +0200, Stefano Garzarella wrote:
[...]
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines
Quoted 2 lines
For now, we're already doing something:
merging the skuffs if they don't have EOM set.Right that's good. You could go further and merge with EOM too if you stick the info about message boundaries somewhere else.
This adds a lot of complexity IMO, but we can try.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Do you have something in mind?
BER is clearly overkill but here's a POC that claude made for me, just to give u an idea. It's clearly has a ton of issues, for example I dislike how
GFP_ATOMICis handled.Okay, I somewhat understand, but clearly this isn't net material
I doubt we have many other options given reverting the regression was ruled out.
Quoted 4 lines
so for now
I think the best thing to do is to merge the fixup I sent (or something similar):
https://lore.kernel.org/netdev/[email protected]/
I reviewed that one, problem is it's a spec violation/change that we'll have to support forever.
Quoted 5 lines
This is a major change that should be merged with more caution. Could this have too much of an impact on performance?
Thanks,
Stefano
It's really a POC, real patch is left as an excersise for the reader :). The correct approach IMHO is to only start using this
when we wasted a lot of memory on small packets.
For example, if sum(truesize) >= buf size.
then we'll not see a perf impact unless it's already pathological.
Quoted 488 lines
Quoted 488 lines
Yet it seems to work fine in light testing.
-->
vsock/virtio: use DWARF ULEB128 to record EOM boundaries, enable cross-EOM skb coalescing
virtio_transport_recv_enqueue()currently refuses to coalesce an incoming skb with the previous one when the previous skb carriesVIRTIO_VSOCK_SEQ_EOM. This forces one skb per seqpacket message. For workloads with many small or zero-byte messages the per-skb overhead (~960 bytes) dominates, causing unbounded memory growth.Decouple message boundary tracking from the skb structure: store boundary offsets in a compact side buffer using DWARF ULEB128 encoding with the EOR flag folded into the low bit, then allow the data of multiple complete messages to be coalesced into a single skb.
Cross-EOM coalescing fires only when:
- both the tail skb and the incoming packet carry EOM (complete msgs)
- the incoming packet fits in the tail skb's tailroom
- no BPF psock is attached (read_skbexpects one msg per skb)On allocation failure the code falls back to separate skbs (existing behaviour). Credit accounting is unchanged; the boundary buffer is capped at
PAGE_SIZE.Signed-off-by Michael S. Tsirkin <[email protected]>Co-Authored-By Claude Opus 4.6 (1M context) <[email protected]>include/linux/virtio_vsock.h +7
-0@@ -12,6 +12,7 @@struct virtio_vsock_skb_cb {bool reply;bool tap_delivered; +bool has_boundary_entries;u32 offset; };@@ -167,6 +168,12 @@ struct virtio_vsock_sock {u32 buf_used;struct sk_buff_head rx_queue;u32 msg_count; + +/* ULEB128-encoded seqpacket message boundary buffer */ +u8 *boundary_buf; +u32 boundary_len; +u32 boundary_alloc; +u32 boundary_off; };struct virtio_vsock_pkt_info {net/vmw_vsock/virtio_transport_common.c +275
-63@@ -11,6 +11,7 @@#include <linux/sched/signal.h> #include <linux/ctype.h> #include <linux/list.h> +#include <linux/skmsg.h> #include <linux/virtio_vsock.h> #include <uapi/linux/vsockmon.h> @@ -26,6 +27,91 @@/* Threshold for detecting small packets to copy */ #define GOOD_COPY_LEN 128 +#define VSOCK_BOUNDARY_BUF_INIT 64 +#define VSOCK_BOUNDARY_BUF_MAX PAGE_SIZE + +/* ULEB128 boundary encoding: value = (msg_len << 1) | eor. +* Each byte carries 7 data bits; bit 7 is set on all but the last byte. +* Max 5 bytes for a u32 msg_len (33 bits with eor shift). +*/ +static int vsock_uleb_encode_boundary (u8 *buf,u32 msg_len,bool eor) +{ +u64 val = ((u64 )msg_len <<1 ) | eor; +int n =0 ; + +do { + buf[n] = val &0x7f ; + val >>=7 ; +if (val) + buf[n] |=0x80 ; + n++; + }while (val); + +return n; +} + +static int vsock_uleb_decode_boundary (const u8 *buf,u32 avail, +u32 *msg_len,bool *eor) +{ +u64 val =0 ; +int shift =0 ; +int n =0 ; + +do { +if (n >= avail || shift >=35 ) +return -EINVAL ; + val |= (u64)(buf[n] &0x7f ) << shift; + shift +=7 ; + }while (buf[n++] &0x80 ); + + *eor = val &1 ; + *msg_len = val >>1 ; +return n; +} + +static void vsock_boundary_buf_compact (struct virtio_vsock_sock *vvs) +{ +if (vvs->boundary_off ==0 ) +return ; + + vvs->boundary_len -= vvs->boundary_off; +memmove (vvs->boundary_buf, vvs->boundary_buf + vvs->boundary_off, + vvs->boundary_len); + vvs->boundary_off =0 ; +} + +static int vsock_boundary_buf_ensure (struct virtio_vsock_sock *vvs,u32 needed) +{ +u32 new_alloc; +u8 *new_buf; + +if (vvs->boundary_alloc >= needed) +return 0 ; + +/* Reclaim consumed space before growing */ +if (vvs->boundary_off) { + needed -= vvs->boundary_off; +vsock_boundary_buf_compact (vvs); +if (vvs->boundary_alloc >= needed) +return 0 ; + } + + new_alloc =max (needed, vvs->boundary_alloc ? vvs->boundary_alloc *2 + :VSOCK_BOUNDARY_BUF_INIT ); +if (new_alloc >VSOCK_BOUNDARY_BUF_MAX ) + new_alloc =VSOCK_BOUNDARY_BUF_MAX ; +if (new_alloc < needed) +return -ENOMEM ; + + new_buf =krealloc (vvs->boundary_buf, new_alloc,GFP_ATOMIC ); +if (!new_buf) +return -ENOMEM ; + + vvs->boundary_buf = new_buf; + vvs->boundary_alloc = new_alloc; +return 0 ; +} +static void virtio_transport_cancel_close_work (struct vsock_sock *vsk,bool cancel_timeout);static s64 virtio_transport_has_space (struct virtio_vsock_sock *vvs);@@ -682,41 +768,74 @@ virtio_transport_seqpacket_do_peek(struct vsock_sock *vsk, total =0 ; len =msg_data_left (msg); - skb_queue_walk(&vvs->rx_queue, skb) { -struct virtio_vsock_hdr *hdr; + skb =skb_peek (&vvs->rx_queue); +if (skb &&VIRTIO_VSOCK_SKB_CB (skb)->has_boundary_entries) { +u32 msg_len, offset; +size_t bytes; +bool eor; +int ret; -if (total < len) { -size_t bytes; + ret =vsock_uleb_decode_boundary ( + vvs->boundary_buf + vvs->boundary_off, + vvs->boundary_len - vvs->boundary_off, + &msg_len, &eor); +if (ret <0 ) + goto unlock; + + offset =VIRTIO_VSOCK_SKB_CB (skb)->offset; + bytes =min (len, (size_t )msg_len); + +if (bytes) {int err; - bytes = len - total; -if (bytes > skb->len) - bytes = skb->len; -spin_unlock_bh (&vvs->rx_lock); - -/* sk_lock is held by caller so no one else can dequeue. -* Unlock rx_lock since skb_copy_datagram_iter() may sleep. -*/ - err =skb_copy_datagram_iter (skb,VIRTIO_VSOCK_SKB_CB (skb)->offset, + err =skb_copy_datagram_iter (skb, offset, &msg->msg_iter, bytes);if (err)return err; -spin_lock_bh (&vvs->rx_lock); } - total += skb->len; - hdr =virtio_vsock_hdr (skb); + total = msg_len; +if (eor) + msg->msg_flags |=MSG_EOR ; + }else { + skb_queue_walk(&vvs->rx_queue, skb) { +struct virtio_vsock_hdr *hdr; -if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ) { -if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOR ) - msg->msg_flags |=MSG_EOR ; +if (total < len) { +size_t bytes; +int err; -break ; + bytes = len - total; +if (bytes > skb->len) + bytes = skb->len; + +spin_unlock_bh (&vvs->rx_lock); + + err =skb_copy_datagram_iter ( + skb, +VIRTIO_VSOCK_SKB_CB (skb)->offset, + &msg->msg_iter, bytes); +if (err) +return err; + +spin_lock_bh (&vvs->rx_lock); + } + + total += skb->len; + hdr =virtio_vsock_hdr (skb); + +if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ) { +if (le32_to_cpu (hdr->flags) & +VIRTIO_VSOCK_SEQ_EOR ) + msg->msg_flags |=MSG_EOR ; +break ; + } } } +unlock:spin_unlock_bh (&vvs->rx_lock);return total;@@ -740,57 +859,105 @@ static int virtio_transport_seqpacket_do_dequeue(struct vsock_sock *vsk, }while (!msg_ready) { -struct virtio_vsock_hdr *hdr; -size_t pkt_len; - - skb =__skb_dequeue (&vvs->rx_queue); + skb =skb_peek (&vvs->rx_queue);if (!skb)break ; - hdr =virtio_vsock_hdr (skb); - pkt_len = (size_t )le32_to_cpu (hdr->len); -if (dequeued_len >=0 ) { +if (VIRTIO_VSOCK_SKB_CB (skb)->has_boundary_entries) {size_t bytes_to_copy; +u32 msg_len, offset; +bool eor; +int ret; - bytes_to_copy =min (user_buf_len, pkt_len); + ret =vsock_uleb_decode_boundary ( + vvs->boundary_buf + vvs->boundary_off, + vvs->boundary_len - vvs->boundary_off, + &msg_len, &eor); +if (ret <0 ) +break ; + vvs->boundary_off += ret; -if (bytes_to_copy) { + offset =VIRTIO_VSOCK_SKB_CB (skb)->offset; + bytes_to_copy =min (user_buf_len, (size_t )msg_len); + +if (bytes_to_copy && dequeued_len >=0 ) {int err; -/* sk_lock is held by caller so no one else can dequeue. -* Unlock rx_lock since skb_copy_datagram_iter() may sleep. -*/ spin_unlock_bh (&vvs->rx_lock); - - err =skb_copy_datagram_iter (skb,0 , + err =skb_copy_datagram_iter (skb, offset, &msg->msg_iter, bytes_to_copy); -if (err) { -/* Copy of message failed. Rest of -* fragments will be freed without copy. -*/ - dequeued_len = err; - }else { - user_buf_len -= bytes_to_copy; - } -spin_lock_bh (&vvs->rx_lock); +if (err) + dequeued_len = err; +else + user_buf_len -= bytes_to_copy; }if (dequeued_len >=0 ) - dequeued_len += pkt_len; - } + dequeued_len += msg_len; -if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ) { +VIRTIO_VSOCK_SKB_CB (skb)->offset += msg_len; msg_ready = true; vvs->msg_count--; -if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOR ) +if (eor) msg->msg_flags |=MSG_EOR ; - } -virtio_transport_dec_rx_pkt (vvs, pkt_len, pkt_len); -kfree_skb (skb); +virtio_transport_dec_rx_pkt (vvs, msg_len, msg_len); + +if (VIRTIO_VSOCK_SKB_CB (skb)->offset >= skb->len) { +__skb_unlink (skb, &vvs->rx_queue); +kfree_skb (skb); + } + +if (vvs->boundary_off >= vvs->boundary_len /2 ) +vsock_boundary_buf_compact (vvs); + }else { +struct virtio_vsock_hdr *hdr; +size_t pkt_len; + + skb =__skb_dequeue (&vvs->rx_queue); +if (!skb) +break ; + hdr =virtio_vsock_hdr (skb); + pkt_len = (size_t )le32_to_cpu (hdr->len); + +if (dequeued_len >=0 ) { +size_t bytes_to_copy; + + bytes_to_copy =min (user_buf_len, pkt_len); + +if (bytes_to_copy) { +int err; + +spin_unlock_bh (&vvs->rx_lock); + err =skb_copy_datagram_iter ( + skb,0 , &msg->msg_iter, + bytes_to_copy); +if (err) + dequeued_len = err; +else + user_buf_len -= bytes_to_copy; +spin_lock_bh (&vvs->rx_lock); + } + +if (dequeued_len >=0 ) + dequeued_len += pkt_len; + } + +if (le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ) { + msg_ready = true; + vvs->msg_count--; + +if (le32_to_cpu (hdr->flags) & +VIRTIO_VSOCK_SEQ_EOR ) + msg->msg_flags |=MSG_EOR ; + } + +virtio_transport_dec_rx_pkt (vvs, pkt_len, pkt_len); +kfree_skb (skb); + } }spin_unlock_bh (&vvs->rx_lock);@@ -1132,6 +1299,7 @@ void virtio_transport_destruct(struct vsock_sock *vsk)virtio_transport_cancel_close_work (vsk, true); +kfree (vvs->boundary_buf);kfree (vvs); vsk->trans =NULL ; }@@ -1224,6 +1392,11 @@ static void virtio_transport_remove_sock(struct vsock_sock *vsk)* removing it. */ __skb_queue_purge (&vvs->rx_queue); +kfree (vvs->boundary_buf); + vvs->boundary_buf =NULL ; + vvs->boundary_len =0 ; + vvs->boundary_alloc =0 ; + vvs->boundary_off =0 ;vsock_remove_sock (vsk); }@@ -1395,23 +1568,62 @@ virtio_transport_recv_enqueue(struct vsock_sock *vsk, !skb_is_nonlinear(skb)) {struct virtio_vsock_hdr *last_hdr;struct sk_buff *last_skb; +bool last_has_eom; +bool has_eom; last_skb =skb_peek_tail (&vvs->rx_queue); last_hdr =virtio_vsock_hdr (last_skb); + last_has_eom =le32_to_cpu (last_hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ; + has_eom =le32_to_cpu (hdr->flags) &VIRTIO_VSOCK_SEQ_EOM ; -/* If there is space in the last packet queued, we copy the -* new packet in its buffer. We avoid this if the last packet -* queued has VIRTIO_VSOCK_SEQ_EOM set, because this is -* delimiter of SEQPACKET message, so 'pkt' is the first packet -* of a new message. -*/ -if (skb->len <skb_tailroom (last_skb) && - !(le32_to_cpu (last_hdr->flags) &VIRTIO_VSOCK_SEQ_EOM )) { -memcpy (skb_put (last_skb, skb->len), skb->data, skb->len); - free_pkt = true; - last_hdr->flags |= hdr->flags; -le32_add_cpu (&last_hdr->len, len); - goto out; +if (skb->len <skb_tailroom (last_skb)) { +if (!last_has_eom) { +/* Same-message coalescing (existing path) */ +memcpy (skb_put (last_skb, skb->len), + skb->data, skb->len); + free_pkt = true; + last_hdr->flags |= hdr->flags; +le32_add_cpu (&last_hdr->len, len); + goto out; + } + +/* Cross-EOM: coalesce complete messages into one skb, +* recording message boundaries in a compact BER buffer. +* Only when incoming packet also has EOM (complete msg). +*/ +if (has_eom && !sk_psock (sk_vsock (vsk))) { +bool prev_eor, cur_eor; +u8 tmp[12 ]; +int n =0 ; + + cur_eor =le32_to_cpu (hdr->flags) & +VIRTIO_VSOCK_SEQ_EOR ; + +if (!VIRTIO_VSOCK_SKB_CB (last_skb)->has_boundary_entries) { +u32 prev_len =le32_to_cpu (last_hdr->len); + + prev_eor =le32_to_cpu (last_hdr->flags) & +VIRTIO_VSOCK_SEQ_EOR ; + n +=vsock_uleb_encode_boundary ( + tmp + n, prev_len, prev_eor); + } + n +=vsock_uleb_encode_boundary ( + tmp + n, len, cur_eor); + +if (!vsock_boundary_buf_ensure ( + vvs, vvs->boundary_len + n)) { +memcpy (vvs->boundary_buf + + vvs->boundary_len, tmp, n); + vvs->boundary_len += n; +VIRTIO_VSOCK_SKB_CB (last_skb)->has_boundary_entries = true; +memcpy (skb_put (last_skb, skb->len), + skb->data, skb->len); + free_pkt = true; + last_hdr->flags |= hdr->flags; +le32_add_cpu (&last_hdr->len, len); + goto out; + } + } } }
Stefano Garzarella
maintainerQuoted 19 lines · Michael S. Tsirkin
Quoted 18 lines · Stefano Garzarella
Quoted 3 lines · Michael S. Tsirkin
Quoted 2 lines · Stefano Garzarella
Quoted 1 line · Michael S. Tsirkin
On Thu, May 07, 2026 at 11:09:47AM +0200, Stefano Garzarella wrote:
[...]
Quoted 11 lines
Quoted 10 lines
Quoted 6 lines
Quoted 2 lines
For now, we're already doing something:
merging the skuffs if they don't have EOM set.Right that's good. You could go further and merge with EOM too if you stick the info about message boundaries somewhere else.
This adds a lot of complexity IMO, but we can try.
Do you have something in mind?
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
BER is clearly overkill but here's a POC that claude made for me, just to give u an idea. It's clearly has a ton of issues, for example I dislike how
GFP_ATOMICis handled.Okay, I somewhat understand, but clearly this isn't net material
I doubt we have many other options given reverting the regression was ruled out.
As Eric pointed out, we can't revert it.
Quoted 9 lines
Quoted 6 lines
so for now
I think the best thing to do is to merge the fixup I sent (or something similar):
https://lore.kernel.org/netdev/[email protected]/I reviewed that one, problem is it's a spec violation/change that we'll have to support forever.
I have a few points to make on this, but let's discuss them there.
Quoted 8 lines
Quoted 6 lines
This is a major change that should be merged with more caution. Could this have too much of an impact on performance?
Thanks,
StefanoIt's really a POC, real patch is left as an excersise for the reader :).
eheh, I see, but honestly, this overcomplication scares me. I'll try to think it over.
Quoted 6 lines
The correct approach IMHO is to only start using this when we wasted a lot of memory on small packets.
For example, if sum(truesize) >= buf size.
then we'll not see a perf impact unless it's already pathological.
Agree on this, which is similar to what I'm doing in that patch. Reducing the advertised buf_alloc only in pathological cases (e.g. overhead > buf_alloc).
Stefano
Quoted 24 lines · Stefano Garzarella
Quoted 23 lines · Michael S. Tsirkin
Quoted 22 lines · Stefano Garzarella
Quoted 3 lines · Michael S. Tsirkin
Quoted 2 lines · Stefano Garzarella
Quoted 1 line · Michael S. Tsirkin
On Thu, May 07, 2026 at 11:09:47AM +0200, Stefano Garzarella wrote:
[...]
Quoted 14 lines
Quoted 10 lines
Quoted 6 lines
Quoted 2 lines
For now, we're already doing something:
merging the skuffs if they don't have EOM set.Right that's good. You could go further and merge with EOM too if you stick the info about message boundaries somewhere else.
This adds a lot of complexity IMO, but we can try.
Do you have something in mind?
BER is clearly overkill but here's a POC that claude made for me, just to give u an idea. It's clearly has a ton of issues, for example I dislike how
GFP_ATOMICis handled.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Okay, I somewhat understand, but clearly this isn't net material
I doubt we have many other options given reverting the regression was ruled out.
As Eric pointed out, we can't revert it.
Could there be an option to disable the mitigation in guests, for the situation where the host is trusted? There are VMMs that implement AF_VSOCK in userspace with a backing AF_UNIX socket.
Quoted 26 lines
Quoted 7 lines
Quoted 4 lines
so for now
I think the best thing to do is to merge the fixup I sent (or something similar):
https://lore.kernel.org/netdev/[email protected]/I reviewed that one, problem is it's a spec violation/change that we'll have to support forever.
I have a few points to make on this, but let's discuss them there.
Quoted 9 lines
Quoted 6 lines
This is a major change that should be merged with more caution. Could this have too much of an impact on performance?
Thanks,
StefanoIt's really a POC, real patch is left as an excersise for the reader :).
eheh, I see, but honestly, this overcomplication scares me. I'll try to think it over.
Quoted 3 lines
The correct approach IMHO is to only start using this when we wasted a lot of memory on small packets.
Quoted 7 lines
Quoted 3 lines
For example, if sum(truesize) >= buf size.
then we'll not see a perf impact unless it's already pathological.
Agree on this, which is similar to what I'm doing in that patch. Reducing the advertised
buf_alloconly in pathological cases (e.g. overhead >buf_alloc).
This isn't enough to prevent data loss due to race conditions.
I'm CCing the virtio-comment list and a few others.
Right now, any application that needs to send massive amount of data over a vsock is simply broken. This isn't just theoretical. It's causing real-world problems for users of Waypipe. Waypipe forwards Wayland protocol messages over AF_VSOCK, so it can send a large amount of traffic over the socket. See https://gitlab.freedesktop.org/mstoceckl/waypipe/work_items/165.
If one is willing to mutate the ring buffer in-place, or to maintain an auxiliary counter, it's possible to store all messages with bounded (in practice) overhead. Specifically:
- If the first byte of a block of data is nonzero, it's a variable-length length. 1 byte for messages less than 128 bytes.
- If the first byte of a block of data is zero, the subsequent bytes are a variable-length counter that stores the number of consecutive zero-byte messages.
That adds a lot of complexity, which is very unfortunate for something that needs to be backported to stable kernels. I also suspect it requires all access to the ring buffer to take a lock rather than being lock-free. But it's the only approach that I can think of that can work with the current spec.
Could there at least be a normative note stating that drivers and devices should treat each message as consuming 1024 bytes + the size of the message itself, and warning that anything that doesn't is going to be broken in practice?
I'm CCing Val Packet (of Invisible Things Lab) and Alyssa Ross (of Spectrum) because both of them are working on systems that rely critically on vsock.
Signature · 2 lines
Sincerely, Demi Marie Obenour (she/her/hers)
Quoted 98 lines · Demi Marie Obenour
Quoted 24 lines · Stefano Garzarella
Quoted 23 lines · Michael S. Tsirkin
Quoted 22 lines · Stefano Garzarella
Quoted 3 lines · Michael S. Tsirkin
Quoted 2 lines · Stefano Garzarella
Quoted 1 line · Michael S. Tsirkin
On Thu, May 07, 2026 at 11:09:47AM +0200, Stefano Garzarella wrote:
[...]
Quoted 14 lines
Quoted 10 lines
Quoted 6 lines
Quoted 2 lines
For now, we're already doing something:
merging the skuffs if they don't have EOM set.Right that's good. You could go further and merge with EOM too if you stick the info about message boundaries somewhere else.
This adds a lot of complexity IMO, but we can try.
Do you have something in mind?
BER is clearly overkill but here's a POC that claude made for me, just to give u an idea. It's clearly has a ton of issues, for example I dislike how
GFP_ATOMICis handled.Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Okay, I somewhat understand, but clearly this isn't net material
I doubt we have many other options given reverting the regression was ruled out.
As Eric pointed out, we can't revert it.
Could there be an option to disable the mitigation in guests, for the situation where the host is trusted? There are VMMs that implement
AF_VSOCKin userspace with a backingAF_UNIXsocket.Quoted 26 lines
Quoted 7 lines
Quoted 4 lines
so for now
I think the best thing to do is to merge the fixup I sent (or something similar):
https://lore.kernel.org/netdev/[email protected]/I reviewed that one, problem is it's a spec violation/change that we'll have to support forever.
I have a few points to make on this, but let's discuss them there.
Quoted 9 lines
Quoted 6 lines
This is a major change that should be merged with more caution. Could this have too much of an impact on performance?
Thanks,
StefanoIt's really a POC, real patch is left as an excersise for the reader :).
eheh, I see, but honestly, this overcomplication scares me. I'll try to think it over.
Quoted 3 lines
The correct approach IMHO is to only start using this when we wasted a lot of memory on small packets.
Quoted 7 lines
Quoted 3 lines
For example, if sum(truesize) >= buf size.
then we'll not see a perf impact unless it's already pathological.
Agree on this, which is similar to what I'm doing in that patch. Reducing the advertised
buf_alloconly in pathological cases (e.g. overhead >buf_alloc).This isn't enough to prevent data loss due to race conditions.
I'm CCing the virtio-comment list and a few others.
Right now, any application that needs to send massive amount of data over a vsock is simply broken. This isn't just theoretical. It's causing real-world problems for users of Waypipe. Waypipe forwards Wayland protocol messages over
AF_VSOCK, so it can send a large amount of traffic over the socket. See https://gitlab.freedesktop.org/mstoceckl/waypipe/work_items/165.If one is willing to mutate the ring buffer in-place, or to maintain an auxiliary counter, it's possible to store all messages with bounded (in practice) overhead. Specifically:
- If the first byte of a block of data is nonzero, it's a variable-length length. 1 byte for messages less than 128 bytes.
- If the first byte of a block of data is zero, the subsequent bytes are a variable-length counter that stores the number of consecutive zero-byte messages.
That adds a lot of complexity, which is very unfortunate for something that needs to be backported to stable kernels. I also suspect it requires all access to the ring buffer to take a lock rather than being lock-free. But it's the only approach that I can think of that can work with the current spec.
Quoted 8 lines
Could there at least be a normative note stating that drivers and devices should treat each message as consuming 1024 bytes + the size of the message itself, and warning that anything that doesn't is going to be broken in practice?
I'm CCing Val Packet (of Invisible Things Lab) and Alyssa Ross (of Spectrum) because both of them are working on systems that rely critically on vsock.
Update: I see that patches have been upstreamed (with CC: stable) that reset the connection instead of data loss.
Signature · 2 lines
Sincerely, Demi Marie Obenour (she/her/hers)
Stefano Garzarella
maintainerQuoted 29 lines · Demi Marie Obenour
Quoted 27 lines · Demi Marie Obenour
Quoted 26 lines · Stefano Garzarella
Quoted 25 lines · Michael S. Tsirkin
Quoted 23 lines · Stefano Garzarella
Quoted 3 lines · Michael S. Tsirkin
Quoted 2 lines · Stefano Garzarella
Quoted 1 line · Michael S. Tsirkin
On Thu, May 07, 2026 at 11:09:47AM +0200, Stefano Garzarella wrote:
[...]
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines
For now, we're already doing something:
merging the skuffs if they don't have EOM set.Right that's good. You could go further and merge with EOM too if you stick the info about message boundaries somewhere else.
Quoted 7 lines
Quoted 3 lines
This adds a lot of complexity IMO, but we can try.
Do you have something in mind?
BER is clearly overkill but here's a POC that claude made for me, just to give u an idea. It's clearly has a ton of issues, for example I dislike how
GFP_ATOMICis handled.Okay, I somewhat understand, but clearly this isn't net material
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
I doubt we have many other options given reverting the regression was ruled out.
As Eric pointed out, we can't revert it.
Could there be an option to disable the mitigation in guests, for the situation where the host is trusted? There are VMMs that implement
AF_VSOCKin userspace with a backingAF_UNIXsocket.
Here the main problem was an attack to the host from the guest.
Can the application set the max (IIRC 4G) of SO_VM_SOCKETS_BUFFER_SIZE ?
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Quoted 8 lines
Quoted 5 lines
so for now
I think the best thing to do is to merge the fixup I sent (or something similar):
https://lore.kernel.org/netdev/[email protected]/I reviewed that one, problem is it's a spec violation/change that we'll have to support forever.
I have a few points to make on this, but let's discuss them there.
Quoted 9 lines
Quoted 6 lines
This is a major change that should be merged with more caution. Could this have too much of an impact on performance?
Thanks,
StefanoIt's really a POC, real patch is left as an excersise for the reader :).
eheh, I see, but honestly, this overcomplication scares me. I'll try to think it over.
Quoted 5 lines
The correct approach IMHO is to only start using this when we wasted a lot of memory on small packets.
For example, if sum(truesize) >= buf size.
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
then we'll not see a perf impact unless it's already pathological.
Agree on this, which is similar to what I'm doing in that patch. Reducing the advertised
buf_alloconly in pathological cases (e.g. overhead >buf_alloc).This isn't enough to prevent data loss due to race conditions.
Yep, we didn't go in that direction.
Quoted 21 lines
Quoted 21 lines
I'm CCing the virtio-comment list and a few others.
Right now, any application that needs to send massive amount of data over a vsock is simply broken. This isn't just theoretical. It's causing real-world problems for users of Waypipe. Waypipe forwards Wayland protocol messages over
AF_VSOCK, so it can send a large amount of traffic over the socket. See https://gitlab.freedesktop.org/mstoceckl/waypipe/work_items/165.If one is willing to mutate the ring buffer in-place, or to maintain an auxiliary counter, it's possible to store all messages with bounded (in practice) overhead. Specifically:
- If the first byte of a block of data is nonzero, it's a variable-length length. 1 byte for messages less than 128 bytes.
- If the first byte of a block of data is zero, the subsequent bytes are a variable-length counter that stores the number of consecutive zero-byte messages.
Quoted 5 lines
Quoted 5 lines
That adds a lot of complexity, which is very unfortunate for something that needs to be backported to stable kernels. I also suspect it requires all access to the ring buffer to take a lock rather than being lock-free. But it's the only approach that I can think of that can work with the current spec.
Yep, we were discussing something like that in this thread, but we are still working on that.
That said, any help on preparing patches for this would be more than welcome.
Quoted 5 lines
Quoted 5 lines
Could there at least be a normative note stating that drivers and devices should treat each message as consuming 1024 bytes + the size of the message itself, and warning that anything that doesn't is going to be broken in practice?
We would like to fix the issue. This is a transitive situation.
Quoted 7 lines
Quoted 4 lines
I'm CCing Val Packet (of Invisible Things Lab) and Alyssa Ross (of Spectrum) because both of them are working on systems that rely critically on vsock.
Update: I see that patches have been upstreamed (with CC: stable) that reset the connection instead of data loss.
Yep, hope that helps the situation. But again, can setting the max SO_VM_SOCKETS_BUFFER_SIZE helps those applications?
Thanks,
Stefano
Michael S. Tsirkin
maintainerQuoted 3 lines · Eric Dumazet
There is always a discrepancy between
skb->lenandskb->truesize. You will not be able to announce a 1MB window, and accept one milliion skb of 1-byte each.
We can if we copy.
Quoted 1 line
This kind of contract is broken.
Quoted 7 lines · Michael S. Tsirkin
Quoted 3 lines · Eric Dumazet
There is always a discrepancy between
skb->lenandskb->truesize. You will not be able to announce a 1MB window, and accept one milliion skb of 1-byte each.We can if we copy.
You mean, ignore VIRTIO_VSOCK_SEQ_EOM?
Michael S. Tsirkin
maintainerQuoted 10 lines · Eric Dumazet
Quoted 7 lines · Michael S. Tsirkin
Quoted 3 lines · Eric Dumazet
There is always a discrepancy between
skb->lenandskb->truesize. You will not be able to announce a 1MB window, and accept one milliion skb of 1-byte each.We can if we copy.
You mean, ignore
VIRTIO_VSOCK_SEQ_EOM?
No I mean saving it in a compact form.
If packet boundaries are the only concern, the overhead needn't exceed 50% even for 1 byte messages. asn.1 ber, for an over-engineered example?