vsock/virtio: SKB allocation improvements
Subsystems (7)
v4 cover vsock/virtio: SKB allocation improvements
Hi all,
Here is version four of the patches I previously posted here:
v1: https://lore.kernel.org/r/[email protected]
v2: https://lore.kernel.org/r/[email protected]
v3: https://lore.kernel.org/r/[email protected]
There are only two minor changes since v3:
* Use unlikely() in payload length check on the virtio rx path
* Add R-b tags from Stefano
Cheers,
Will
--->8
Will Deacon (9):
vhost/vsock: Avoid allocating arbitrarily-sized SKBs vsock/virtio: Validate length in packet header before skb_put() vsock/virtio: Move length check to callers of
virtio_vsock_skb_rx_put()
vsock/virtio: Resize receive buffers so that each SKB fits in a 4K
page
vsock/virtio: Rename virtio_vsock_alloc_skb() vsock/virtio: Move SKB allocation lower-bound check to callers vhost/vsock: Allocate nonlinear SKBs for handling large receive
buffers
vsock/virtio: Rename virtio_vsock_skb_rx_put() vsock/virtio: Allocate nonlinear SKBs for handling large transmit
buffers
drivers/vhost/vsock.c | 15 ++++---- include/linux/virtio_vsock.h | 46 +++++++++++++++++++------ net/vmw_vsock/virtio_transport.c | 20 ++++++++--- net/vmw_vsock/virtio_transport_common.c | 3 +- 4 files changed, 60 insertions(+), 24 deletions(-)
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
PATCH v4 1/9 vhost/vsock: Avoid allocating arbitrarily-sized SKBs
vhost_vsock_alloc_skb() returns NULL for packets advertising a length larger than VIRTIO_VSOCK_MAX_PKT_BUF_SIZE in the packet header. However, this is only checked once the SKB has been allocated and, if the length in the packet header is zero, the SKB may not be freed immediately.
Hoist the size check before the SKB allocation so that an iovec larger than VIRTIO_VSOCK_MAX_PKT_BUF_SIZE + the header size is rejected outright. The subsequent check on the length field in the header can then simply check that the allocated SKB is indeed large enough to hold the packet.
--- drivers/vhost/vsock.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/vhost/vsock.c +4 -2
@@ -344,6 +344,9 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq, len =iov_length (vq->iov, out); +if (len >VIRTIO_VSOCK_MAX_PKT_BUF_SIZE +VIRTIO_VSOCK_SKB_HEADROOM ) +return NULL ; +/* len contains both payload and hdr */ skb =virtio_vsock_alloc_skb (len,GFP_KERNEL );if (!skb)
@@ -367,8 +370,7 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,return skb;/* The pkt is too big or the length in the header is invalid */ -if (payload_len >VIRTIO_VSOCK_MAX_PKT_BUF_SIZE || - payload_len +sizeof (*hdr) > len) { +if (payload_len +sizeof (*hdr) > len) {kfree_skb (skb);return NULL ; }
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
Re: [PATCH v4 1/9] vhost/vsock: Avoid allocating arbitrarily-sized SKBs
Quoted 5 lines · Will Deacon
vhost_vsock_alloc_skb()returns NULL for packets advertising a length larger thanVIRTIO_VSOCK_MAX_PKT_BUF_SIZEin the packet header. However, this is only checked once the SKB has been allocated and, if the length in the packet header is zero, the SKB may not be freed immediately.
Can this be triggered from the guest? (I guess yes) Did we need to consider it as a security issue?
Quoted 40 lines
Hoist the size check before the SKB allocation so that an iovec larger than
VIRTIO_VSOCK_MAX_PKT_BUF_SIZE+ the header size is rejected outright. The subsequent check on the length field in the header can then simply check that the allocated SKB is indeed large enough to hold the packet.Cc <[email protected]>Fixes 71dc9ec9ac7d ("virtio/vsock: replace virtio_vsock_pkt with sk_buff")Reviewed-by Stefano Garzarella <[email protected]>Signed-off-by Will Deacon <[email protected]>--- drivers/vhost/vsock.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)drivers/vhost/vsock.c +4
-3@@ -344,6 +344,9 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq, len =iov_length (vq->iov, out); +if (len >VIRTIO_VSOCK_MAX_PKT_BUF_SIZE +VIRTIO_VSOCK_SKB_HEADROOM ) +return NULL ; +/* len contains both payload and hdr */ skb =virtio_vsock_alloc_skb (len,GFP_KERNEL );if (!skb)@@ -367,8 +370,7 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,return skb;/* The pkt is too big or the length in the header is invalid */ -if (payload_len >VIRTIO_VSOCK_MAX_PKT_BUF_SIZE || - payload_len +sizeof (*hdr) > len) { +if (payload_len +sizeof (*hdr) > len) {kfree_skb (skb);return NULL ; } --
Quoted 2 lines
2.50.0.727.gbf7dc18ff4-goog
Thanks
Stefano Garzarella
maintainerRe: [PATCH v4 1/9] vhost/vsock: Avoid allocating arbitrarily-sized SKBs
Quoted 10 lines · Jason Wang
Quoted 5 lines · Will Deacon
vhost_vsock_alloc_skb()returns NULL for packets advertising a length larger thanVIRTIO_VSOCK_MAX_PKT_BUF_SIZEin the packet header. However, this is only checked once the SKB has been allocated and, if the length in the packet header is zero, the SKB may not be freed immediately.Can this be triggered from the guest? (I guess yes) Did we need to consider it as a security issue?
Yep, but then the packet would still be discarded later, and the memory released, so it can only increase the pressure on allocation, but the guest can still do so by sending packets for example on an unopened port.
Stefano
Quoted 46 lines
Quoted 41 lines
Hoist the size check before the SKB allocation so that an iovec larger than
VIRTIO_VSOCK_MAX_PKT_BUF_SIZE+ the header size is rejected outright. The subsequent check on the length field in the header can then simply check that the allocated SKB is indeed large enough to hold the packet.Cc <[email protected]>Fixes 71dc9ec9ac7d ("virtio/vsock: replace virtio_vsock_pkt with sk_buff")Reviewed-by Stefano Garzarella <[email protected]>Signed-off-by Will Deacon <[email protected]>--- drivers/vhost/vsock.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)drivers/vhost/vsock.c +4
-3@@ -344,6 +344,9 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq, len =iov_length (vq->iov, out); +if (len >VIRTIO_VSOCK_MAX_PKT_BUF_SIZE +VIRTIO_VSOCK_SKB_HEADROOM ) +return NULL ; +/* len contains both payload and hdr */ skb =virtio_vsock_alloc_skb (len,GFP_KERNEL );if (!skb)@@ -367,8 +370,7 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,return skb;/* The pkt is too big or the length in the header is invalid */ -if (payload_len >VIRTIO_VSOCK_MAX_PKT_BUF_SIZE || - payload_len +sizeof (*hdr) > len) { +if (payload_len +sizeof (*hdr) > len) {kfree_skb (skb);return NULL ; } --Quoted 2 lines
2.50.0.727.gbf7dc18ff4-goog
Thanks
PATCH v4 2/9 vsock/virtio: Validate length in packet header before skb_put()
In reply to Will DeaconWhen receiving a vsock packet in the guest, only the virtqueue buffer size is validated prior to virtio_vsock_skb_rx_put(). Unfortunately, virtio_vsock_skb_rx_put() uses the length from the packet header as the length argument to skb_put(), potentially resulting in SKB overflow if the host has gone wonky.
Validate the length as advertised by the packet header before calling virtio_vsock_skb_rx_put().
--- net/vmw_vsock/virtio_transport.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
net/vmw_vsock/virtio_transport.c +10 -2
@@ -624,8 +624,9 @@ static void virtio_transport_rx_work(struct work_struct *work) do { virtqueue_disable_cb(vq); for (;;) { + unsigned int len, payload_len; + struct virtio_vsock_hdr *hdr; struct sk_buff *skb; - unsigned int len; if (!virtio_transport_more_replies(vsock)) { /* Stop rx until the device processes already
@@ -642,12 +643,19 @@ static void virtio_transport_rx_work(struct work_struct *work) vsock->rx_buf_nr--;/* Drop short/long packets */ -if (unlikely (len <sizeof (struct virtio_vsock_hdr ) || +if (unlikely (len <sizeof (*hdr) || len >virtio_vsock_skb_len (skb))) {kfree_skb (skb);continue ; } + hdr =virtio_vsock_hdr (skb); + payload_len =le32_to_cpu (hdr->len); +if (unlikely (payload_len > len -sizeof (*hdr))) { +kfree_skb (skb); +continue ; + } +virtio_vsock_skb_rx_put (skb);virtio_transport_deliver_tap_pkt (skb);virtio_transport_recv_pkt (&virtio_transport, skb);
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
Re: [PATCH v4 2/9] vsock/virtio: Validate length in packet header before skb_put()
Quoted 9 lines · Will Deacon
When receiving a vsock packet in the guest, only the virtqueue buffer size is validated prior to
virtio_vsock_skb_rx_put(). Unfortunately,virtio_vsock_skb_rx_put()uses the length from the packet header as the length argument toskb_put(), potentially resulting in SKB overflow if the host has gone wonky.Validate the length as advertised by the packet header before calling
virtio_vsock_skb_rx_put().
Quoted 6 lines
Cc <[email protected]>Fixes 71dc9ec9ac7d ("virtio/vsock: replace virtio_vsock_pkt with sk_buff")Signed-off-by Will Deacon <[email protected]>---
net/vmw_vsock/virtio_transport.c| 12 ++++++++++--1 file changed, 10 insertions(+), 2 deletions(-)
Quoted 40 lines
net/vmw_vsock/virtio_transport.c +10
-2@@ -624,8 +624,9 @@ static void virtio_transport_rx_work(struct work_struct *work) do { virtqueue_disable_cb(vq); for (;;) { + unsigned int len, payload_len; + struct virtio_vsock_hdr *hdr; struct sk_buff *skb; - unsigned int len; if (!virtio_transport_more_replies(vsock)) { /* Stop rx until the device processes already@@ -642,12 +643,19 @@ static void virtio_transport_rx_work(struct work_struct *work) vsock->rx_buf_nr--;/* Drop short/long packets */ -if (unlikely (len <sizeof (struct virtio_vsock_hdr ) || +if (unlikely (len <sizeof (*hdr) || len >virtio_vsock_skb_len (skb))) {kfree_skb (skb);continue ; } + hdr =virtio_vsock_hdr (skb); + payload_len =le32_to_cpu (hdr->len); +if (unlikely (payload_len > len -sizeof (*hdr))) { +kfree_skb (skb); +continue ; + } +virtio_vsock_skb_rx_put (skb);virtio_transport_deliver_tap_pkt (skb);virtio_transport_recv_pkt (&virtio_transport, skb);Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
PATCH v4 3/9 vsock/virtio: Move length check to callers of virtio_vsock_skb_rx_put()
In reply to Will Deaconvirtio_vsock_skb_rx_put() only calls skb_put() if the length in the packet header is not zero even though skb_put() handles this case gracefully.
Remove the functionally redundant check from virtio_vsock_skb_rx_put() and, on the assumption that this is a worthwhile optimisation for handling credit messages, augment the existing length checks in virtio_transport_rx_work() to elide the call for zero-length payloads. Since the callers all have the length, extend virtio_vsock_skb_rx_put() to take it as an additional parameter rather than fish it back out of the packet header.
Note that the vhost code already has similar logic in vhost_vsock_alloc_skb().
--- drivers/vhost/vsock.c | 2 +- include/linux/virtio_vsock.h | 9 ++------- net/vmw_vsock/virtio_transport.c | 4 +++- 3 files changed, 6 insertions(+), 9 deletions(-)
drivers/vhost/vsock.c +1 -1
@@ -375,7 +375,7 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,return NULL ; } -virtio_vsock_skb_rx_put (skb); +virtio_vsock_skb_rx_put (skb, payload_len); nbytes =copy_from_iter (skb->data, payload_len, &iov_iter);if (nbytes != payload_len) {
include/linux/virtio_vsock.h +2 -7
@@ -47,14 +47,9 @@ static inline void virtio_vsock_skb_clear_tap_delivered(struct sk_buff *skb)VIRTIO_VSOCK_SKB_CB (skb)->tap_delivered = false; } -static inline void virtio_vsock_skb_rx_put (struct sk_buff *skb) +static inline void virtio_vsock_skb_rx_put (struct sk_buff *skb,u32 len) { -u32 len; - - len =le32_to_cpu (virtio_vsock_hdr (skb)->len); - -if (len >0 ) -skb_put (skb, len); +skb_put (skb, len); }static inline struct sk_buff *virtio_vsock_alloc_skb (unsigned int size,gfp_t mask)
net/vmw_vsock/virtio_transport.c +3 -1
@@ -656,7 +656,9 @@ static void virtio_transport_rx_work(struct work_struct *work)continue ; } -virtio_vsock_skb_rx_put (skb); +if (payload_len) +virtio_vsock_skb_rx_put (skb, payload_len); +virtio_transport_deliver_tap_pkt (skb);virtio_transport_recv_pkt (&virtio_transport, skb); }
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
PATCH v4 4/9 vsock/virtio: Resize receive buffers so that each SKB fits in a 4K page
In reply to Will DeaconWhen allocating receive buffers for the vsock virtio RX virtqueue, an SKB is allocated with a 4140 data payload (the 44-byte packet header + VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE). Even when factoring in the SKB overhead, the resulting 8KiB allocation thanks to the rounding in kmalloc_reserve() is wasteful (~3700 unusable bytes) and results in a higher-order page allocation on systems with 4KiB pages just for the sake of a few hundred bytes of packet data.
Limit the vsock virtio RX buffers to 4KiB per SKB, resulting in much better memory utilisation and removing the need to allocate higher-order pages entirely.
--- include/linux/virtio_vsock.h | 7 ++++++- net/vmw_vsock/virtio_transport.c | 2 +- 2 files changed, 7 insertions(+), 2 deletions(-)
include/linux/virtio_vsock.h +6 -1
@@ -106,7 +106,12 @@ static inline size_t virtio_vsock_skb_len(struct sk_buff *skb)return (size_t )(skb_end_pointer (skb) - skb->head); } -#define VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE (1024 * 4) +/* Dimension the RX SKB so that the entire thing fits exactly into +* a single 4KiB page. This avoids wasting memory due to alloc_skb() +* rounding up to the next page order and also means that we +* don't leave higher-order pages sitting around in the RX queue. +*/ +#define VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE SKB_WITH_OVERHEAD(1024 * 4)#define VIRTIO_VSOCK_MAX_BUF_SIZE 0xFFFFFFFFUL#define VIRTIO_VSOCK_MAX_PKT_BUF_SIZE (1024 * 64)
net/vmw_vsock/virtio_transport.c +1 -1
@@ -307,7 +307,7 @@ virtio_transport_cancel_pkt(struct vsock_sock *vsk)static void virtio_vsock_rx_fill (struct virtio_vsock *vsock) { -int total_len =VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE +VIRTIO_VSOCK_SKB_HEADROOM ; +int total_len =VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE ;struct scatterlist pkt, *p;struct virtqueue *vq;struct sk_buff *skb;
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
Re: [PATCH v4 4/9] vsock/virtio: Resize receive buffers so that each SKB fits in a 4K page
Quoted 9 lines · Will Deacon
include/linux/virtio_vsock.h +6
-1-#define VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE (1024 * 4) +/* Dimension the RX SKB so that the entire thing fits exactly into +* a single 4KiB page. This avoids wasting memory due to alloc_skb() +* rounding up to the next page order and also means that we +* don't leave higher-order pages sitting around in the RX queue. +*/ +#define
VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZESKB_WITH_OVERHEAD(1024 * 4)
Should this be SKB_WITH_OVERHEAD()?
Or should it subtract VIRTIO_VSOCK_SKB_HEADROOM instead?
(And also, I have use cases where I want to expand this to 64KiB. Can I make it controllable with a sockopt? module param?)
Stefano Garzarella
maintainerRe: [PATCH v4 4/9] vsock/virtio: Resize receive buffers so that each SKB fits in a 4K page
Quoted 12 lines · David Woodhouse
Quoted 9 lines · Will Deacon
include/linux/virtio_vsock.h +6
-1-#define VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE (1024 * 4) +/* Dimension the RX SKB so that the entire thing fits exactly into +* a single 4KiB page. This avoids wasting memory due to alloc_skb() +* rounding up to the next page order and also means that we +* don't leave higher-order pages sitting around in the RX queue. +*/ +#define
VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZESKB_WITH_OVERHEAD(1024 * 4)Should this be
SKB_WITH_OVERHEAD()?
ehm, is what the patch is doing, no?
Quoted 2 lines
Or should it subtract
VIRTIO_VSOCK_SKB_HEADROOMinstead?
Why?
IIRC the goal of the patch was to have an SKB that fit entirely on one page, to avoid wasting memory, so yes, we are reducing the payload a little bit (4K vs 4K - VIRTIO_VSOCK_SKB_HEADROOM - SKB_OVERHEAD), but we are also reducing segmentation.
Quoted 3 lines
(And also, I have use cases where I want to expand this to 64KiB. Can I make it controllable with a sockopt? module param?)
I'm not sure about sockopt, because this is really device specific and can't be linked to a specific socket, since the device will pre-fill the queue with buffers that can be assigned to different sockets.
But yeah, perhaps a module parameter would suffice, provided that it can only be modified at load time, otherwise we would have to do something similar to NIC and ethtool, but I feel that would be too complicated for this use case.
Thanks,
Stefano
Re: [PATCH v4 4/9] vsock/virtio: Resize receive buffers so that each SKB fits in a 4K page
Quoted 17 lines · Stefano Garzarella
Quoted 12 lines · David Woodhouse
Quoted 9 lines · Will Deacon
include/linux/virtio_vsock.h +6
-1-#define VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE (1024 * 4) +/* Dimension the RX SKB so that the entire thing fits exactly into +* a single 4KiB page. This avoids wasting memory due to alloc_skb() +* rounding up to the next page order and also means that we +* don't leave higher-order pages sitting around in the RX queue. +*/ +#define
VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZESKB_WITH_OVERHEAD(1024 * 4)Should this be
SKB_WITH_OVERHEAD()?ehm, is what the patch is doing, no?
Quoted 1 line
Quoted 12 lines
Quoted 1 line
Or should it subtract
VIRTIO_VSOCK_SKB_HEADROOMinstead?Why?
IIRC the goal of the patch was to have an SKB that fit entirely on one page, to avoid wasting memory, so yes, we are reducing the payload a little bit (4K vs 4K -
VIRTIO_VSOCK_SKB_HEADROOM-SKB_OVERHEAD), but we are also reducing segmentation.Quoted 3 lines
(And also, I have use cases where I want to expand this to 64KiB. Can I make it controllable with a sockopt? module param?)
What page size are you using? At some point I had this as PAGE_SIZE but it wasn't popular:
https://lore.kernel.org/all/20250701201400.52442b0e@pumpkin/
Quoted 8 lines
I'm not sure about sockopt, because this is really device specific and can't be linked to a specific socket, since the device will pre-fill the queue with buffers that can be assigned to different sockets.
But yeah, perhaps a module parameter would suffice, provided that it can only be modified at load time, otherwise we would have to do something similar to NIC and ethtool, but I feel that would be too complicated for this use case.
FWIW, we carried something similar in Android for a while on the transmit side and it was a bit of a pain to maintain; we ended up in situations where the guest and the host had to be configured similarly for things to work, although the non-linear support should solve those issues now. I'm not against the idea, I just wouldn't wish that pain on anybody else!
Anyway, if we wanted to support something similar upstream for the rx buffers, I'd suggest specifying it as a page-order for the entire SKB allocation and clamping it to PAGE_ALLOC_COSTLY_ORDER.
Will
PATCH v4 5/9 vsock/virtio: Rename virtio_vsock_alloc_skb()
In reply to Will DeaconIn preparation for nonlinear allocations for large SKBs, rename virtio_vsock_alloc_skb() to virtio_vsock_alloc_linear_skb() to indicate that it returns linear SKBs unconditionally and switch all callers over to this new interface for now.
No functional change.
--- drivers/vhost/vsock.c | 2 +- include/linux/virtio_vsock.h | 3 ++- net/vmw_vsock/virtio_transport.c | 2 +- net/vmw_vsock/virtio_transport_common.c | 2 +- 4 files changed, 5 insertions(+), 4 deletions(-)
drivers/vhost/vsock.c +1 -1
@@ -348,7 +348,7 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,return NULL ;/* len contains both payload and hdr */ - skb =virtio_vsock_alloc_skb (len,GFP_KERNEL ); + skb =virtio_vsock_alloc_linear_skb (len,GFP_KERNEL );if (!skb)return NULL ;
include/linux/virtio_vsock.h +2 -1
@@ -52,7 +52,8 @@ static inline void virtio_vsock_skb_rx_put(struct sk_buff *skb, u32 len)skb_put (skb, len); } -static inline struct sk_buff *virtio_vsock_alloc_skb (unsigned int size,gfp_t mask) +static inline struct sk_buff * +virtio_vsock_alloc_linear_skb (unsigned int size,gfp_t mask) {struct sk_buff *skb;
net/vmw_vsock/virtio_transport.c +1 -1
@@ -316,7 +316,7 @@ static void virtio_vsock_rx_fill(struct virtio_vsock *vsock) vq = vsock->vqs[VSOCK_VQ_RX ]; do { - skb = virtio_vsock_alloc_skb(total_len, GFP_KERNEL); + skb = virtio_vsock_alloc_linear_skb(total_len, GFP_KERNEL); if (!skb) break;
net/vmw_vsock/virtio_transport_common.c +1 -1
@@ -261,7 +261,7 @@ static struct sk_buff *virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *if (!zcopy) skb_len += payload_len; - skb =virtio_vsock_alloc_skb (skb_len,GFP_KERNEL ); + skb =virtio_vsock_alloc_linear_skb (skb_len,GFP_KERNEL );if (!skb)return NULL ;
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
PATCH v4 6/9 vsock/virtio: Move SKB allocation lower-bound check to callers
In reply to Will Deaconvirtio_vsock_alloc_linear_skb() checks that the requested size is at least big enough for the packet header (VIRTIO_VSOCK_SKB_HEADROOM).
Of the three callers of virtio_vsock_alloc_linear_skb(), only vhost_vsock_alloc_skb() can potentially pass a packet smaller than the header size and, as it already has a check against the maximum packet size, extend its bounds checking to consider the minimum packet size and remove the check from virtio_vsock_alloc_linear_skb().
--- drivers/vhost/vsock.c | 3 ++- include/linux/virtio_vsock.h | 3 --- 2 files changed, 2 insertions(+), 4 deletions(-)
drivers/vhost/vsock.c +2 -1
@@ -344,7 +344,8 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq, len =iov_length (vq->iov, out); -if (len >VIRTIO_VSOCK_MAX_PKT_BUF_SIZE +VIRTIO_VSOCK_SKB_HEADROOM ) +if (len <VIRTIO_VSOCK_SKB_HEADROOM || + len >VIRTIO_VSOCK_MAX_PKT_BUF_SIZE +VIRTIO_VSOCK_SKB_HEADROOM )return NULL ;/* len contains both payload and hdr */
include/linux/virtio_vsock.h +0 -3
@@ -57,9 +57,6 @@ virtio_vsock_alloc_linear_skb(unsigned int size, gfp_t mask) {struct sk_buff *skb; -if (size <VIRTIO_VSOCK_SKB_HEADROOM ) -return NULL ; - skb =alloc_skb (size, mask);if (!skb)return NULL ;
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
PATCH v4 7/9 vhost/vsock: Allocate nonlinear SKBs for handling large receive buffers
In reply to Will DeaconWhen receiving a packet from a guest, vhost_vsock_handle_tx_kick() calls vhost_vsock_alloc_linear_skb() to allocate and fill an SKB with the receive data. Unfortunately, these are always linear allocations and can therefore result in significant pressure on kmalloc() considering that the maximum packet size (VIRTIO_VSOCK_MAX_PKT_BUF_SIZE + VIRTIO_VSOCK_SKB_HEADROOM) is a little over 64KiB, resulting in a 128KiB allocation for each packet.
Rework the vsock SKB allocation so that, for sizes with page order greater than PAGE_ALLOC_COSTLY_ORDER, a nonlinear SKB is allocated instead with the packet header in the SKB and the receive data in the fragments. Finally, add a debug warning if virtio_vsock_skb_rx_put() is ever called on an SKB with a non-zero length, as this would be destructive for the nonlinear case.
--- drivers/vhost/vsock.c | 8 +++----- include/linux/virtio_vsock.h | 40 +++++++++++++++++++++++++++++------- 2 files changed, 36 insertions(+), 12 deletions(-)
drivers/vhost/vsock.c +3 -5
@@ -349,7 +349,7 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,return NULL ;/* len contains both payload and hdr */ - skb =virtio_vsock_alloc_linear_skb (len,GFP_KERNEL ); + skb =virtio_vsock_alloc_skb (len,GFP_KERNEL );if (!skb)return NULL ;
@@ -378,10 +378,8 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,virtio_vsock_skb_rx_put (skb, payload_len); - nbytes =copy_from_iter (skb->data, payload_len, &iov_iter); -if (nbytes != payload_len) { -vq_err (vq,"Expected %zu byte payload, got %zu bytes\n" , - payload_len, nbytes); +if (skb_copy_datagram_from_iter (skb,0 , &iov_iter, payload_len)) { +vq_err (vq,"Failed to copy %zu byte payload\n" , payload_len);kfree_skb (skb);return NULL ; }
include/linux/virtio_vsock.h +33 -7
@@ -49,20 +49,46 @@ static inline void virtio_vsock_skb_clear_tap_delivered(struct sk_buff *skb)static inline void virtio_vsock_skb_rx_put (struct sk_buff *skb,u32 len) { -skb_put (skb, len); +DEBUG_NET_WARN_ON_ONCE (skb->len); + +if (skb_is_nonlinear (skb)) + skb->len = len; +else +skb_put (skb, len); +} + +static inline struct sk_buff * +__virtio_vsock_alloc_skb_with_frags (unsigned int header_len, +unsigned int data_len, +gfp_t mask) +{ +struct sk_buff *skb; +int err; + + skb =alloc_skb_with_frags (header_len, data_len, +PAGE_ALLOC_COSTLY_ORDER , &err, mask); +if (!skb) +return NULL ; + +skb_reserve (skb,VIRTIO_VSOCK_SKB_HEADROOM ); + skb->data_len = data_len; +return skb; }static inline struct sk_buff *virtio_vsock_alloc_linear_skb (unsigned int size,gfp_t mask) { -struct sk_buff *skb; +return __virtio_vsock_alloc_skb_with_frags (size,0 , mask); +} - skb =alloc_skb (size, mask); -if (!skb) -return NULL ; +static inline struct sk_buff *virtio_vsock_alloc_skb (unsigned int size,gfp_t mask) +{ +if (size <=SKB_WITH_OVERHEAD (PAGE_SIZE <<PAGE_ALLOC_COSTLY_ORDER )) +return virtio_vsock_alloc_linear_skb (size, mask); -skb_reserve (skb,VIRTIO_VSOCK_SKB_HEADROOM ); -return skb; + size -=VIRTIO_VSOCK_SKB_HEADROOM ; +return __virtio_vsock_alloc_skb_with_frags (VIRTIO_VSOCK_SKB_HEADROOM , + size, mask); } static inline void
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
PATCH v4 8/9 vsock/virtio: Rename virtio_vsock_skb_rx_put()
In reply to Will DeaconIn preparation for using virtio_vsock_skb_rx_put() when populating SKBs on the vsock TX path, rename virtio_vsock_skb_rx_put() to virtio_vsock_skb_put().
No functional change.
--- drivers/vhost/vsock.c | 2 +- include/linux/virtio_vsock.h | 2 +- net/vmw_vsock/virtio_transport.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)
drivers/vhost/vsock.c +1 -1
@@ -376,7 +376,7 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,return NULL ; } -virtio_vsock_skb_rx_put (skb, payload_len); +virtio_vsock_skb_put (skb, payload_len);if (skb_copy_datagram_from_iter (skb,0 , &iov_iter, payload_len)) {vq_err (vq,"Failed to copy %zu byte payload\n" , payload_len);
include/linux/virtio_vsock.h +1 -1
@@ -47,7 +47,7 @@ static inline void virtio_vsock_skb_clear_tap_delivered(struct sk_buff *skb)VIRTIO_VSOCK_SKB_CB (skb)->tap_delivered = false; } -static inline void virtio_vsock_skb_rx_put (struct sk_buff *skb,u32 len) +static inline void virtio_vsock_skb_put (struct sk_buff *skb,u32 len) {DEBUG_NET_WARN_ON_ONCE (skb->len);
net/vmw_vsock/virtio_transport.c +1 -1
@@ -657,7 +657,7 @@ static void virtio_transport_rx_work(struct work_struct *work) }if (payload_len) -virtio_vsock_skb_rx_put (skb, payload_len); +virtio_vsock_skb_put (skb, payload_len);virtio_transport_deliver_tap_pkt (skb);virtio_transport_recv_pkt (&virtio_transport, skb);
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
PATCH v4 9/9 vsock/virtio: Allocate nonlinear SKBs for handling large transmit buffers
In reply to Will DeaconWhen transmitting a vsock packet, virtio_transport_send_pkt_info() calls virtio_transport_alloc_linear_skb() to allocate and fill SKBs with the transmit data. Unfortunately, these are always linear allocations and can therefore result in significant pressure on kmalloc() considering that the maximum packet size (VIRTIO_VSOCK_MAX_PKT_BUF_SIZE + VIRTIO_VSOCK_SKB_HEADROOM) is a little over 64KiB, resulting in a 128KiB allocation for each packet.
Rework the vsock SKB allocation so that, for sizes with page order greater than PAGE_ALLOC_COSTLY_ORDER, a nonlinear SKB is allocated instead with the packet header in the SKB and the transmit data in the fragments. Note that this affects both the vhost and virtio transports.
--- net/vmw_vsock/virtio_transport_common.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
net/vmw_vsock/virtio_transport_common.c +3 -2
@@ -109,7 +109,8 @@ static int virtio_transport_fill_skb(struct sk_buff *skb,return __zerocopy_sg_from_iter (info->msg,NULL , skb, &info->msg->msg_iter, len,NULL ); -return memcpy_from_msg (skb_put (skb, len), info->msg, len); +virtio_vsock_skb_put (skb, len); +return skb_copy_datagram_from_iter (skb,0 , &info->msg->msg_iter, len); } static void virtio_transport_init_hdr(struct sk_buff *skb,
@@ -261,7 +262,7 @@ static struct sk_buff *virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *if (!zcopy) skb_len += payload_len; - skb =virtio_vsock_alloc_linear_skb (skb_len,GFP_KERNEL ); + skb =virtio_vsock_alloc_skb (skb_len,GFP_KERNEL );if (!skb)return NULL ;
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
Michael S. Tsirkin
maintainerRe: [PATCH v4 9/9] vsock/virtio: Allocate nonlinear SKBs for handling large transmit buffers
Quoted 8 lines · Will Deacon
When transmitting a vsock packet,
virtio_transport_send_pkt_info()callsvirtio_transport_alloc_linear_skb()to allocate and fill SKBs with the transmit data. Unfortunately, these are always linear allocations and can therefore result in significant pressure onkmalloc()considering that the maximum packet size (VIRTIO_VSOCK_MAX_PKT_BUF_SIZE+VIRTIO_VSOCK_SKB_HEADROOM) is a little over 64KiB, resulting in a 128KiB allocation for each packet.
Quoted 7 lines
Rework the vsock SKB allocation so that, for sizes with page order greater than
PAGE_ALLOC_COSTLY_ORDER, a nonlinear SKB is allocated instead with the packet header in the SKB and the transmit data in the fragments. Note that this affects both the vhost and virtio transports.Reviewed-by Stefano Garzarella <[email protected]>Signed-off-by Will Deacon <[email protected]>
So this caused a regression, see syzbot report:
https://lore.kernel.org/all/[email protected]
I'm inclined to revert unless we have a fix quickly.
Quoted 29 lines
--- net/vmw_vsock/virtio_transport_common.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)net/vmw_vsock/virtio_transport_common.c +3
-2@@ -109,7 +109,8 @@ static int virtio_transport_fill_skb(struct sk_buff *skb,return __zerocopy_sg_from_iter (info->msg,NULL , skb, &info->msg->msg_iter, len,NULL ); -return memcpy_from_msg (skb_put (skb, len), info->msg, len); +virtio_vsock_skb_put (skb, len); +return skb_copy_datagram_from_iter (skb,0 , &info->msg->msg_iter, len); } static void virtio_transport_init_hdr(struct sk_buff *skb,@@ -261,7 +262,7 @@ static struct sk_buff *virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *if (!zcopy) skb_len += payload_len; - skb =virtio_vsock_alloc_linear_skb (skb_len,GFP_KERNEL ); + skb =virtio_vsock_alloc_skb (skb_len,GFP_KERNEL );if (!skb)return NULL ;Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
Re: [PATCH v4 9/9] vsock/virtio: Allocate nonlinear SKBs for handling large transmit buffers
Quoted 14 lines · On Wed, 13 Aug 2025 04:41:09 -0400 "Michael S. Tsirkin" wrote:
Quoted 13 lines · Will Deacon
When transmitting a vsock packet,
virtio_transport_send_pkt_info()callsvirtio_transport_alloc_linear_skb()to allocate and fill SKBs with the transmit data. Unfortunately, these are always linear allocations and can therefore result in significant pressure onkmalloc()considering that the maximum packet size (VIRTIO_VSOCK_MAX_PKT_BUF_SIZE+VIRTIO_VSOCK_SKB_HEADROOM) is a little over 64KiB, resulting in a 128KiB allocation for each packet.Rework the vsock SKB allocation so that, for sizes with page order greater than
PAGE_ALLOC_COSTLY_ORDER, a nonlinear SKB is allocated instead with the packet header in the SKB and the transmit data in the fragments. Note that this affects both the vhost and virtio transports.
Quoted 9 lines
Quoted 2 lines
Reviewed-by Stefano Garzarella <[email protected]>Signed-off-by Will Deacon <[email protected]>So this caused a regression, see syzbot report:
https://lore.kernel.org/all/[email protected]
I'm inclined to revert unless we have a fix quickly.
Because recomputing skb len survived the syzbot test [1], Will looks innocent.
Michael S. Tsirkin
maintainerRe: [PATCH v4 9/9] vsock/virtio: Allocate nonlinear SKBs for handling large transmit buffers
Quoted 20 lines · Hillf Danton
Quoted 19 lines · On Wed, 13 Aug 2025 04:41:09 -0400 "Michael S. Tsirkin" wrote:
Quoted 8 lines · Will Deacon
When transmitting a vsock packet,
virtio_transport_send_pkt_info()callsvirtio_transport_alloc_linear_skb()to allocate and fill SKBs with the transmit data. Unfortunately, these are always linear allocations and can therefore result in significant pressure onkmalloc()considering that the maximum packet size (VIRTIO_VSOCK_MAX_PKT_BUF_SIZE+VIRTIO_VSOCK_SKB_HEADROOM) is a little over 64KiB, resulting in a 128KiB allocation for each packet.Quoted 7 lines
Rework the vsock SKB allocation so that, for sizes with page order greater than
PAGE_ALLOC_COSTLY_ORDER, a nonlinear SKB is allocated instead with the packet header in the SKB and the transmit data in the fragments. Note that this affects both the vhost and virtio transports.Reviewed-by Stefano Garzarella <[email protected]>Signed-off-by Will Deacon <[email protected]>So this caused a regression, see syzbot report:
Quoted 7 lines
Quoted 4 lines
https://lore.kernel.org/all/[email protected]
I'm inclined to revert unless we have a fix quickly.
Because recomputing skb len survived the syzbot test [1], Will looks innocent.
I'm not sure I follow that patch though. Do you mind submitting with an explanation in the commit log?
Signature · 1 line
MST
Re: [PATCH v4 9/9] vsock/virtio: Allocate nonlinear SKBs for handling large transmit buffers
Quoted 23 lines · On Fri, 15 Aug 2025 06:22:56 -0400 "Michael S. Tsirkin" wrote:
Quoted 22 lines · Hillf Danton
Quoted 21 lines · On Wed, 13 Aug 2025 04:41:09 -0400 "Michael S. Tsirkin" wrote:
Quoted 8 lines · Will Deacon
When transmitting a vsock packet,
virtio_transport_send_pkt_info()callsvirtio_transport_alloc_linear_skb()to allocate and fill SKBs with the transmit data. Unfortunately, these are always linear allocations and can therefore result in significant pressure onkmalloc()considering that the maximum packet size (VIRTIO_VSOCK_MAX_PKT_BUF_SIZE+VIRTIO_VSOCK_SKB_HEADROOM) is a little over 64KiB, resulting in a 128KiB allocation for each packet.Quoted 7 lines
Rework the vsock SKB allocation so that, for sizes with page order greater than
PAGE_ALLOC_COSTLY_ORDER, a nonlinear SKB is allocated instead with the packet header in the SKB and the transmit data in the fragments. Note that this affects both the vhost and virtio transports.Reviewed-by Stefano Garzarella <[email protected]>Signed-off-by Will Deacon <[email protected]>So this caused a regression, see syzbot report:
Quoted 9 lines
Quoted 5 lines
Quoted 2 lines
I'm inclined to revert unless we have a fix quickly.
Because recomputing skb len survived the syzbot test [1], Will looks innocent.
I'm not sure I follow that patch though. Do you mind submitting with an explanation in the commit log?
It is a simple debug patch to test if Will's work is good at least in the syzbot scenario, but stil a couple miles away from a patch with the SOB tag.
Michael S. Tsirkin
maintainerRe: [PATCH v4 9/9] vsock/virtio: Allocate nonlinear SKBs for handling large transmit buffers
Quoted 28 lines · Hillf Danton
Quoted 27 lines · On Fri, 15 Aug 2025 06:22:56 -0400 "Michael S. Tsirkin" wrote:
Quoted 26 lines · Hillf Danton
Quoted 14 lines · On Wed, 13 Aug 2025 04:41:09 -0400 "Michael S. Tsirkin" wrote:
Quoted 13 lines · Will Deacon
When transmitting a vsock packet,
virtio_transport_send_pkt_info()callsvirtio_transport_alloc_linear_skb()to allocate and fill SKBs with the transmit data. Unfortunately, these are always linear allocations and can therefore result in significant pressure onkmalloc()considering that the maximum packet size (VIRTIO_VSOCK_MAX_PKT_BUF_SIZE+VIRTIO_VSOCK_SKB_HEADROOM) is a little over 64KiB, resulting in a 128KiB allocation for each packet.Rework the vsock SKB allocation so that, for sizes with page order greater than
PAGE_ALLOC_COSTLY_ORDER, a nonlinear SKB is allocated instead with the packet header in the SKB and the transmit data in the fragments. Note that this affects both the vhost and virtio transports.Quoted 9 lines
Quoted 2 lines
Reviewed-by Stefano Garzarella <[email protected]>Signed-off-by Will Deacon <[email protected]>So this caused a regression, see syzbot report:
https://lore.kernel.org/all/[email protected]
I'm inclined to revert unless we have a fix quickly.
Because recomputing skb len survived the syzbot test [1], Will looks innocent.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
I'm not sure I follow that patch though. Do you mind submitting with an explanation in the commit log?
It is a simple debug patch to test if Will's work is good at least in the syzbot scenario, but stil a couple miles away from a patch with the SOB tag.
Oh that makes sense then.
Quoted 10 lines · Will Deacon
Hi all,
Here is version four of the patches I previously posted here:
v1: https://lore.kernel.org/r/[email protected]
v2: https://lore.kernel.org/r/[email protected]
v3: https://lore.kernel.org/r/[email protected]There are only two minor changes since v3:
Quoted 7 lines
* Use
unlikely()in payload length check on the virtio rx path* Add R-b tags from Stefano
Cheers,
Will
Who's applying them, me?
Quoted 37 lines
Cc Keir Fraser <[email protected]>Cc Steven Moreland <[email protected]>Cc Frederick Mayle <[email protected]>Cc Stefan Hajnoczi <[email protected]>Cc Stefano Garzarella <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc "Eugenio Pérez" <[email protected]>--->8
Will Deacon (9):
vhost/vsock: Avoid allocating arbitrarily-sized SKBs vsock/virtio: Validate length in packet header beforeskb_put()vsock/virtio: Move length check to callers of
virtio_vsock_skb_rx_put()
vsock/virtio: Resize receive buffers so that each SKB fits in a 4K
page
vsock/virtio: Renamevirtio_vsock_alloc_skb()vsock/virtio: Move SKB allocation lower-bound check to callers vhost/vsock: Allocate nonlinear SKBs for handling large receive
buffers
vsock/virtio: Renamevirtio_vsock_skb_rx_put()vsock/virtio: Allocate nonlinear SKBs for handling large transmit
buffersdrivers/vhost/vsock.c | 15 ++++---- include/linux/virtio_vsock.h | 46 +++++++++++++++++++------ net/vmw_vsock/virtio_transport.c | 20 ++++++++--- net/vmw_vsock/virtio_transport_common.c | 3 +- 4 files changed, 60 insertions(+), 24 deletions(-)Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
Stefano Garzarella
maintainerQuoted 16 lines · Michael S. Tsirkin
Quoted 14 lines · Will Deacon
Hi all,
Here is version four of the patches I previously posted here:
v1: https://lore.kernel.org/r/[email protected]
v2: https://lore.kernel.org/r/[email protected]
v3: https://lore.kernel.org/r/[email protected]There are only two minor changes since v3:
* Use
unlikely()in payload length check on the virtio rx path* Add R-b tags from Stefano
Quoted 8 lines
Quoted 3 lines
Cheers,
Will
Who's applying them, me?
It's pretty much only about virtio/vhost, so I think yours is the right tree.
I completely reviewed it, so it can go for me.
Thanks,
Stefano
Michael S. Tsirkin
maintainerQuoted 10 lines · Will Deacon
Hi all,
Here is version four of the patches I previously posted here:
v1: https://lore.kernel.org/r/[email protected]
v2: https://lore.kernel.org/r/[email protected]
v3: https://lore.kernel.org/r/[email protected]There are only two minor changes since v3:
Quoted 7 lines
* Use
unlikely()in payload length check on the virtio rx path* Add R-b tags from Stefano
Cheers,
Will
in my tree now. Let's see how does it go.
Quoted 37 lines
Cc Keir Fraser <[email protected]>Cc Steven Moreland <[email protected]>Cc Frederick Mayle <[email protected]>Cc Stefan Hajnoczi <[email protected]>Cc Stefano Garzarella <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc "Eugenio Pérez" <[email protected]>--->8
Will Deacon (9):
vhost/vsock: Avoid allocating arbitrarily-sized SKBs vsock/virtio: Validate length in packet header beforeskb_put()vsock/virtio: Move length check to callers of
virtio_vsock_skb_rx_put()
vsock/virtio: Resize receive buffers so that each SKB fits in a 4K
page
vsock/virtio: Renamevirtio_vsock_alloc_skb()vsock/virtio: Move SKB allocation lower-bound check to callers vhost/vsock: Allocate nonlinear SKBs for handling large receive
buffers
vsock/virtio: Renamevirtio_vsock_skb_rx_put()vsock/virtio: Allocate nonlinear SKBs for handling large transmit
buffersdrivers/vhost/vsock.c | 15 ++++---- include/linux/virtio_vsock.h | 46 +++++++++++++++++++------ net/vmw_vsock/virtio_transport.c | 20 ++++++++--- net/vmw_vsock/virtio_transport_common.c | 3 +- 4 files changed, 60 insertions(+), 24 deletions(-)Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog