Skip to contentSkip to conversation

Small packet processing handling changes

PATCH In review Parav Pandit 3 participants 11 messages Jan 13, 2023 – Jan 16, 2023

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover Small packet processing handling changes

Hi,

These two changes improve the small packet handling.

Patch summary:
patch-1 fixes the length check by considering Ethernet 60B frame size patch-2 avoids code duplication by reuses existing buffer free helper

Please review.

Parav Pandit (2):
virtio_net: Fix short frame length check
virtio_net: Reuse buffer free function

 drivers/net/virtio_net.c | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)
Signature · 1 line
2.26.2
#1 lore raw

PATCH 1/2 virtio_net: Fix short frame length check

A smallest Ethernet frame defined by IEEE 802.3 is 60 bytes without any preemble and CRC.

Current code only checks for minimal 14 bytes of Ethernet header length. Correct it to consider the minimum Ethernet frame length.

Fixes 296f96fcfc16 ("Net driver using virtio")
---
 drivers/net/virtio_net.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/virtio_net.c +1 -1
index 7723b2a49d8e..d45e140b6852 100644
@@ -1248,7 +1248,7 @@ static void receive_buf(struct virtnet_info *vi, struct receive_queue *rq,
 	struct sk_buff *skb;
 	struct virtio_net_hdr_mrg_rxbuf *hdr;
 
-	if (unlikely(len < vi->hdr_len + ETH_HLEN)) {
+	if (unlikely(len < vi->hdr_len + ETH_ZLEN)) {
 		pr_debug("%s: short packet %i\n", dev->name, len);
 		dev->stats.rx_length_errors++;
 		if (vi->mergeable_rx_bufs) {
Signature · 1 line
2.26.2
#2 lore raw

Re: [PATCH net-next 1/2] virtio_net: Fix short frame length check

Quoted 17 lines · Parav Pandit

A smallest Ethernet frame defined by IEEE 802.3 is 60 bytes without any preemble and CRC.

Current code only checks for minimal 14 bytes of Ethernet header length. Correct it to consider the minimum Ethernet frame length.

Fixes 296f96fcfc16 ("Net driver using virtio")
---
 drivers/net/virtio_net.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/virtio_net.c +0 -0
index 7723b2a49d8e..d45e140b6852 100644

drivers/net/virtio_net.c static void receive_buf(struct virtnet_info *vi, struct receive_queue *rq, · line 1254

Quoted 8 lines
drivers/net/virtio_net.c +1 -1
@@ -1248,7 +1248,7 @@ static void receive_buf(struct virtnet_info *vi, struct receive_queue *rq,
 	struct sk_buff *skb;
 	struct virtio_net_hdr_mrg_rxbuf *hdr;
 
-	if (unlikely(len < vi->hdr_len + ETH_HLEN)) {
+	if (unlikely(len < vi->hdr_len + ETH_ZLEN)) {
 		pr_debug("%s: short packet %i\n", dev->name, len);
 		dev->stats.rx_length_errors++;
 		if (vi->mergeable_rx_bufs) {

I'm not sure I agree with this change as packets are only 60B if they have gone across the wire as they are usually padded out on the transmit side. There may be cases where software routed packets may not be 60B.

As such rather than changing out ETH_HLEN for ETH_ZLEN I wonder if we should look at maybe making this a "<=" comparison instead since that is the only case I can think of where the packet would end up being entirely empty after eth_type_trans is called and we would be passing an skb with length 0.

#3 lore raw

RE: [PATCH net-next 1/2] virtio_net: Fix short frame length check

Quoted 26 lines

From: Alexander H Duyck <[email protected]>
Sent: Friday, January 13, 2023 6:24 PM

drivers/net/virtio_net.c static void receive_buf(struct virtnet_info *vi,

Quoted 17 lines · Parav Pandit

A smallest Ethernet frame defined by IEEE 802.3 is 60 bytes without any preemble and CRC.

Current code only checks for minimal 14 bytes of Ethernet header length. Correct it to consider the minimum Ethernet frame length.

Fixes 296f96fcfc16 ("Net driver using virtio")
---
 drivers/net/virtio_net.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/virtio_net.c index renamed from drivers/net/virtio_net.c +0 -0

7723b2a49d8e..d45e140b6852 100644

drivers/net/virtio_net.c +0 -0
@@ -1248,7 +1248,7 @@ static void receive_buf(struct virtnet_info *vi,

struct receive_queue *rq,

Quoted 3 lines
drivers/net/virtio_net.c +0 -0
 	struct sk_buff *skb;
 	struct virtio_net_hdr_mrg_rxbuf *hdr;
 
Quoted 10 lines

drivers/net/virtio_net.c static void receive_buf(struct virtnet_info *vi, struct receive_queue *rq, · line 1254

Quoted 5 lines
drivers/net/virtio_net.c +1 -1
-	if (unlikely(len < vi->hdr_len + ETH_HLEN)) {
+	if (unlikely(len < vi->hdr_len + ETH_ZLEN)) {
 		pr_debug("%s: short packet %i\n", dev->name, len);
 		dev->stats.rx_length_errors++;
 		if (vi->mergeable_rx_bufs) {

I'm not sure I agree with this change as packets are only 60B if they have gone across the wire as they are usually padded out on the transmit side. There may be cases where software routed packets may not be 60B.

Do you mean Linux kernel software? Any link to it would be helpful.

Quoted 4 lines

As such rather than changing out ETH_HLEN for ETH_ZLEN I wonder if we should look at maybe making this a "<=" comparison instead since that is the only case I can think of where the packet would end up being entirely empty after eth_type_trans is called and we would be passing an skb with length 0.

I likely didn’t understand your comment.
This driver check is before creating the skb for the received packet.
So, purpose is to not even process the packet header or prepare the skb if it not an Ethernet frame.

It is interesting to know when we get < 60B frame.

#4 lore raw

Re: [PATCH net-next 1/2] virtio_net: Fix short frame length check

Quoted 33 lines · Parav Pandit
Quoted 33 lines

From: Alexander H Duyck <[email protected]>
Sent: Friday, January 13, 2023 6:24 PM

drivers/net/virtio_net.c static void receive_buf(struct virtnet_info *vi,

Quoted 17 lines · Parav Pandit

A smallest Ethernet frame defined by IEEE 802.3 is 60 bytes without any preemble and CRC.

Current code only checks for minimal 14 bytes of Ethernet header length. Correct it to consider the minimum Ethernet frame length.

Fixes 296f96fcfc16 ("Net driver using virtio")
---
 drivers/net/virtio_net.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/virtio_net.c index renamed from drivers/net/virtio_net.c +0 -0

7723b2a49d8e..d45e140b6852 100644

drivers/net/virtio_net.c +0 -0
@@ -1248,7 +1248,7 @@ static void receive_buf(struct virtnet_info *vi,

struct receive_queue *rq,

Quoted 8 lines
drivers/net/virtio_net.c +1 -1
    struct sk_buff *skb;
    struct virtio_net_hdr_mrg_rxbuf *hdr;
 
-   if (unlikely(len < vi->hdr_len + ETH_HLEN)) {
+   if (unlikely(len < vi->hdr_len + ETH_ZLEN)) {
            pr_debug("%s: short packet %i\n", dev->name, len);
            dev->stats.rx_length_errors++;
            if (vi->mergeable_rx_bufs) {
Quoted 5 lines
Quoted 4 lines

I'm not sure I agree with this change as packets are only 60B if they have gone across the wire as they are usually padded out on the transmit side. There may be cases where software routed packets may not be 60B.

Do you mean Linux kernel software? Any link to it would be helpful.

The problem is there are several software paths involved and that is why I am wanting to be cautious. As I recall this would impact Qemu itself, DPDK, the Linux Kernel and several others if I am not mistaken. That is why I am tending to err on the side of caution as this is a pretty significant change.

Quoted 10 lines
Quoted 4 lines

As such rather than changing out ETH_HLEN for ETH_ZLEN I wonder if we should look at maybe making this a "<=" comparison instead since that is the only case I can think of where the packet would end up being entirely empty after eth_type_trans is called and we would be passing an skb with length 0.

I likely didn’t understand your comment.
This driver check is before creating the skb for the received packet.
So, purpose is to not even process the packet header or prepare the skb if it not an Ethernet frame.

It is interesting to know when we get < 60B frame.

If I recall, a UDPv4 frame can easily do it since Ethernet is 14B, IP header is 20, and UDP is only 8 so that only comes to 42B if I recall correctly. Similarly I think a TCPv4 Frame can be as small as 54B if you disable all the option headers.

A quick and dirty test would be to run something like a netperf UDP_RR test. I know in the case of the network stack we see the transmits that go out are less than 60B until they are padded on xmit, usually by the device. My concern is wanting to make sure all those paths are covered before we assume that all the packets will be padded.
_______________________________________________
Virtualization mailing list [email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

#5 lore raw

Re: [PATCH net-next 1/2] virtio_net: Fix short frame length check

Quoted 63 lines · Alexander Duyck
Quoted 33 lines · Parav Pandit
Quoted 33 lines

From: Alexander H Duyck <[email protected]>
Sent: Friday, January 13, 2023 6:24 PM

drivers/net/virtio_net.c static void receive_buf(struct virtnet_info *vi,

Quoted 17 lines · Parav Pandit

A smallest Ethernet frame defined by IEEE 802.3 is 60 bytes without any preemble and CRC.

Current code only checks for minimal 14 bytes of Ethernet header length. Correct it to consider the minimum Ethernet frame length.

Fixes 296f96fcfc16 ("Net driver using virtio")
---
 drivers/net/virtio_net.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/virtio_net.c index renamed from drivers/net/virtio_net.c +0 -0

7723b2a49d8e..d45e140b6852 100644

drivers/net/virtio_net.c +0 -0
@@ -1248,7 +1248,7 @@ static void receive_buf(struct virtnet_info *vi,

struct receive_queue *rq,

Quoted 8 lines
drivers/net/virtio_net.c +1 -1
    struct sk_buff *skb;
    struct virtio_net_hdr_mrg_rxbuf *hdr;
 
-   if (unlikely(len < vi->hdr_len + ETH_HLEN)) {
+   if (unlikely(len < vi->hdr_len + ETH_ZLEN)) {
            pr_debug("%s: short packet %i\n", dev->name, len);
            dev->stats.rx_length_errors++;
            if (vi->mergeable_rx_bufs) {
Quoted 5 lines
Quoted 4 lines

I'm not sure I agree with this change as packets are only 60B if they have gone across the wire as they are usually padded out on the transmit side. There may be cases where software routed packets may not be 60B.

Do you mean Linux kernel software? Any link to it would be helpful.

The problem is there are several software paths involved and that is why I am wanting to be cautious. As I recall this would impact Qemu itself, DPDK, the Linux Kernel and several others if I am not mistaken. That is why I am tending to err on the side of caution as this is a pretty significant change.

Quoted 10 lines
Quoted 4 lines

As such rather than changing out ETH_HLEN for ETH_ZLEN I wonder if we should look at maybe making this a "<=" comparison instead since that is the only case I can think of where the packet would end up being entirely empty after eth_type_trans is called and we would be passing an skb with length 0.

I likely didn’t understand your comment.
This driver check is before creating the skb for the received packet.
So, purpose is to not even process the packet header or prepare the skb if it not an Ethernet frame.

It is interesting to know when we get < 60B frame.

If I recall, a UDPv4 frame can easily do it since Ethernet is 14B, IP header is 20, and UDP is only 8 so that only comes to 42B if I recall correctly. Similarly I think a TCPv4 Frame can be as small as 54B if you disable all the option headers.

Quoted 5 lines

A quick and dirty test would be to run something like a netperf UDP_RR test. I know in the case of the network stack we see the transmits that go out are less than 60B until they are padded on xmit, usually by the device. My concern is wanting to make sure all those paths are covered before we assume that all the packets will be padded.

I was curious so I decided to try verifying things with a qemu w/ user networking and virtio-net. From what I can tell it looks like it is definitely not padding them out.

19:34:38.331376 IP (tos 0x0, ttl 64, id 31799, offset 0, flags [DF], proto UDP (17), length 29)

    localhost.localdomain.59579 > _gateway.52701: [udp sum ok] UDP, length 1
        0x0000:  5255 0a00 0202 5254 0012 3456 0800 4500
        0x0010:  001d 7c37 4000 4011 a688 0a00 020f 0a00
        0x0020:  0202 e8bb cddd 0009 c331 6e
19:34:38.331431 IP (tos 0x0, ttl 64, id 45459, offset 0, flags [none],
proto UDP (17), length 29)
    _gateway.52701 > localhost.localdomain.59579: [udp sum ok] UDP, length 1
        0x0000:  5254 0012 3456 5255 0a00 0202 0800 4500
        0x0010:  001d b193 0000 4011 b12c 0a00 0202 0a00
        0x0020:  020f cddd e8bb 0009 c331 6e

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

#6 lore raw

RE: [PATCH net-next 1/2] virtio_net: Fix short frame length check

In reply to Alexander Duyck
Quoted 61 lines

From: Alexander Duyck <[email protected]>
Sent: Friday, January 13, 2023 7:24 PM

Quoted 35 lines · Parav Pandit
Quoted 35 lines

From: Alexander H Duyck <[email protected]>
Sent: Friday, January 13, 2023 6:24 PM

Quoted 18 lines · Parav Pandit

A smallest Ethernet frame defined by IEEE 802.3 is 60 bytes without any preemble and CRC.

Current code only checks for minimal 14 bytes of Ethernet header length. Correct it to consider the minimum Ethernet frame length.

Fixes 296f96fcfc16 ("Net driver using virtio")
---
 drivers/net/virtio_net.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/virtio_net.c +0 -0

index
7723b2a49d8e..d45e140b6852 100644

drivers/net/virtio_net.c static void receive_buf(struct virtnet_info · line 1248

Quoted 1 line
drivers/net/virtio_net.c +0 -0
@@ -1248,7 +1248,7 @@ static void receive_buf(struct virtnet_info
 *vi,

struct receive_queue *rq,

Quoted 8 lines
drivers/net/virtio_net.c +1 -1
    struct sk_buff *skb;
    struct virtio_net_hdr_mrg_rxbuf *hdr;
 
-   if (unlikely(len < vi->hdr_len + ETH_HLEN)) {
+   if (unlikely(len < vi->hdr_len + ETH_ZLEN)) {
            pr_debug("%s: short packet %i\n", dev->name, len);
            dev->stats.rx_length_errors++;
            if (vi->mergeable_rx_bufs) {
Quoted 3 lines
Quoted 3 lines

I'm not sure I agree with this change as packets are only 60B if they have gone across the wire as they are usually padded out on the transmit side. There may be cases where software routed packets may not

be 60B.

Quoted 2 lines
Quoted 1 line

Do you mean Linux kernel software? Any link to it would be helpful.

The problem is there are several software paths involved and that is why I am wanting to be cautious. As I recall this would impact Qemu itself, DPDK, the Linux Kernel and several others if I am not mistaken. That is why I am tending to err on the side of caution as this is a pretty significant change.

Quoted 4 lines
Quoted 4 lines

As such rather than changing out ETH_HLEN for ETH_ZLEN I wonder if we should look at maybe making this a "<=" comparison instead since that is the only case I can think of where the packet would end up being entirely empty after eth_type_trans is called and we would be passing

an skb with length 0.

Quoted 4 lines

I likely didn’t understand your comment.
This driver check is before creating the skb for the received packet. So, purpose is to not even process the packet header or prepare the skb if it

Quoted 7 lines

not an Ethernet frame.

Quoted 2 lines

It is interesting to know when we get < 60B frame.

If I recall, a UDPv4 frame can easily do it since Ethernet is 14B, IP header is 20, and UDP is only 8 so that only comes to 42B if I recall correctly. Similarly I think a TCPv4 Frame can be as small as 54B if you disable all the option headers.

Yes for sure < 60B Ethernet payload is very common which is usually padded by the nic tx. I am familiar with it. :)

I missed the part that when virtio is sw emulated, the tx short frame(not padded by stack) never left the sw stack. (never sent to the hw nic).
Hence, it was never padded, and it was looped back.
This will reach as short frame to virtio driver.

So yes, this patch breaks it. I will drop this patch. Thanks Alexander for the catch.

#7 lore raw

PATCH 2/2 virtio_net: Reuse buffer free function

In reply to Parav Pandit

virtnet_rq_free_unused_buf() helper function to free the buffer already exists. Avoid code duplication by reusing existing function.

---
 drivers/net/virtio_net.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
drivers/net/virtio_net.c +1 -7
index d45e140b6852..c1faaf11fbcd 100644
@@ -1251,13 +1251,7 @@ static void receive_buf(struct virtnet_info *vi, struct receive_queue *rq,
 	if (unlikely(len < vi->hdr_len + ETH_ZLEN)) {
 		pr_debug("%s: short packet %i\n", dev->name, len);
 		dev->stats.rx_length_errors++;
-		if (vi->mergeable_rx_bufs) {
-			put_page(virt_to_head_page(buf));
-		} else if (vi->big_packets) {
-			give_pages(rq, buf);
-		} else {
-			put_page(virt_to_head_page(buf));
-		}
+		virtnet_rq_free_unused_buf(rq->vq, buf);
 		return;
 	}
 
Signature · 1 line
2.26.2
Reviewed-by Alexander Duyck on v1 2/2
#8 lore raw

Re: [PATCH net-next 2/2] virtio_net: Reuse buffer free function

Quoted 17 lines · Parav Pandit

virtnet_rq_free_unused_buf() helper function to free the buffer already exists. Avoid code duplication by reusing existing function.

---
 drivers/net/virtio_net.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
drivers/net/virtio_net.c +0 -1
index d45e140b6852..c1faaf11fbcd 100644
@@ -1251,13 +1251,7 @@ static void receive_buf(struct virtnet_info *vi, struct receive_queue *rq,
 	if (unlikely(len < vi->hdr_len + ETH_ZLEN)) {
 		pr_debug("%s: short packet %i\n", dev->name, len);
 		dev->stats.rx_length_errors++;
-		if (vi->mergeable_rx_bufs) {

drivers/net/virtio_net.c static void receive_buf(struct virtnet_info *vi, struct receive_queue *rq, · line 1257

Quoted 10 lines
drivers/net/virtio_net.c +1 -6
-			put_page(virt_to_head_page(buf));
-		} else if (vi->big_packets) {
-			give_pages(rq, buf);
-		} else {
-			put_page(virt_to_head_page(buf));
-		}
+		virtnet_rq_free_unused_buf(rq->vq, buf);
 		return;
 	}
 

Looks good to me.

reviewer Reviewed-by on v1 2/2
#9 lore raw

Re: [PATCH net-next 2/2] virtio_net: Reuse buffer free function

In reply to Parav Pandit
Quoted 4 lines · Parav Pandit

virtnet_rq_free_unused_buf() helper function to free the buffer already exists. Avoid code duplication by reusing existing function.

Quoted 26 lines
---
 drivers/net/virtio_net.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
drivers/net/virtio_net.c +1 -8
index d45e140b6852..c1faaf11fbcd 100644
@@ -1251,13 +1251,7 @@ static void receive_buf(struct virtnet_info *vi, struct receive_queue *rq,
 	if (unlikely(len < vi->hdr_len + ETH_ZLEN)) {
 		pr_debug("%s: short packet %i\n", dev->name, len);
 		dev->stats.rx_length_errors++;
-		if (vi->mergeable_rx_bufs) {
-			put_page(virt_to_head_page(buf));
-		} else if (vi->big_packets) {
-			give_pages(rq, buf);
-		} else {
-			put_page(virt_to_head_page(buf));
-		}
+		virtnet_rq_free_unused_buf(rq->vq, buf);
 		return;
 	}
 
--

2.26.2

#10 lore raw
In reply to Parav Pandit

Hi Jakub, Dave,

Quoted 11 lines

From: Parav Pandit <[email protected]>
Sent: Friday, January 13, 2023 5:36 PM

Hi,

These two changes improve the small packet handling.

Patch summary:
patch-1 fixes the length check by considering Ethernet 60B frame size patch-2 avoids code duplication by reuses existing buffer free helper

Quoted 6 lines

Please review.

Parav Pandit (2):
virtio_net: Fix short frame length check
virtio_net: Reuse buffer free function

Please drop this series.
I will drop first patch as it was wrong.
Will send 2nd patch as v2 which was reviewed.