Segmentation fault in ixgbe_rxtx_vec.c:444 with 1.8.0

Hi everybody,

we quite recently updated one of our applications to DPDK 1.8.0 and are now seeing a segmentation fault in ixgbe_rxtx_vec.c:444 after a few minutes. I just did some quick debugging and I only have a very limited understanding of the code in question but it seems that the 'continue' in line 445 without increasing 'buf_idx' might cause the problem. In one debugging session when the crash occurred the value of 'buf_idx' was 2 and the value of 'pkt_idx' was 8965.
Any help with this issue would be greatly appreciated. If you need any further information just let me know.

Martin

Hi again,

I did some further testing and it seems like this issue is linked to jumbo frames. I think a similar issue has already been reported by Prashant Upadhyaya with the subject 'Packet Rx issue with DPDK1.8'. In our application we use the following rxmode port configuration:

.mq_mode = ETH_MQ_RX_RSS,
.split_hdr_size = 0,
.header_split = 0,
.hw_ip_checksum = 1,
.hw_vlan_filter = 0,
.jumbo_frame = 1,
.hw_strip_crc = 1,
.max_rx_pkt_len = 9000,

and the mbuf size is calculated like the following:

(2048 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)

This works fine with DPDK 1.7 and jumbo frames are split into buffer chains and can be forwarded on another port without a problem. With DPDK 1.8 and the default configuration (CONFIG_RTE_IXGBE_INC_VECTOR enabled) the application sometimes crashes like described in my first mail and sometimes packet receiving stops with subsequently arriving packets counted as rx errors. When CONFIG_RTE_IXGBE_INC_VECTOR is disabled the packet processing also comes to a halt as soon as jumbo frames arrive with a the slightly different effect that now rte_eth_tx_burst refuses to send any previously received packets.

Is there anything special to consider regarding jumbo frames when moving from DPDK 1.7 to 1.8 that we might have missed?

Martin

Quoted 15 lines · Martin Weiser

Hi everybody,

we quite recently updated one of our applications to DPDK 1.8.0 and are now seeing a segmentation fault in ixgbe_rxtx_vec.c:444 after a few minutes. I just did some quick debugging and I only have a very limited understanding of the code in question but it seems that the 'continue' in line 445 without increasing 'buf_idx' might cause the problem. In one debugging session when the crash occurred the value of 'buf_idx' was 2 and the value of 'pkt_idx' was 8965.
Any help with this issue would be greatly appreciated. If you need any further information just let me know.

Martin

Quoted 30 lines · Martin Weiser

Hi again,

I did some further testing and it seems like this issue is linked to jumbo frames. I think a similar issue has already been reported by Prashant Upadhyaya with the subject 'Packet Rx issue with DPDK1.8'. In our application we use the following rxmode port configuration:

.mq_mode = ETH_MQ_RX_RSS,
.split_hdr_size = 0,
.header_split = 0,
.hw_ip_checksum = 1,
.hw_vlan_filter = 0,
.jumbo_frame = 1,
.hw_strip_crc = 1,
.max_rx_pkt_len = 9000,

and the mbuf size is calculated like the following:

(2048 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)

This works fine with DPDK 1.7 and jumbo frames are split into buffer chains and can be forwarded on another port without a problem. With DPDK 1.8 and the default configuration (CONFIG_RTE_IXGBE_INC_VECTOR enabled) the application sometimes crashes like described in my first mail and sometimes packet receiving stops with subsequently arriving packets counted as rx errors. When CONFIG_RTE_IXGBE_INC_VECTOR is disabled the packet processing also comes to a halt as soon as jumbo frames arrive with a the slightly different effect that now rte_eth_tx_burst refuses to send any previously received packets.

Quoted 12 lines

Is there anything special to consider regarding jumbo frames when moving from DPDK 1.7 to 1.8 that we might have missed?

Martin

Quoted 4 lines · Martin Weiser

Hi everybody,

we quite recently updated one of our applications to DPDK 1.8.0 and are now seeing a segmentation fault in ixgbe_rxtx_vec.c:444 after a few

minutes.

Quoted 3 lines

Quoted 3 lines

I just did some quick debugging and I only have a very limited understanding of the code in question but it seems that the 'continue' in line 445 without increasing 'buf_idx' might cause the problem. In

one

Quoted 9 lines

Quoted 8 lines

debugging session when the crash occurred the value of 'buf_idx' was 2 and the value of 'pkt_idx' was 8965.
Any help with this issue would be greatly appreciated. If you need any further information just let me know.

Martin

Hi Martin, Prashant,

I've managed to reproduce the issue here and had a look at it. Could you both perhaps try the proposed change below and see if it fixes the problem for
you and gives you a working system? If so, I'll submit this as a patch fix officially - or go back to the drawing board, if not. :-)

lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +2 -2

index b54cb19..dfaccee 100644
@@ -402,10 +402,10 @@ reassemble_packets(struct igb_rx_queue *rxq, struct
 rte_mbuf **rx_bufs,
        struct rte_mbuf *pkts[RTE_IXGBE_VPMD_RX_BURST]; /*finished pkts*/
        struct rte_mbuf *start = rxq->pkt_first_seg;
        struct rte_mbuf *end =  rxq->pkt_last_seg;
-       unsigned pkt_idx = 0, buf_idx = 0;
+       unsigned pkt_idx, buf_idx;
 
 
-       while (buf_idx < nb_bufs) {
+       for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) {
                if (end != NULL) {
                        /* processing a split packet */

                        end->next = rx_bufs[buf_idx];

lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +0 -1

@@ -448,7 +448,6 @@ reassemble_packets(struct igb_rx_queue *rxq, struct
 rte_mbuf **rx_bufs,
                        rx_bufs[buf_idx]->data_len += rxq->crc_len;
                        rx_bufs[buf_idx]->pkt_len += rxq->crc_len;
                }
-               buf_idx++;
        }
 

        /* save the partial packet for next time */

Regards,
/Bruce

Hi Bruce,

Quoted 95 lines · Bruce Richardson

Quoted 30 lines · Martin Weiser

Hi again,

I did some further testing and it seems like this issue is linked to jumbo frames. I think a similar issue has already been reported by Prashant Upadhyaya with the subject 'Packet Rx issue with DPDK1.8'. In our application we use the following rxmode port configuration:

.mq_mode = ETH_MQ_RX_RSS,
.split_hdr_size = 0,
.header_split = 0,
.hw_ip_checksum = 1,
.hw_vlan_filter = 0,
.jumbo_frame = 1,
.hw_strip_crc = 1,
.max_rx_pkt_len = 9000,

and the mbuf size is calculated like the following:

(2048 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)

This works fine with DPDK 1.7 and jumbo frames are split into buffer chains and can be forwarded on another port without a problem. With DPDK 1.8 and the default configuration (CONFIG_RTE_IXGBE_INC_VECTOR enabled) the application sometimes crashes like described in my first mail and sometimes packet receiving stops with subsequently arriving packets counted as rx errors. When CONFIG_RTE_IXGBE_INC_VECTOR is disabled the packet processing also comes to a halt as soon as jumbo frames arrive with a the slightly different effect that now rte_eth_tx_burst refuses to send any previously received packets.

Quoted 12 lines

Is there anything special to consider regarding jumbo frames when moving from DPDK 1.7 to 1.8 that we might have missed?

Martin

Quoted 4 lines · Martin Weiser

Hi everybody,

we quite recently updated one of our applications to DPDK 1.8.0 and are now seeing a segmentation fault in ixgbe_rxtx_vec.c:444 after a few

minutes.

Quoted 3 lines

Quoted 3 lines

I just did some quick debugging and I only have a very limited understanding of the code in question but it seems that the 'continue' in line 445 without increasing 'buf_idx' might cause the problem. In

one

Quoted 9 lines

Quoted 8 lines

debugging session when the crash occurred the value of 'buf_idx' was 2 and the value of 'pkt_idx' was 8965.
Any help with this issue would be greatly appreciated. If you need any further information just let me know.

Martin

Hi Martin, Prashant,

I've managed to reproduce the issue here and had a look at it. Could you both perhaps try the proposed change below and see if it fixes the problem for
you and gives you a working system? If so, I'll submit this as a patch fix officially - or go back to the drawing board, if not. :-)

lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +2 -2

index b54cb19..dfaccee 100644
@@ -402,10 +402,10 @@ reassemble_packets(struct igb_rx_queue *rxq, struct
 rte_mbuf **rx_bufs,
        struct rte_mbuf *pkts[RTE_IXGBE_VPMD_RX_BURST]; /*finished pkts*/
        struct rte_mbuf *start = rxq->pkt_first_seg;
        struct rte_mbuf *end =  rxq->pkt_last_seg;
-       unsigned pkt_idx = 0, buf_idx = 0;
+       unsigned pkt_idx, buf_idx;
 
 
-       while (buf_idx < nb_bufs) {
+       for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) {
                if (end != NULL) {
                        /* processing a split packet */

                        end->next = rx_bufs[buf_idx];

lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +0 -1

@@ -448,7 +448,6 @@ reassemble_packets(struct igb_rx_queue *rxq, struct
 rte_mbuf **rx_bufs,
                        rx_bufs[buf_idx]->data_len += rxq->crc_len;
                        rx_bufs[buf_idx]->pkt_len += rxq->crc_len;
                }
-               buf_idx++;
        }
 

        /* save the partial packet for next time */

Quoted 4 lines

Regards,
/Bruce

Hi Bruce,

I am afraid your patch did not work for me. In my case I am not trying to receive jumbo frames but normal frames. They are not received at my application. Further, your patched function is not getting stimulated in my usecase.

Regards
-Prashant

Quoted 30 lines · Martin Weiser

Hi again,

I did some further testing and it seems like this issue is linked to jumbo frames. I think a similar issue has already been reported by Prashant Upadhyaya with the subject 'Packet Rx issue with DPDK1.8'. In our application we use the following rxmode port configuration:

.mq_mode = ETH_MQ_RX_RSS,
.split_hdr_size = 0,
.header_split = 0,
.hw_ip_checksum = 1,
.hw_vlan_filter = 0,
.jumbo_frame = 1,
.hw_strip_crc = 1,
.max_rx_pkt_len = 9000,

and the mbuf size is calculated like the following:

(2048 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)

This works fine with DPDK 1.7 and jumbo frames are split into buffer chains and can be forwarded on another port without a problem. With DPDK 1.8 and the default configuration (CONFIG_RTE_IXGBE_INC_VECTOR enabled) the application sometimes crashes like described in my first mail and sometimes packet receiving stops with subsequently arriving packets counted as rx errors. When CONFIG_RTE_IXGBE_INC_VECTOR is disabled the packet processing also comes to a halt as soon as jumbo frames arrive with a the slightly different effect that now rte_eth_tx_burst refuses to send any previously received packets.

Quoted 23 lines

Is there anything special to consider regarding jumbo frames when moving from DPDK 1.7 to 1.8 that we might have missed?

Martin

Quoted 15 lines · Martin Weiser

Hi everybody,

we quite recently updated one of our applications to DPDK 1.8.0 and are now seeing a segmentation fault in ixgbe_rxtx_vec.c:444 after a few minutes. I just did some quick debugging and I only have a very limited understanding of the code in question but it seems that the 'continue' in line 445 without increasing 'buf_idx' might cause the problem. In one debugging session when the crash occurred the value of 'buf_idx' was 2 and the value of 'pkt_idx' was 8965.
Any help with this issue would be greatly appreciated. If you need any further information just let me know.

Martin

Hi Martin, Prashant,

I've managed to reproduce the issue here and had a look at it. Could you both perhaps try the proposed change below and see if it fixes the problem for you and gives you a working system? If so, I'll submit this as a patch fix officially - or go back to the drawing board, if not. :-)

lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +2 -3

index b54cb19..dfaccee 100644
@@ -402,10 +402,10 @@ reassemble_packets(struct igb_rx_queue *rxq, struct rte_mbuf **rx_bufs,
        struct rte_mbuf *pkts[RTE_IXGBE_VPMD_RX_BURST]; /*finished pkts*/
        struct rte_mbuf *start = rxq->pkt_first_seg;
        struct rte_mbuf *end =  rxq->pkt_last_seg;
-       unsigned pkt_idx = 0, buf_idx = 0;
+       unsigned pkt_idx, buf_idx;
 
 
-       while (buf_idx < nb_bufs) {
+       for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) {
                if (end != NULL) {
                        /* processing a split packet */
                        end->next = rx_bufs[buf_idx];

@@ -448,7 +448,6 @@ reassemble_packets(struct igb_rx_queue *rxq, struct rte_mbuf **rx_bufs,
                        rx_bufs[buf_idx]->data_len += rxq->crc_len;
                        rx_bufs[buf_idx]->pkt_len += rxq->crc_len;
                }
-               buf_idx++;
        }
 
        /* save the partial packet for next time */

Regards,
/Bruce

Hi Bruce,

I now had the chance to reproduce the issue we are seeing with a DPDK example app.
I started out with a vanilla DPDK 1.8.0 and only made the following changes:

examples/l2fwd/main.c +2 -1

index e684234..48e6b7c 100644
@@ -118,8 +118,9 @@ static const struct rte_eth_conf port_conf = {
                .header_split   = 0, /**< Header Split disabled */
                .hw_ip_checksum = 0, /**< IP checksum offload disabled */
                .hw_vlan_filter = 0, /**< VLAN filtering disabled */
-               .jumbo_frame    = 0, /**< Jumbo Frame Support disabled */
+               .jumbo_frame    = 1, /**< Jumbo Frame Support disabled */
                .hw_strip_crc   = 0, /**< CRC stripped by hardware */
+               .max_rx_pkt_len = 9000,
        },
        .txmode = {
                .mq_mode = ETH_MQ_TX_NONE,

lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +2 -2

index b54cb19..dfaccee 100644
@@ -402,10 +402,10 @@ reassemble_packets(struct igb_rx_queue *rxq,
 struct rte_mbuf **rx_bufs,
        struct rte_mbuf *pkts[RTE_IXGBE_VPMD_RX_BURST]; /*finished pkts*/
        struct rte_mbuf *start = rxq->pkt_first_seg;
        struct rte_mbuf *end =  rxq->pkt_last_seg;
-       unsigned pkt_idx = 0, buf_idx = 0;
+       unsigned pkt_idx, buf_idx;
 
 
-       while (buf_idx < nb_bufs) {
+       for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) {
                if (end != NULL) {
                        /* processing a split packet */

                        end->next = rx_bufs[buf_idx];

lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +0 -1

@@ -448,7 +448,6 @@ reassemble_packets(struct igb_rx_queue *rxq, struct
 rte_mbuf **rx_bufs,
                        rx_bufs[buf_idx]->data_len += rxq->crc_len;
                        rx_bufs[buf_idx]->pkt_len += rxq->crc_len;
                }
-               buf_idx++;
        }
 

        /* save the partial packet for next time */

This includes your previously posted fix and makes a small modification to the l2fwd example app to enable jumbo frames of up to 9000 bytes. The system is equipped with a two port Intel 82599 card and both ports are hooked up to a packet generator. The packet generator produces simple Ethernet/IPv4/UDP packets.
I started the l2fwd app with the following command line:

$ ./build/l2fwd -c f -n 4 -- -q 8 -p 3

Both build variants that I have tested (CONFIG_RTE_IXGBE_INC_VECTOR=y and CONFIG_RTE_IXGBE_INC_VECTOR=n) now give me the same result:
As long as the packet size is <= 2048 bytes the application behaves normally and all packets are forwarded as expected.
As soon as the packet size exceeds 2048 bytes the application will only forward some packets and then stop forwarding altogether. Even small packets will not be forwarded anymore.

If you want me to try out anything else just let me know.

Best regards,
Martin

Quoted 91 lines · Martin Weiser

Quoted 30 lines

Hi again,

I did some further testing and it seems like this issue is linked to jumbo frames. I think a similar issue has already been reported by Prashant Upadhyaya with the subject 'Packet Rx issue with DPDK1.8'. In our application we use the following rxmode port configuration:

.mq_mode = ETH_MQ_RX_RSS,
.split_hdr_size = 0,
.header_split = 0,
.hw_ip_checksum = 1,
.hw_vlan_filter = 0,
.jumbo_frame = 1,
.hw_strip_crc = 1,
.max_rx_pkt_len = 9000,

and the mbuf size is calculated like the following:

(2048 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)

This works fine with DPDK 1.7 and jumbo frames are split into buffer chains and can be forwarded on another port without a problem. With DPDK 1.8 and the default configuration (CONFIG_RTE_IXGBE_INC_VECTOR enabled) the application sometimes crashes like described in my first mail and sometimes packet receiving stops with subsequently arriving packets counted as rx errors. When CONFIG_RTE_IXGBE_INC_VECTOR is disabled the packet processing also comes to a halt as soon as jumbo frames arrive with a the slightly different effect that now rte_eth_tx_burst refuses to send any previously received packets.

Quoted 23 lines

Is there anything special to consider regarding jumbo frames when moving from DPDK 1.7 to 1.8 that we might have missed?

Martin

Quoted 15 lines · Martin Weiser

Hi everybody,

we quite recently updated one of our applications to DPDK 1.8.0 and are now seeing a segmentation fault in ixgbe_rxtx_vec.c:444 after a few minutes. I just did some quick debugging and I only have a very limited understanding of the code in question but it seems that the 'continue' in line 445 without increasing 'buf_idx' might cause the problem. In one debugging session when the crash occurred the value of 'buf_idx' was 2 and the value of 'pkt_idx' was 8965.
Any help with this issue would be greatly appreciated. If you need any further information just let me know.

Martin

Hi Martin, Prashant,

I've managed to reproduce the issue here and had a look at it. Could you both perhaps try the proposed change below and see if it fixes the problem for you and gives you a working system? If so, I'll submit this as a patch fix officially - or go back to the drawing board, if not. :-)

lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +2 -3

index b54cb19..dfaccee 100644
@@ -402,10 +402,10 @@ reassemble_packets(struct igb_rx_queue *rxq, struct rte_mbuf **rx_bufs,
        struct rte_mbuf *pkts[RTE_IXGBE_VPMD_RX_BURST]; /*finished pkts*/
        struct rte_mbuf *start = rxq->pkt_first_seg;
        struct rte_mbuf *end =  rxq->pkt_last_seg;
-       unsigned pkt_idx = 0, buf_idx = 0;
+       unsigned pkt_idx, buf_idx;
 
 
-       while (buf_idx < nb_bufs) {
+       for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) {
                if (end != NULL) {
                        /* processing a split packet */
                        end->next = rx_bufs[buf_idx];

@@ -448,7 +448,6 @@ reassemble_packets(struct igb_rx_queue *rxq, struct rte_mbuf **rx_bufs,
                        rx_bufs[buf_idx]->data_len += rxq->crc_len;
                        rx_bufs[buf_idx]->pkt_len += rxq->crc_len;
                }
-               buf_idx++;
        }
 
        /* save the partial packet for next time */

Regards,
/Bruce

Quoted 68 lines · Martin Weiser

Hi Bruce,

I now had the chance to reproduce the issue we are seeing with a DPDK example app.
I started out with a vanilla DPDK 1.8.0 and only made the following changes:

examples/l2fwd/main.c +2 -1

index e684234..48e6b7c 100644
@@ -118,8 +118,9 @@ static const struct rte_eth_conf port_conf = {
                .header_split   = 0, /**< Header Split disabled */
                .hw_ip_checksum = 0, /**< IP checksum offload disabled */
                .hw_vlan_filter = 0, /**< VLAN filtering disabled */
-               .jumbo_frame    = 0, /**< Jumbo Frame Support disabled */
+               .jumbo_frame    = 1, /**< Jumbo Frame Support disabled */
                .hw_strip_crc   = 0, /**< CRC stripped by hardware */
+               .max_rx_pkt_len = 9000,
        },
        .txmode = {
                .mq_mode = ETH_MQ_TX_NONE,

lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +2 -2

index b54cb19..dfaccee 100644
@@ -402,10 +402,10 @@ reassemble_packets(struct igb_rx_queue *rxq,
 struct rte_mbuf **rx_bufs,
        struct rte_mbuf *pkts[RTE_IXGBE_VPMD_RX_BURST]; /*finished pkts*/
        struct rte_mbuf *start = rxq->pkt_first_seg;
        struct rte_mbuf *end =  rxq->pkt_last_seg;
-       unsigned pkt_idx = 0, buf_idx = 0;
+       unsigned pkt_idx, buf_idx;
 
 
-       while (buf_idx < nb_bufs) {
+       for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) {
                if (end != NULL) {
                        /* processing a split packet */

                        end->next = rx_bufs[buf_idx];

lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +0 -1

@@ -448,7 +448,6 @@ reassemble_packets(struct igb_rx_queue *rxq, struct
 rte_mbuf **rx_bufs,
                        rx_bufs[buf_idx]->data_len += rxq->crc_len;
                        rx_bufs[buf_idx]->pkt_len += rxq->crc_len;
                }
-               buf_idx++;
        }
 

        /* save the partial packet for next time */

This includes your previously posted fix and makes a small modification to the l2fwd example app to enable jumbo frames of up to 9000 bytes. The system is equipped with a two port Intel 82599 card and both ports are hooked up to a packet generator. The packet generator produces simple Ethernet/IPv4/UDP packets.
I started the l2fwd app with the following command line:

$ ./build/l2fwd -c f -n 4 -- -q 8 -p 3

Both build variants that I have tested (CONFIG_RTE_IXGBE_INC_VECTOR=y and CONFIG_RTE_IXGBE_INC_VECTOR=n) now give me the same result:
As long as the packet size is <= 2048 bytes the application behaves normally and all packets are forwarded as expected.
As soon as the packet size exceeds 2048 bytes the application will only forward some packets and then stop forwarding altogether. Even small packets will not be forwarded anymore.

Quoted 6 lines

If you want me to try out anything else just let me know.

Best regards,
Martin

I think the txq flags are at fault here. The default txq flags setting for the l2fwd sample application includes the flag ETH_TXQ_FLAGS_NOMULTSEGS which disables support for sending packets with multiple segments i.e. jumbo frames in this case. If you change l2fwd to explicitly pass a txqflags parameter in as part of the port setup (as was the case in previous releases), and set txqflags to 0, does the problem go away?

/Bruce

Quoted 93 lines · Bruce Richardson

Quoted 91 lines · Martin Weiser

Quoted 30 lines

Hi again,

I did some further testing and it seems like this issue is linked to jumbo frames. I think a similar issue has already been reported by Prashant Upadhyaya with the subject 'Packet Rx issue with DPDK1.8'. In our application we use the following rxmode port configuration:

.mq_mode = ETH_MQ_RX_RSS,
.split_hdr_size = 0,
.header_split = 0,
.hw_ip_checksum = 1,
.hw_vlan_filter = 0,
.jumbo_frame = 1,
.hw_strip_crc = 1,
.max_rx_pkt_len = 9000,

and the mbuf size is calculated like the following:

(2048 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)

This works fine with DPDK 1.7 and jumbo frames are split into buffer chains and can be forwarded on another port without a problem. With DPDK 1.8 and the default configuration (CONFIG_RTE_IXGBE_INC_VECTOR enabled) the application sometimes crashes like described in my first mail and sometimes packet receiving stops with subsequently arriving packets counted as rx errors. When CONFIG_RTE_IXGBE_INC_VECTOR is disabled the packet processing also comes to a halt as soon as jumbo frames arrive with a the slightly different effect that now rte_eth_tx_burst refuses to send any previously received packets.

Quoted 23 lines

Is there anything special to consider regarding jumbo frames when moving from DPDK 1.7 to 1.8 that we might have missed?

Martin

Quoted 15 lines · Martin Weiser

Hi everybody,

we quite recently updated one of our applications to DPDK 1.8.0 and are now seeing a segmentation fault in ixgbe_rxtx_vec.c:444 after a few minutes. I just did some quick debugging and I only have a very limited understanding of the code in question but it seems that the 'continue' in line 445 without increasing 'buf_idx' might cause the problem. In one debugging session when the crash occurred the value of 'buf_idx' was 2 and the value of 'pkt_idx' was 8965.
Any help with this issue would be greatly appreciated. If you need any further information just let me know.

Martin

Hi Martin, Prashant,

I've managed to reproduce the issue here and had a look at it. Could you both perhaps try the proposed change below and see if it fixes the problem for you and gives you a working system? If so, I'll submit this as a patch fix officially - or go back to the drawing board, if not. :-)

lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +2 -3

index b54cb19..dfaccee 100644
@@ -402,10 +402,10 @@ reassemble_packets(struct igb_rx_queue *rxq, struct rte_mbuf **rx_bufs,
        struct rte_mbuf *pkts[RTE_IXGBE_VPMD_RX_BURST]; /*finished pkts*/
        struct rte_mbuf *start = rxq->pkt_first_seg;
        struct rte_mbuf *end =  rxq->pkt_last_seg;
-       unsigned pkt_idx = 0, buf_idx = 0;
+       unsigned pkt_idx, buf_idx;
 
 
-       while (buf_idx < nb_bufs) {
+       for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) {
                if (end != NULL) {
                        /* processing a split packet */
                        end->next = rx_bufs[buf_idx];

@@ -448,7 +448,6 @@ reassemble_packets(struct igb_rx_queue *rxq, struct rte_mbuf **rx_bufs,
                        rx_bufs[buf_idx]->data_len += rxq->crc_len;
                        rx_bufs[buf_idx]->pkt_len += rxq->crc_len;
                }
-               buf_idx++;
        }
 
        /* save the partial packet for next time */

Regards,
/Bruce

When the vector pmd was receiving a mix of packets of various sizes, some of which were split across multiple mbufs, there was an issue with reassembly of the jumbo frames. This was due to a skipped increment when using "continue" in a while loop. Changing the loop to a "for" loop fixes this problem, by ensuring the increment is always performed.

Reported-by Prashant Upadhyaya <[email protected]>

Reported-by Martin Weiser <[email protected]>

Signed-off-by Bruce Richardson <[email protected]>

Tested-by Martin Weiser <[email protected]>