Skip to contentSkip to conversation

ip_frag: add IPv4 fragment copy packet API

PATCH Superseded Huichao Cai 5 participants 26 messages Jun 9, 2022 – Aug 29, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ip_frag: add IPv4 fragment copy packet API

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool and has refcnt = 1) offload. In order to adapt to this offload function, add this API. Add some test data for this API.

---
 app/test/test_ipfrag.c               |   8 +-
 lib/ip_frag/rte_ip_frag.h            |  26 ++++++
 lib/ip_frag/rte_ipv4_fragmentation.c | 165 +++++++++++++++++++++++++++++++++++
 lib/ip_frag/version.map              |   1 +
 4 files changed, 199 insertions(+), 1 deletion(-)
app/test/test_ipfrag.c +7 -1
index 610a86b..e565e86 100644
@@ -408,10 +408,16 @@ static void ut_teardown(void)
 		}
 
 		if (tests[i].ipv == 4)
-			len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
+			if (i % 2)
+				len = rte_ipv4_fragment_packet(b, pkts_out,
+						       BURST,
 						       tests[i].mtu_size,
 						       direct_pool,
 						       indirect_pool);
+			else
+				len = rte_ipv4_fragment_copy_packet(b, pkts_out,
+							   BURST,
+							   tests[i].mtu_size);
 		else if (tests[i].ipv == 6)
 			len = rte_ipv6_fragment_packet(b, pkts_out, BURST,
 						       tests[i].mtu_size,
lib/ip_frag/rte_ip_frag.h +26 -0
index 7d2abe1..767320b 100644
@@ -179,6 +179,32 @@ int32_t rte_ipv4_fragment_packet(struct rte_mbuf *pkt_in,
 			struct rte_mempool *pool_indirect);
 
 /**
+ * IPv4 fragmentation by copy.
+ *
+ * This function implements the fragmentation of IPv4 packets by copy.
+ *
+ * @param pkt_in
+ *   The input packet.
+ * @param pkts_out
+ *   Array storing the output fragments.
+ * @param nb_pkts_out
+ *   The size of the array that stores the output fragments.
+ * @param mtu_size
+ *   Size in bytes of the Maximum Transfer Unit (MTU) for the outgoing IPv4
+ *   datagrams. This value includes the size of the IPv4 header.
+ * @return
+ *   Upon successful completion - number of output fragments placed
+ *   in the pkts_out array.
+ *   Otherwise - (-1) * <errno>.
+ */
+__rte_experimental
+int32_t
+rte_ipv4_fragment_copy_packet(struct rte_mbuf *pkt_in,
+	struct rte_mbuf **pkts_out,
+	uint16_t nb_pkts_out,
+	uint16_t mtu_size);
+
+/**
  * This function implements reassembly of fragmented IPv4 packets.
  * Incoming mbufs should have its l2_len/l3_len fields setup correctly.
  *
lib/ip_frag/rte_ipv4_fragmentation.c +165 -0

168 lines not on this page · raw message

lib/ip_frag/version.map +1 -0
index e537224..4aa66bc 100644
@@ -17,4 +17,5 @@ EXPERIMENTAL {
 	global:
 
 	rte_ip_frag_table_del_expired_entries;
+	rte_ipv4_fragment_copy_packet;
 };
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH v2 1/1 ip_frag: add IPv4 fragment copy packet API

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool and has refcnt = 1) offload. In order to adapt to this offload function, add this API. Add some test data for this API.

---
 app/test/test_ipfrag.c               |   8 +-
 lib/ip_frag/rte_ip_frag.h            |  26 ++++++
 lib/ip_frag/rte_ipv4_fragmentation.c | 165 +++++++++++++++++++++++++++++++++++
 lib/ip_frag/version.map              |   1 +
 4 files changed, 199 insertions(+), 1 deletion(-)
app/test/test_ipfrag.c +7 -1
index 610a86b..e565e86 100644
@@ -408,10 +408,16 @@ static void ut_teardown(void)
 		}
 
 		if (tests[i].ipv == 4)
-			len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
+			if (i % 2)
+				len = rte_ipv4_fragment_packet(b, pkts_out,
+						       BURST,
 						       tests[i].mtu_size,
 						       direct_pool,
 						       indirect_pool);
+			else
+				len = rte_ipv4_fragment_copy_packet(b, pkts_out,
+							   BURST,
+							   tests[i].mtu_size);
 		else if (tests[i].ipv == 6)
 			len = rte_ipv6_fragment_packet(b, pkts_out, BURST,
 						       tests[i].mtu_size,
lib/ip_frag/rte_ip_frag.h +26 -0
index 7d2abe1..3e1a9ac 100644
@@ -179,6 +179,32 @@ int32_t rte_ipv4_fragment_packet(struct rte_mbuf *pkt_in,
 			struct rte_mempool *pool_indirect);
 
 /**
+ * IPv4 fragmentation by copy.
+ *
+ * This function implements the fragmentation of IPv4 packets by copy.
+ *
+ * @param pkt_in
+ *   The input packet.
+ * @param pkts_out
+ *   Array storing the output fragments.
+ * @param nb_pkts_out
+ *   The size of the array that stores the output fragments.
+ * @param mtu_size
+ *   Size in bytes of the Maximum Transfer Unit (MTU) for the outgoing IPv4
+ *   datagrams. This value includes the size of the IPv4 header.
+ * @return
+ *   Upon successful completion - number of output fragments placed
+ *   in the pkts_out array.
+ *   Otherwise - (-1) * errno.
+ */
+__rte_experimental
+int32_t
+rte_ipv4_fragment_copy_packet(struct rte_mbuf *pkt_in,
+	struct rte_mbuf **pkts_out,
+	uint16_t nb_pkts_out,
+	uint16_t mtu_size);
+
+/**
  * This function implements reassembly of fragmented IPv4 packets.
  * Incoming mbufs should have its l2_len/l3_len fields setup correctly.
  *
lib/ip_frag/rte_ipv4_fragmentation.c +165 -0

168 lines not on this page · raw message

lib/ip_frag/version.map +1 -0
index e537224..4aa66bc 100644
@@ -17,4 +17,5 @@ EXPERIMENTAL {
 	global:
 
 	rte_ip_frag_table_del_expired_entries;
+	rte_ipv4_fragment_copy_packet;
 };
Signature · 1 line
1.8.3.1
#2 lore raw
Quoted 8 lines

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool and has refcnt = 1) offload. In order to adapt to this offload function, add this API. Add some test data for this API.

---

...

Quoted 126 lines
lib/ip_frag/rte_ip_frag.h +12 -0
@@ -179,6 +179,32 @@ int32_t rte_ipv4_fragment_packet(struct rte_mbuf
 *pkt_in,
              struct rte_mempool *pool_indirect);
 
  /**
+ * IPv4 fragmentation by copy.
+ *
+ * This function implements the fragmentation of IPv4 packets by copy.
+ *
+ * @param pkt_in
+ *   The input packet.
+ * @param pkts_out
+ *   Array storing the output fragments.
+ * @param nb_pkts_out
+ *   The size of the array that stores the output fragments.
+ * @param mtu_size
+ *   Size in bytes of the Maximum Transfer Unit (MTU) for the outgoing

IPv4
+ * datagrams. This value includes the size of the IPv4 header.
+ * @return

+ *   Upon successful completion - number of output fragments placed
+ *   in the pkts_out array.
+ *   Otherwise - (-1) * errno.

+ */

lib/ip_frag/rte_ip_frag.h +8 -0
+__rte_experimental
+int32_t
+rte_ipv4_fragment_copy_packet(struct rte_mbuf *pkt_in,
+    struct rte_mbuf **pkts_out,
+    uint16_t nb_pkts_out,
+    uint16_t mtu_size);
+
+/**
   * This function implements reassembly of fragmented IPv4 packets.
   * Incoming mbufs should have its l2_len/l3_len fields setup correctly.
   *
lib/ip_frag/rte_ipv4_fragmentation.c +14 -0
index a562424..9e050cc 100644
@@ -262,3 +262,168 @@ static inline uint16_t
 __create_ipopt_frag_hdr(uint8_t *iph,
 
      return out_pkt_pos;
  }
+
+/**
+ * IPv4 fragmentation by copy.
+ *
+ * This function implements the fragmentation of IPv4 packets by copy.
+ *
+ * @param pkt_in
+ *   The input packet.
+ * @param pkts_out
+ *   Array storing the output fragments.
+ * @param nb_pkts_out
+ *   The size of the array that stores the output fragments.
+ * @param mtu_size
+ *   Size in bytes of the Maximum Transfer Unit (MTU) for the outgoing

IPv4
+ * datagrams. This value includes the size of the IPv4 header.
+ * @return

+ *   Upon successful completion - number of output fragments placed
+ *   in the pkts_out array.
+ *   Otherwise - (-1) * errno.

+ */

lib/ip_frag/rte_ipv4_fragmentation.c +58 -0
+int32_t
+rte_ipv4_fragment_copy_packet(struct rte_mbuf *pkt_in,
+    struct rte_mbuf **pkts_out,
+    uint16_t nb_pkts_out,
+    uint16_t mtu_size)
+{
+    struct rte_mbuf *in_seg = NULL;
+    struct rte_ipv4_hdr *in_hdr;
+    uint32_t out_pkt_pos, in_seg_data_pos;
+    uint32_t more_in_segs;
+    uint16_t fragment_offset, flag_offset, frag_size, header_len;
+    uint16_t frag_bytes_remaining;
+    uint8_t ipopt_frag_hdr[IPV4_HDR_MAX_LEN];
+    uint16_t ipopt_len;
+
+    /*
+     * Formal parameter checking.
+     */
+    if (unlikely(pkt_in == NULL) || unlikely(pkts_out == NULL) ||
+        unlikely(nb_pkts_out == 0) ||
+        unlikely(pkt_in->pool == NULL) ||
+        unlikely(mtu_size < RTE_ETHER_MIN_MTU))
+        return -EINVAL;
+
+    in_hdr = rte_pktmbuf_mtod(pkt_in, struct rte_ipv4_hdr *);
+    header_len = (in_hdr->version_ihl & RTE_IPV4_HDR_IHL_MASK) *
+        RTE_IPV4_IHL_MULTIPLIER;
+
+    /* Check IP header length */
+    if (unlikely(pkt_in->data_len < header_len) ||
+        unlikely(mtu_size < header_len))
+        return -EINVAL;
+
+    /*
+     * Ensure the IP payload length of all fragments is aligned to a
+     * multiple of 8 bytes as per RFC791 section 2.3.
+     */
+    frag_size = RTE_ALIGN_FLOOR((mtu_size - header_len),
+                    IPV4_HDR_FO_ALIGN);
+
+    flag_offset = rte_cpu_to_be_16(in_hdr->fragment_offset);
+
+    /* If Don't Fragment flag is set */
+    if (unlikely((flag_offset & IPV4_HDR_DF_MASK) != 0))
+        return -ENOTSUP;
+
+    /* Check that pkts_out is big enough to hold all fragments */
+    if (unlikely(frag_size * nb_pkts_out <
+        (uint16_t)(pkt_in->pkt_len - header_len)))
+        return -EINVAL;
+
+    in_seg = pkt_in;
+    in_seg_data_pos = header_len;
+    out_pkt_pos = 0;
+    fragment_offset = 0;
+
+    ipopt_len = header_len - sizeof(struct rte_ipv4_hdr);
+    if (unlikely(ipopt_len > RTE_IPV4_HDR_OPT_MAX_LEN))

lib/ip_frag/rte_ipv4_fragmentation.c static inline uint16_t __create_ipopt_frag_hdr(uint8_t *iph, · line 352

Quoted 10 lines
lib/ip_frag/rte_ipv4_fragmentation.c +10 -0
+        return -EINVAL;
+
+    more_in_segs = 1;
+    while (likely(more_in_segs)) {
+        struct rte_mbuf *out_pkt = NULL;
+        uint32_t more_out_segs;
+        struct rte_ipv4_hdr *out_hdr;
+
+        /* Allocate buffer from pkt_in->pool*/
+        out_pkt = rte_pktmbuf_alloc(pkt_in->pool);

Instead of implicitly assuming that output mbufs will be allocated from pkt_in pool, it would be better to have output_pool as explicit parameter for that function.
In a same way we have it for rte_ipv4_fragment_packet().

lib/ip_frag/rte_ipv4_fragmentation.c static inline uint16_t __create_ipopt_frag_hdr(uint8_t *iph, · line 364

Quoted 12 lines
lib/ip_frag/rte_ipv4_fragmentation.c +12 -0
+        if (unlikely(out_pkt == NULL)) {
+            __free_fragments(pkts_out, out_pkt_pos);
+            return -ENOMEM;
+        }
+
+        /* Reserve space for the IP header that will be built later */
+        out_pkt->data_len = header_len;
+        out_pkt->pkt_len = header_len;
+        frag_bytes_remaining = frag_size;
+
+        more_out_segs = 1;
+        while (likely(more_out_segs && more_in_segs)) {

If I understand correctly, here you assume that out_pkt will always
be big enough to hold entire fragment, right?
But that can not always be the case and probably we shouldn't assume
that for generic function.
I suppose safest way would be either use rte_pktmbuf_copy() here directly or do something similar to what that function doing ourselves here.

Quoted 75 lines
lib/ip_frag/rte_ipv4_fragmentation.c +65 -0
+            uint32_t len;
+
+            len = frag_bytes_remaining;
+            if (len > (in_seg->data_len - in_seg_data_pos))
+                len = in_seg->data_len - in_seg_data_pos;
+
+            rte_memcpy(
+                rte_pktmbuf_mtod_offset(
+                    out_pkt, char *, out_pkt->pkt_len),
+                rte_pktmbuf_mtod_offset(
+                    in_seg, char *, in_seg_data_pos),
+                len);
+            out_pkt->data_len = (uint16_t)(len +
+                out_pkt->data_len);
+
+            out_pkt->pkt_len = (uint16_t)(len +
+                out_pkt->pkt_len);
+            in_seg_data_pos += len;
+            frag_bytes_remaining -= len;
+
+            /* Current output packet (i.e. fragment) done ? */
+            if (unlikely(frag_bytes_remaining == 0))
+                more_out_segs = 0;
+
+            /* Current input segment done ? */
+            if (unlikely(in_seg_data_pos == in_seg->data_len)) {
+                in_seg = in_seg->next;
+                in_seg_data_pos = 0;
+
+                if (unlikely(in_seg == NULL))
+                    more_in_segs = 0;
+            }
+        }
+
+        /* Build the IP header */
+
+        out_hdr = rte_pktmbuf_mtod(out_pkt, struct rte_ipv4_hdr *);
+
+        __fill_ipv4hdr_frag(out_hdr, in_hdr, header_len,
+            (uint16_t)out_pkt->pkt_len,
+            flag_offset, fragment_offset, more_in_segs);
+
+        if (unlikely((fragment_offset == 0) && (ipopt_len) &&
+                ((flag_offset & RTE_IPV4_HDR_OFFSET_MASK) == 0))) {
+            ipopt_len = __create_ipopt_frag_hdr((uint8_t *)in_hdr,
+                ipopt_len, ipopt_frag_hdr);
+            fragment_offset = (uint16_t)(fragment_offset +
+                out_pkt->pkt_len - header_len);
+            out_pkt->l3_len = header_len;
+
+            header_len = sizeof(struct rte_ipv4_hdr) + ipopt_len;
+            in_hdr = (struct rte_ipv4_hdr *)ipopt_frag_hdr;
+        } else {
+            fragment_offset = (uint16_t)(fragment_offset +
+                out_pkt->pkt_len - header_len);
+            out_pkt->l3_len = header_len;
+        }
+
+        /* Write the fragment to the output list */
+        pkts_out[out_pkt_pos] = out_pkt;
+        out_pkt_pos++;
+    }
+
+    return out_pkt_pos;
+}
lib/ip_frag/version.map +1 -0
index e537224..4aa66bc 100644
@@ -17,4 +17,5 @@ EXPERIMENTAL {
      global:
 
      rte_ip_frag_table_del_expired_entries;
+    rte_ipv4_fragment_copy_packet;
  };
#3 lore raw

11/07/2022 00:35, Konstantin Ananyev пишет:

Quoted 77 lines
Quoted 8 lines

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool and has refcnt = 1) offload. In order to adapt to this offload function, add this API. Add some test data for this API.

---

...

Quoted 34 lines
lib/ip_frag/rte_ip_frag.h +12 -0
@@ -179,6 +179,32 @@ int32_t rte_ipv4_fragment_packet(struct rte_mbuf
 *pkt_in,
                struct rte_mempool *pool_indirect);
 
    /**
+ * IPv4 fragmentation by copy.
+ *
+ * This function implements the fragmentation of IPv4 packets by copy.
+ *
+ * @param pkt_in
+ *   The input packet.
+ * @param pkts_out
+ *   Array storing the output fragments.
+ * @param nb_pkts_out
+ *   The size of the array that stores the output fragments.
+ * @param mtu_size
+ *   Size in bytes of the Maximum Transfer Unit (MTU) for the outgoing

IPv4
+ *   datagrams. This value includes the size of the IPv4 header.
+ * @return
+ *   Upon successful completion - number of output fragments placed
+ *   in the pkts_out array.
+ *   Otherwise - (-1) * errno.
+ */

lib/ip_frag/rte_ip_frag.h +8 -0
+__rte_experimental
+int32_t
+rte_ipv4_fragment_copy_packet(struct rte_mbuf *pkt_in,
+    struct rte_mbuf **pkts_out,
+    uint16_t nb_pkts_out,
+    uint16_t mtu_size);
+
+/**
Quoted 2 lines

    * This function implements reassembly of fragmented IPv4 packets.
    * Incoming mbufs should have its l2_len/l3_len fields setup

correctly.

Quoted 29 lines

    *

lib/ip_frag/rte_ipv4_fragmentation.c +0 -0
index a562424..9e050cc 100644
@@ -262,3 +262,168 @@ static inline uint16_t
 __create_ipopt_frag_hdr(uint8_t *iph,
 
        return out_pkt_pos;

   }

lib/ip_frag/rte_ipv4_fragmentation.c +14 -0
+
+/**
+ * IPv4 fragmentation by copy.
+ *
+ * This function implements the fragmentation of IPv4 packets by copy.
+ *
+ * @param pkt_in
+ *   The input packet.
+ * @param pkts_out
+ *   Array storing the output fragments.
+ * @param nb_pkts_out
+ *   The size of the array that stores the output fragments.
+ * @param mtu_size
+ *   Size in bytes of the Maximum Transfer Unit (MTU) for the outgoing

IPv4
+ *   datagrams. This value includes the size of the IPv4 header.
+ * @return
+ *   Upon successful completion - number of output fragments placed

Quoted 8 lines

lib/ip_frag/rte_ipv4_fragmentation.c static inline uint16_t __create_ipopt_frag_hdr(uint8_t *iph, · line 289

Quoted 8 lines

+ *   in the pkts_out array.
+ *   Otherwise - (-1) * errno.
+ */

lib/ip_frag/rte_ipv4_fragmentation.c +5 -0
+int32_t
+rte_ipv4_fragment_copy_packet(struct rte_mbuf *pkt_in,
+    struct rte_mbuf **pkts_out,
+    uint16_t nb_pkts_out,
+    uint16_t mtu_size)

Forgot to mention, new API has to be experimental.

Quoted 166 lines
Quoted 53 lines
lib/ip_frag/rte_ipv4_fragmentation.c +53 -0
+{
+    struct rte_mbuf *in_seg = NULL;
+    struct rte_ipv4_hdr *in_hdr;
+    uint32_t out_pkt_pos, in_seg_data_pos;
+    uint32_t more_in_segs;
+    uint16_t fragment_offset, flag_offset, frag_size, header_len;
+    uint16_t frag_bytes_remaining;
+    uint8_t ipopt_frag_hdr[IPV4_HDR_MAX_LEN];
+    uint16_t ipopt_len;
+
+    /*
+     * Formal parameter checking.
+     */
+    if (unlikely(pkt_in == NULL) || unlikely(pkts_out == NULL) ||
+        unlikely(nb_pkts_out == 0) ||
+        unlikely(pkt_in->pool == NULL) ||
+        unlikely(mtu_size < RTE_ETHER_MIN_MTU))
+        return -EINVAL;
+
+    in_hdr = rte_pktmbuf_mtod(pkt_in, struct rte_ipv4_hdr *);
+    header_len = (in_hdr->version_ihl & RTE_IPV4_HDR_IHL_MASK) *
+        RTE_IPV4_IHL_MULTIPLIER;
+
+    /* Check IP header length */
+    if (unlikely(pkt_in->data_len < header_len) ||
+        unlikely(mtu_size < header_len))
+        return -EINVAL;
+
+    /*
+     * Ensure the IP payload length of all fragments is aligned to a
+     * multiple of 8 bytes as per RFC791 section 2.3.
+     */
+    frag_size = RTE_ALIGN_FLOOR((mtu_size - header_len),
+                    IPV4_HDR_FO_ALIGN);
+
+    flag_offset = rte_cpu_to_be_16(in_hdr->fragment_offset);
+
+    /* If Don't Fragment flag is set */
+    if (unlikely((flag_offset & IPV4_HDR_DF_MASK) != 0))
+        return -ENOTSUP;
+
+    /* Check that pkts_out is big enough to hold all fragments */
+    if (unlikely(frag_size * nb_pkts_out <
+        (uint16_t)(pkt_in->pkt_len - header_len)))
+        return -EINVAL;
+
+    in_seg = pkt_in;
+    in_seg_data_pos = header_len;
+    out_pkt_pos = 0;
+    fragment_offset = 0;
+
+    ipopt_len = header_len - sizeof(struct rte_ipv4_hdr);
+    if (unlikely(ipopt_len > RTE_IPV4_HDR_OPT_MAX_LEN))

lib/ip_frag/rte_ipv4_fragmentation.c static inline uint16_t __create_ipopt_frag_hdr(uint8_t *iph, · line 352

Quoted 10 lines
lib/ip_frag/rte_ipv4_fragmentation.c +10 -0
+        return -EINVAL;
+
+    more_in_segs = 1;
+    while (likely(more_in_segs)) {
+        struct rte_mbuf *out_pkt = NULL;
+        uint32_t more_out_segs;
+        struct rte_ipv4_hdr *out_hdr;
+
+        /* Allocate buffer from pkt_in->pool*/
+        out_pkt = rte_pktmbuf_alloc(pkt_in->pool);

Instead of implicitly assuming that output mbufs will be allocated from pkt_in pool, it would be better to have output_pool as explicit parameter for that function.
In a same way we have it for rte_ipv4_fragment_packet().

lib/ip_frag/rte_ipv4_fragmentation.c static inline uint16_t __create_ipopt_frag_hdr(uint8_t *iph, · line 364

Quoted 12 lines

+        if (unlikely(out_pkt == NULL)) {
+            __free_fragments(pkts_out, out_pkt_pos); +            return -ENOMEM;
+        }

lib/ip_frag/rte_ipv4_fragmentation.c +8 -0
+
+        /* Reserve space for the IP header that will be built later */
+        out_pkt->data_len = header_len;
+        out_pkt->pkt_len = header_len;
+        frag_bytes_remaining = frag_size;
+
+        more_out_segs = 1;
+        while (likely(more_out_segs && more_in_segs)) {

If I understand correctly, here you assume that out_pkt will always
be big enough to hold entire fragment, right?
But that can not always be the case and probably we shouldn't assume
that for generic function.
I suppose safest way would be either use rte_pktmbuf_copy() here directly or do something similar to what that function doing ourselves here.

Quoted 75 lines

+            uint32_t len;

lib/ip_frag/rte_ipv4_fragmentation.c +64 -0
+
+            len = frag_bytes_remaining;
+            if (len > (in_seg->data_len - in_seg_data_pos))
+                len = in_seg->data_len - in_seg_data_pos;
+
+            rte_memcpy(
+                rte_pktmbuf_mtod_offset(
+                    out_pkt, char *, out_pkt->pkt_len),
+                rte_pktmbuf_mtod_offset(
+                    in_seg, char *, in_seg_data_pos),
+                len);
+            out_pkt->data_len = (uint16_t)(len +
+                out_pkt->data_len);
+
+            out_pkt->pkt_len = (uint16_t)(len +
+                out_pkt->pkt_len);
+            in_seg_data_pos += len;
+            frag_bytes_remaining -= len;
+
+            /* Current output packet (i.e. fragment) done ? */
+            if (unlikely(frag_bytes_remaining == 0))
+                more_out_segs = 0;
+
+            /* Current input segment done ? */
+            if (unlikely(in_seg_data_pos == in_seg->data_len)) {
+                in_seg = in_seg->next;
+                in_seg_data_pos = 0;
+
+                if (unlikely(in_seg == NULL))
+                    more_in_segs = 0;
+            }
+        }
+
+        /* Build the IP header */
+
+        out_hdr = rte_pktmbuf_mtod(out_pkt, struct rte_ipv4_hdr *);
+
+        __fill_ipv4hdr_frag(out_hdr, in_hdr, header_len,
+            (uint16_t)out_pkt->pkt_len,
+            flag_offset, fragment_offset, more_in_segs);
+
+        if (unlikely((fragment_offset == 0) && (ipopt_len) &&
+                ((flag_offset & RTE_IPV4_HDR_OFFSET_MASK) == 0))) {
+            ipopt_len = __create_ipopt_frag_hdr((uint8_t *)in_hdr,
+                ipopt_len, ipopt_frag_hdr);
+            fragment_offset = (uint16_t)(fragment_offset +
+                out_pkt->pkt_len - header_len);
+            out_pkt->l3_len = header_len;
+
+            header_len = sizeof(struct rte_ipv4_hdr) + ipopt_len;
+            in_hdr = (struct rte_ipv4_hdr *)ipopt_frag_hdr;
+        } else {
+            fragment_offset = (uint16_t)(fragment_offset +
+                out_pkt->pkt_len - header_len);
+            out_pkt->l3_len = header_len;
+        }
+
+        /* Write the fragment to the output list */
+        pkts_out[out_pkt_pos] = out_pkt;
+        out_pkt_pos++;
+    }
+
+    return out_pkt_pos;
+}
lib/ip_frag/version.map +0 -0
index e537224..4aa66bc 100644
@@ -17,4 +17,5 @@ EXPERIMENTAL {

       global:

       rte_ip_frag_table_del_expired_entries;
+    rte_ipv4_fragment_copy_packet;
   };

#4 lore raw

Hi Konstantin,

I've been busy lately, sorry to reply to you late.

> Instead of implicitly assuming that output mbufs will be allocated

Quoted 6 lines

from pkt_in pool, it would be better to have output_pool as explicit parameter for that function.

In a same way we have it for rte_ipv4_fragment_packet().

Quoted 6 lines

If I understand correctly, here you assume that out_pkt will always be big enough to hold entire fragment, right?
But that can not always be the case and probably we shouldn't assume that for generic function.
I suppose safest way would be either use rte_pktmbuf_copy() here directly or do something similar to what that function doing ourselves here.

reply: Thanks for the reminder, I will use explicit parameters and rte_pktmbuf_copy();

Quoted 1 line

Forgot to mention, new API has to be experimental.

reply: Does this mean adding _rte_experimental when declaring a function?

Huichao,Cai

#5 lore raw

Hi Huichao,

Quoted 2 lines

    I've been busy lately, sorry to reply to you late.

No worries at all.

Quoted 14 lines
Quoted 7 lines

Instead of implicitly assuming that output mbufs will be allocated

from pkt_in pool, it would be better to have output_pool as explicit parameter for that function.
In a same way we have it for rte_ipv4_fragment_packet().

Quoted 6 lines

If I understand correctly, here you assume that out_pkt will always be big enough to hold entire fragment, right?
But that can not always be the case and probably we shouldn't assume that for generic function.
I suppose safest way would be either use rte_pktmbuf_copy() here directly or do something similar to what that function doing ourselves here.

Quoted 5 lines

reply: Thanks for the reminder, I will use explicit parameters and rte_pktmbuf_copy();

Quoted 1 line

Forgot to mention, new API has to be experimental.

reply: Does this mean adding _rte_experimental when declaring a function?

Yes, I meant _rte_experimental tag.
Thanks
Konstantin

#6 lore raw

PATCH v3 1/1 ip_frag: add IPv4 fragment copy packet API

In reply to Huichao Cai

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool and has refcnt = 1) offload. In order to adapt to this offload function, add this API. Add some test data for this API.

---
 app/test/test_ipfrag.c               |   8 +-
 lib/ip_frag/rte_ip_frag.h            |  27 +++++
 lib/ip_frag/rte_ipv4_fragmentation.c | 208 +++++++++++++++++++++++++++++++++++
 lib/ip_frag/version.map              |   1 +
 4 files changed, 243 insertions(+), 1 deletion(-)
app/test/test_ipfrag.c +7 -1
index ba0ffd0..bb7c4d3 100644
@@ -418,10 +418,16 @@ static void ut_teardown(void)
 		}
 
 		if (tests[i].ipv == 4)
-			len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
+			if (i % 2)
+				len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
 						       tests[i].mtu_size,
 						       direct_pool,
 						       indirect_pool);
+			else
+				len = rte_ipv4_fragment_copy_packet(b, pkts_out,
+						       BURST,
+						       tests[i].mtu_size,
+						       pkt_pool);
 		else if (tests[i].ipv == 6)
 			len = rte_ipv6_fragment_packet(b, pkts_out, BURST,
 						       tests[i].mtu_size,
lib/ip_frag/rte_ip_frag.h +27 -0
index 7d2abe1..c2a7e1e 100644
@@ -179,6 +179,33 @@ int32_t rte_ipv4_fragment_packet(struct rte_mbuf *pkt_in,
 			struct rte_mempool *pool_indirect);
 
 /**
+ * IPv4 fragmentation by copy.
+ *
+ * This function implements the fragmentation of IPv4 packets by copy.
+ *
+ * @param pkt_in
+ *   The input packet.
+ * @param pkts_out
+ *   Array storing the output fragments.
+ * @param mtu_size
+ *   Size in bytes of the Maximum Transfer Unit (MTU) for the outgoing IPv4
+ *   datagrams. This value includes the size of the IPv4 header.
+ * @param pool_direct
+ *   MBUF pool used for allocating direct buffers for the output fragments.
+ * @return
+ *   Upon successful completion - number of output fragments placed
+ *   in the pkts_out array.
+ *   Otherwise - (-1) * errno.
+ */
+__rte_experimental
+int32_t
+rte_ipv4_fragment_copy_packet(struct rte_mbuf *pkt_in,
+	struct rte_mbuf **pkts_out,
+	uint16_t nb_pkts_out,
+	uint16_t mtu_size,
+	struct rte_mempool *pool_direct);
+
+/**
  * This function implements reassembly of fragmented IPv4 packets.
  * Incoming mbufs should have its l2_len/l3_len fields setup correctly.
  *
lib/ip_frag/rte_ipv4_fragmentation.c +208 -0

217 lines not on this page · raw message

lib/ip_frag/version.map +1 -0
index b9c1cca..122ae8d 100644
@@ -17,4 +17,5 @@ EXPERIMENTAL {
 	global:
 
 	rte_ip_frag_table_del_expired_entries;
+	rte_ipv4_fragment_copy_packet;
 };
Signature · 1 line
1.8.3.1
#7 lore raw
Quoted 108 lines

From: Huichao Cai [mailto:[email protected]]
Sent: Friday, 22 July 2022 15.02
To: [email protected]
Cc: [email protected]
Subject: [PATCH v3] ip_frag: add IPv4 fragment copy packet API

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool and has refcnt = 1) offload. In order to adapt to this offload function, add this API. Add some test data for this API.

---
 app/test/test_ipfrag.c               |   8 +-
 lib/ip_frag/rte_ip_frag.h            |  27 +++++
 lib/ip_frag/rte_ipv4_fragmentation.c | 208
+++++++++++++++++++++++++++++++++++
 lib/ip_frag/version.map              |   1 +
 4 files changed, 243 insertions(+), 1 deletion(-)
app/test/test_ipfrag.c +7 -1
index ba0ffd0..bb7c4d3 100644
@@ -418,10 +418,16 @@ static void ut_teardown(void)
 		}
 
 		if (tests[i].ipv == 4)
-			len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
+			if (i % 2)
+				len = rte_ipv4_fragment_packet(b, pkts_out,
 BURST,
 						       tests[i].mtu_size,
 						       direct_pool,
 						       indirect_pool);
+			else
+				len = rte_ipv4_fragment_copy_packet(b,
 pkts_out,
+						       BURST,
+						       tests[i].mtu_size,
+						       pkt_pool);
 		else if (tests[i].ipv == 6)
app/test/test_ipfrag.c +0 -0
 			len = rte_ipv6_fragment_packet(b, pkts_out, BURST,
 						       tests[i].mtu_size,
lib/ip_frag/rte_ip_frag.h +10 -0
index 7d2abe1..c2a7e1e 100644
@@ -179,6 +179,33 @@ int32_t rte_ipv4_fragment_packet(struct rte_mbuf
 *pkt_in,
 			struct rte_mempool *pool_indirect);
 
 /**
+ * IPv4 fragmentation by copy.
+ *
+ * This function implements the fragmentation of IPv4 packets by copy.
+ *
+ * @param pkt_in
+ *   The input packet.
+ * @param pkts_out
+ *   Array storing the output fragments.
+ * @param mtu_size
+ *   Size in bytes of the Maximum Transfer Unit (MTU) for the outgoing

IPv4
+ * datagrams. This value includes the size of the IPv4 header.
+ * @param pool_direct
+ * MBUF pool used for allocating direct buffers for the output
fragments.
+ * @return

+ *   Upon successful completion - number of output fragments placed
+ *   in the pkts_out array.
+ *   Otherwise - (-1) * errno.

+ */

lib/ip_frag/rte_ip_frag.h +9 -0
+__rte_experimental
+int32_t
+rte_ipv4_fragment_copy_packet(struct rte_mbuf *pkt_in,
+	struct rte_mbuf **pkts_out,
+	uint16_t nb_pkts_out,
+	uint16_t mtu_size,
+	struct rte_mempool *pool_direct);
+
+/**
  * This function implements reassembly of fragmented IPv4 packets.
  * Incoming mbufs should have its l2_len/l3_len fields setup

correctly.
*

lib/ip_frag/rte_ipv4_fragmentation.c +11 -0
index 27a8ad2..cb15781 100644
@@ -83,6 +83,48 @@ static inline uint16_t
 __create_ipopt_frag_hdr(uint8_t *iph,
 	return ipopt_len;
 }
 
+static struct rte_mbuf *
+__copy_to_pktmbuf(char *src, struct rte_mbuf *m_head,
+	struct rte_mbuf *m_tail, struct rte_mempool *mp, uint32_t len)
+{
+	struct rte_mbuf *m_last, **prev;
+
+	m_last = m_tail;
+	prev = &m_last->next;
+	while (len > 0) {
+		uint32_t copy_len;
+

lib/ip_frag/rte_ipv4_fragmentation.c static inline uint16_t · line 107

Quoted 10 lines
lib/ip_frag/rte_ipv4_fragmentation.c +10 -0
+		/* current buffer is full, chain a new one */
+		if (unlikely(rte_pktmbuf_tailroom(m_last) == 0)) {
+			m_last = rte_pktmbuf_alloc(mp);
+			if (unlikely(m_last == NULL))
+				return NULL;
+
+			++m_head->nb_segs;
+			*prev = m_last;
+			prev = &m_last->next;
+		}

I think that MBUF_FAST_FREE also requires non-segmented packets, although this requirement is missing in the documentation. I have asked Olivier (as MBUF maintainer) to confirm this requirement [1].

[1] http://inbox.dpdk.org/dev/[email protected]/

Quoted 25 lines
lib/ip_frag/rte_ipv4_fragmentation.c +21 -0
+
+		/*
+		 * copy the min of data in len
+		 * vs space available in output (m_last)
+		 */
+		copy_len = RTE_MIN(rte_pktmbuf_tailroom(m_last), len);
+
+		/* append from seg to m_last */
+		memcpy(rte_pktmbuf_mtod_offset(m_last, char *, m_last-
 >data_len),
+			src, copy_len);
+
+		/* update offsets and lengths */
+		m_last->data_len += copy_len;
+		m_head->pkt_len += copy_len;
+		src += copy_len;
+		len -= copy_len;
+	}
+
+	return m_last;
+}
+
 /**

* IPv4 fragmentation.
*

#8 lore raw
In reply to Huichao Cai
Quoted 7 lines · On Fri, 22 Jul 2022 21:01:50 +0800 Huichao Cai <[email protected]> wrote:

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool and has refcnt = 1) offload. In order to adapt to this offload function, add this API. Add some test data for this API.

The code should just be checking that refcnt == 1 directly.

There are cases where sender passes a cloned mbuf. This is independent of the fast free optimization.

Similar to what Linux kernel does with skb_cow().

#9 lore raw
Quoted 12 lines

From: Stephen Hemminger [mailto:[email protected]]
Sent: Friday, 22 July 2022 16.49

Quoted 6 lines · On Fri, 22 Jul 2022 21:01:50 +0800 Huichao Cai <[email protected]> wrote:

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool and has refcnt = 1) offload. In order to adapt to this offload function, add this API. Add some test data for this API.

Quoted 8 lines
Quoted 1 line

The code should just be checking that refcnt == 1 directly.

There are cases where sender passes a cloned mbuf. This is independent of the fast free optimization.

Similar to what Linux kernel does with skb_cow().

Olivier just confirmed that MBUF_FAST_FREE requires that the mbufs are direct and non-segmented, although these requirements are not yet documented.

This means that you should not generate segmented mbufs with this patch. I don't know what to do instead; probably fail with an appropriate errno.

#10 lore raw
Quoted 19 lines · Morten Brørup
Quoted 19 lines

From: Stephen Hemminger [mailto:[email protected]]
Sent: Friday, 22 July 2022 16.49

Quoted 7 lines · On Fri, 22 Jul 2022 21:01:50 +0800 Huichao Cai <[email protected]> wrote:

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool and has refcnt = 1) offload. In order to adapt to this offload function, add this API. Add some test data for this API.

The code should just be checking that refcnt == 1 directly.

There are cases where sender passes a cloned mbuf. This is independent of the fast free optimization.

Quoted 6 lines
Quoted 1 line

Similar to what Linux kernel does with skb_cow().

Olivier just confirmed that MBUF_FAST_FREE requires that the mbufs are direct and non-segmented, although these requirements are not yet documented.

This means that you should not generate segmented mbufs with this patch. I don't know what to do instead; probably fail with an appropriate errno.

When the bnxt driver sends mbuf, it will take the mbuf segments apart and hang it to the tx_buf_ring, so there is no mbuf segments when it is released.Does this mean that there can be mbuf segments?

#11 lore raw

From: Huichao Cai [mailto:[email protected]]
Sent: Friday, 22 July 2022 17.59

Quoted 20 lines
Quoted 19 lines · Morten Brørup
Quoted 19 lines

From: Stephen Hemminger [mailto:[email protected]]
Sent: Friday, 22 July 2022 16.49

Quoted 7 lines · On Fri, 22 Jul 2022 21:01:50 +0800 Huichao Cai <[email protected]> wrote:

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool and has refcnt = 1) offload. In order to adapt to this offload function, add this API. Add some test data for this API.

The code should just be checking that refcnt == 1 directly.

There are cases where sender passes a cloned mbuf. This is independent of the fast free optimization.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Similar to what Linux kernel does with skb_cow().

Olivier just confirmed that MBUF_FAST_FREE requires that the mbufs are direct and non-segmented, although these requirements are not yet documented.

This means that you should not generate segmented mbufs with this patch. I don't know what to do instead; probably fail with an appropriate errno.

When the bnxt driver sends mbuf, it will take the mbuf segments apart and hang it to the tx_buf_ring, so there is no mbuf segments when it is released. Does this mean that there can be mbuf segments?

Only if the bnxt driver also resets the segmentation fields (nb_segs and next) in those mbufs, which I suppose it does, if it supports MBUF_FAST_FREE with segmented packets.

However, other Ethernet drivers don't do that, so a generic library function cannot rely on it. These missing requirements for MBUF_FAST_FREE is a bug, either in the MBUF_FAST_FREE documentation, or in the drivers where MBUF_FAST_FREE only works correctly with direct and non-segmented mbufs.

#12 lore raw

22/07/2022 17:14, Morten Brørup пишет:

Quoted 27 lines

From: Huichao Cai [mailto:[email protected]]
Sent: Friday, 22 July 2022 17.59

Quoted 24 lines
Quoted 23 lines · Morten Brørup
Quoted 12 lines

From: Stephen Hemminger [mailto:[email protected]]
Sent: Friday, 22 July 2022 16.49

Quoted 6 lines · On Fri, 22 Jul 2022 21:01:50 +0800 Huichao Cai <[email protected]> wrote:

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool and has refcnt = 1) offload. In order to adapt to this offload function, add this API. Add some test data for this API.

Quoted 8 lines
Quoted 1 line

The code should just be checking that refcnt == 1 directly.

There are cases where sender passes a cloned mbuf. This is independent of the fast free optimization.

Similar to what Linux kernel does with skb_cow().

Olivier just confirmed that MBUF_FAST_FREE requires that the mbufs are direct and non-segmented, although these requirements are not yet documented.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

This means that you should not generate segmented mbufs with this patch. I don't know what to do instead; probably fail with an appropriate errno.

When the bnxt driver sends mbuf, it will take the mbuf segments apart and hang it to the tx_buf_ring, so there is no mbuf segments when it is released. Does this mean that there can be mbuf segments?

Only if the bnxt driver also resets the segmentation fields (nb_segs and next) in those mbufs, which I suppose it does, if it supports MBUF_FAST_FREE with segmented packets.

However, other Ethernet drivers don't do that, so a generic library function cannot rely on it. These missing requirements for MBUF_FAST_FREE is a bug, either in the MBUF_FAST_FREE documentation, or in the drivers where MBUF_FAST_FREE only works correctly with direct and non-segmented mbufs.

I believe multi-segment packets work ok with MBUF_FAST_FREE (as long as other requirements are met).

#13 lore raw

+CC: i40e maintainers
+CC: mlx5 maintainers

Quoted 52 lines

From: Konstantin Ananyev [mailto:[email protected]]
Sent: Saturday, 23 July 2022 00.35

22/07/2022 17:14, Morten Brørup пишет:

Quoted 4 lines

From: Huichao Cai [mailto:[email protected]]
Sent: Friday, 22 July 2022 17.59

Quoted 1 line

At 2022-07-22 23:52:28, "Morten Brørup" <[email protected]>

Quoted 7 lines · wrote:
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

From: Stephen Hemminger [mailto:[email protected]]
Sent: Friday, 22 July 2022 16.49

Quoted 1 line · On Fri, 22 Jul 2022 21:01:50 +0800 Huichao Cai <[email protected]> wrote:

Some NIC drivers support MBUF_FAST_FREE(Device supports

optimization

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

for fast release of mbufs. When set application must guarantee

that

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

per-queue all mbufs comes from the same mempool and has refcnt =

1)

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

offload. In order to adapt to this offload function, add this

API.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

Add some test data for this API.

The code should just be checking that refcnt == 1 directly.

There are cases where sender passes a cloned mbuf. This is

independent

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

of the fast free optimization.

Similar to what Linux kernel does with skb_cow().

Olivier just confirmed that MBUF_FAST_FREE requires that the mbufs

are direct and non-segmented, although these requirements are not yet documented.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

This means that you should not generate segmented mbufs with this

patch. I don't know what to do instead; probably fail with an appropriate errno.

Quoted 2 lines
Quoted 2 lines

When the bnxt driver sends mbuf, it will take the mbuf segments

apart and hang it to the tx_buf_ring, so there is no mbuf segments when it is released. Does this mean that there can be mbuf segments?

Quoted 2 lines

Only if the bnxt driver also resets the segmentation fields (nb_segs

and next) in those mbufs, which I suppose it does, if it supports MBUF_FAST_FREE with segmented packets.

Quoted 2 lines

However, other Ethernet drivers don't do that, so a generic library

Quoted 8 lines

function cannot rely on it. These missing requirements for MBUF_FAST_FREE is a bug, either in the MBUF_FAST_FREE documentation, or in the drivers where MBUF_FAST_FREE only works correctly with direct and non-segmented mbufs.

Quoted 1 line

I believe multi-segment packets work ok with MBUF_FAST_FREE (as long as other requirements are met).

Looking at the i40e and mlx5 drivers, they both seem to call rte_mempool_put_bulk() without first calling rte_pktmbuf_prefree_seg(). So segmented packets freed with MBUF_FAST_FREE, will be stored in the mbuf pool without m->nb_segs and m->next being reset first.

I don't have deep knowledge of these drivers, so maybe I have overlooked something.

The point of MBUF_FAST_FREE is to bypass a lot of code under certain conditions. So I believe that these two undocumented requirements should remain, so the drivers can bypass this code. Otherwise, don't use MBUF_FAST_FREE.

#14 lore raw

23/07/2022 09:24, Morten Brørup пишет:

Quoted 61 lines
(quoted diff) +2 -0
+CC: i40e maintainers
+CC: mlx5 maintainers
Quoted 59 lines

From: Konstantin Ananyev [mailto:[email protected]]
Sent: Saturday, 23 July 2022 00.35

22/07/2022 17:14, Morten Brørup пишет:

Quoted 4 lines

From: Huichao Cai [mailto:[email protected]]
Sent: Friday, 22 July 2022 17.59

Quoted 1 line

At 2022-07-22 23:52:28, "Morten Brørup" <[email protected]>

Quoted 7 lines · wrote:
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

From: Stephen Hemminger [mailto:[email protected]]
Sent: Friday, 22 July 2022 16.49

Quoted 1 line · On Fri, 22 Jul 2022 21:01:50 +0800 Huichao Cai <[email protected]> wrote:

Some NIC drivers support MBUF_FAST_FREE(Device supports

optimization

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

for fast release of mbufs. When set application must guarantee

that

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

per-queue all mbufs comes from the same mempool and has refcnt =

1)

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

offload. In order to adapt to this offload function, add this

API.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

Add some test data for this API.

The code should just be checking that refcnt == 1 directly.

There are cases where sender passes a cloned mbuf. This is

independent

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

of the fast free optimization.

Similar to what Linux kernel does with skb_cow().

Olivier just confirmed that MBUF_FAST_FREE requires that the mbufs

are direct and non-segmented, although these requirements are not yet documented.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

This means that you should not generate segmented mbufs with this

patch. I don't know what to do instead; probably fail with an appropriate errno.

Quoted 2 lines
Quoted 2 lines

When the bnxt driver sends mbuf, it will take the mbuf segments

apart and hang it to the tx_buf_ring, so there is no mbuf segments when it is released. Does this mean that there can be mbuf segments?

Quoted 2 lines

Only if the bnxt driver also resets the segmentation fields (nb_segs

and next) in those mbufs, which I suppose it does, if it supports MBUF_FAST_FREE with segmented packets.

Quoted 2 lines

However, other Ethernet drivers don't do that, so a generic library

function cannot rely on it. These missing requirements for MBUF_FAST_FREE is a bug, either in the MBUF_FAST_FREE documentation, or in the drivers where MBUF_FAST_FREE only works correctly with direct and non-segmented mbufs.

Quoted 1 line
Quoted 9 lines
Quoted 2 lines

I believe multi-segment packets work ok with MBUF_FAST_FREE (as long as other requirements are met).

Looking at the i40e and mlx5 drivers, they both seem to call rte_mempool_put_bulk() without first calling rte_pktmbuf_prefree_seg(). So segmented packets freed with MBUF_FAST_FREE, will be stored in the mbuf pool without m->nb_segs and m->next being reset first.

I don't have deep knowledge of these drivers, so maybe I have overlooked something.

The point of MBUF_FAST_FREE is to bypass a lot of code under certain conditions. So I believe that these two undocumented requirements should remain, so the drivers can bypass this code. Otherwise, don't use MBUF_FAST_FREE.

Actually, after another look, I think you and Olivier are right - multi-seg packets should not be used together with MBUF_FAST_FREE. I forgot that mbuf_prefree() is responsible to reset both 'next' and 'nb_segs' fields of the mbuf.
It might keep working for some simple forwarding app (like l3fwd), as most PMDs reset these fields at RX path anyway, but that's just a coincidence we shouldn't rely on.
We probably need to update l3fwd (and other examples) to dis-allow MBUF_FAST_FREE when TX_OFFLOAD_MULTI_SEGS is selected.

Konstantin

#15 lore raw
Quoted 88 lines

From: Konstantin Ananyev [mailto:[email protected]]
Sent: Saturday, 23 July 2022 20.25

23/07/2022 09:24, Morten Brørup пишет:

Quoted 15 lines
(quoted diff) +2 -0
+CC: i40e maintainers
+CC: mlx5 maintainers
Quoted 13 lines

From: Konstantin Ananyev [mailto:[email protected]]
Sent: Saturday, 23 July 2022 00.35

22/07/2022 17:14, Morten Brørup пишет:

Quoted 4 lines

From: Huichao Cai [mailto:[email protected]]
Sent: Friday, 22 July 2022 17.59

Quoted 1 line

At 2022-07-22 23:52:28, "Morten Brørup" <[email protected]>

Quoted 3 lines · wrote:
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

From: Stephen Hemminger [mailto:[email protected]]
Sent: Friday, 22 July 2022 16.49

Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line · On Fri, 22 Jul 2022 21:01:50 +0800 Huichao Cai <[email protected]> wrote:

Some NIC drivers support MBUF_FAST_FREE(Device supports

optimization

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

for fast release of mbufs. When set application must guarantee

that

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

per-queue all mbufs comes from the same mempool and has refcnt

=

Quoted 9 lines
Quoted 9 lines

1)

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

offload. In order to adapt to this offload function, add this

API.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines

Add some test data for this API.

The code should just be checking that refcnt == 1 directly.

Quoted 7 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

There are cases where sender passes a cloned mbuf. This is

independent

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

of the fast free optimization.

Similar to what Linux kernel does with skb_cow().

Olivier just confirmed that MBUF_FAST_FREE requires that the

mbufs

Quoted 1 line
Quoted 1 line

are direct and non-segmented, although these requirements are not

yet

Quoted 8 lines
Quoted 8 lines

documented.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

This means that you should not generate segmented mbufs with this

patch. I don't know what to do instead; probably fail with an appropriate errno.

Quoted 2 lines
Quoted 2 lines

When the bnxt driver sends mbuf, it will take the mbuf segments

apart and hang it to the tx_buf_ring, so there is no mbuf segments

when

Quoted 3 lines
Quoted 3 lines

it is released. Does this mean that there can be mbuf segments?

Quoted 2 lines

Only if the bnxt driver also resets the segmentation fields

(nb_segs

Quoted 6 lines
Quoted 6 lines

and next) in those mbufs, which I suppose it does, if it supports MBUF_FAST_FREE with segmented packets.

Quoted 2 lines

However, other Ethernet drivers don't do that, so a generic library

function cannot rely on it. These missing requirements for MBUF_FAST_FREE is a bug, either in the MBUF_FAST_FREE documentation,

or

Quoted 8 lines
Quoted 6 lines

in the drivers where MBUF_FAST_FREE only works correctly with direct and non-segmented mbufs.

Quoted 1 line

I believe multi-segment packets work ok with MBUF_FAST_FREE (as long as other requirements are met).

Looking at the i40e and mlx5 drivers, they both seem to call

rte_mempool_put_bulk() without first calling rte_pktmbuf_prefree_seg(). So segmented packets freed with MBUF_FAST_FREE, will be stored in the mbuf pool without m->nb_segs and m->next being reset first.

Quoted 2 lines

I don't have deep knowledge of these drivers, so maybe I have

overlooked something.

Quoted 2 lines

The point of MBUF_FAST_FREE is to bypass a lot of code under certain

conditions. So I believe that these two undocumented requirements should remain, so the drivers can bypass this code. Otherwise, don't use MBUF_FAST_FREE.

Quoted 1 line
Quoted 7 lines

Actually, after another look, I think you and Olivier are right - multi-seg packets should not be used together with MBUF_FAST_FREE. I forgot that mbuf_prefree() is responsible to reset both 'next' and 'nb_segs' fields of the mbuf.
It might keep working for some simple forwarding app (like l3fwd), as most PMDs reset these fields at RX path anyway, but that's just a coincidence we shouldn't rely on.

I hope the PMDs don't reset these fields in their RX path, unless they are creating multi-seg packets and therefore must. It might cause an extra cache miss per packet, if the PMD unnecessarily sets m->next, which is in the second cache line of the mbuf.

Or perhaps everyone has forgotten about this RX/TX split of the first/second cache line of the mbufs, because all tests are based on run-to-completion, where the second cache line will be written shortly afterwards anyway. :-(

Quoted 2 lines

We probably need to update l3fwd (and other examples) to dis-allow MBUF_FAST_FREE when TX_OFFLOAD_MULTI_SEGS is selected.

+1

Quoted 2 lines

Konstantin

#16 lore raw

PATCH v4 1/1 ip_frag: add IPv4 fragment copy packet API

In reply to Huichao Cai

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool and has refcnt = 1) offload. In order to adapt to this offload function, add this API. Add some test data for this API.

---
 app/test/test_ipfrag.c               |   8 +-
 lib/ip_frag/rte_ip_frag.h            |  29 +++++
 lib/ip_frag/rte_ipv4_fragmentation.c | 208 +++++++++++++++++++++++++++++++++++
 lib/ip_frag/version.map              |   1 +
 4 files changed, 245 insertions(+), 1 deletion(-)
app/test/test_ipfrag.c +7 -1
index ba0ffd0..8eddbe9 100644
@@ -418,10 +418,16 @@ static void ut_teardown(void)
 		}
 
 		if (tests[i].ipv == 4)
-			len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
+			if (i % 2)
+				len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
 						       tests[i].mtu_size,
 						       direct_pool,
 						       indirect_pool);
+			else
+				len = rte_ipv4_fragment_copy_packet(b, pkts_out,
+						       BURST,
+						       tests[i].mtu_size,
+						       direct_pool);
 		else if (tests[i].ipv == 6)
 			len = rte_ipv6_fragment_packet(b, pkts_out, BURST,
 						       tests[i].mtu_size,
lib/ip_frag/rte_ip_frag.h +29 -0
index 7d2abe1..a84e5f7 100644
@@ -179,6 +179,35 @@ int32_t rte_ipv4_fragment_packet(struct rte_mbuf *pkt_in,
 			struct rte_mempool *pool_indirect);
 
 /**
+ * IPv4 fragmentation by copy.
+ *
+ * This function implements the fragmentation of IPv4 packets by copy.
+ *
+ * @param pkt_in
+ *   The input packet.
+ * @param pkts_out
+ *   Array storing the output fragments.
+ * @param nb_pkts_out
+ *   Number of fragments.
+ * @param mtu_size
+ *   Size in bytes of the Maximum Transfer Unit (MTU) for the outgoing IPv4
+ *   datagrams. This value includes the size of the IPv4 header.
+ * @param pool_direct
+ *   MBUF pool used for allocating direct buffers for the output fragments.
+ * @return
+ *   Upon successful completion - number of output fragments placed
+ *   in the pkts_out array.
+ *   Otherwise - (-1) * errno.
+ */
+__rte_experimental
+int32_t
+rte_ipv4_fragment_copy_packet(struct rte_mbuf *pkt_in,
+	struct rte_mbuf **pkts_out,
+	uint16_t nb_pkts_out,
+	uint16_t mtu_size,
+	struct rte_mempool *pool_direct);
+
+/**
  * This function implements reassembly of fragmented IPv4 packets.
  * Incoming mbufs should have its l2_len/l3_len fields setup correctly.
  *
lib/ip_frag/rte_ipv4_fragmentation.c +208 -0

217 lines not on this page · raw message

lib/ip_frag/version.map +1 -0
index b9c1cca..122ae8d 100644
@@ -17,4 +17,5 @@ EXPERIMENTAL {
 	global:
 
 	rte_ip_frag_table_del_expired_entries;
+	rte_ipv4_fragment_copy_packet;
 };
Signature · 1 line
1.8.3.1
#17 lore raw

PATCH v5 1/1 ip_frag: add IPv4 fragment copy packet API

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool,has refcnt = 1,direct and non-segmented.)offload. In order to adapt to this offload function, add this API. Add some test data for this API.

---
 app/test/test_ipfrag.c               |   9 +-
 lib/ip_frag/rte_ip_frag.h            |  34 +++++++
 lib/ip_frag/rte_ipv4_fragmentation.c | 174 +++++++++++++++++++++++++++++++++++
 lib/ip_frag/version.map              |   1 +
 4 files changed, 217 insertions(+), 1 deletion(-)
app/test/test_ipfrag.c +8 -1
index ba0ffd0..88cc4cd 100644
@@ -418,10 +418,17 @@ static void ut_teardown(void)
 		}
 
 		if (tests[i].ipv == 4)
-			len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
+			if (i % 2)
+				len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
 						       tests[i].mtu_size,
 						       direct_pool,
 						       indirect_pool);
+			else
+				len = rte_ipv4_fragment_copy_nonseg_packet(b,
+						       pkts_out,
+						       BURST,
+						       tests[i].mtu_size,
+						       direct_pool);
 		else if (tests[i].ipv == 6)
 			len = rte_ipv6_fragment_packet(b, pkts_out, BURST,
 						       tests[i].mtu_size,
lib/ip_frag/rte_ip_frag.h +34 -0
index 7d2abe1..4a2b150 100644
@@ -179,6 +179,40 @@ int32_t rte_ipv4_fragment_packet(struct rte_mbuf *pkt_in,
 			struct rte_mempool *pool_indirect);
 
 /**
+ * IPv4 fragmentation by copy.
+ *
+ * This function implements the fragmentation of IPv4 packets by copy
+ * non-segmented mbuf.
+ * This function is mainly used to adapt TX MBUF_FAST_FREE offload.
+ * MBUF_FAST_FREE: Device supports optimization for fast release of mbufs.
+ * When set application must guarantee that per-queue all mbufs comes from
+ * the same mempool,has refcnt = 1,direct and non-segmented.
+ *
+ * @param pkt_in
+ *   The input packet.
+ * @param pkts_out
+ *   Array storing the output fragments.
+ * @param nb_pkts_out
+ *   Number of fragments.
+ * @param mtu_size
+ *   Size in bytes of the Maximum Transfer Unit (MTU) for the outgoing IPv4
+ *   datagrams. This value includes the size of the IPv4 header.
+ * @param pool_direct
+ *   MBUF pool used for allocating direct buffers for the output fragments.
+ * @return
+ *   Upon successful completion - number of output fragments placed
+ *   in the pkts_out array.
+ *   Otherwise - (-1) * errno.
+ */
+__rte_experimental
+int32_t
+rte_ipv4_fragment_copy_nonseg_packet(struct rte_mbuf *pkt_in,
+	struct rte_mbuf **pkts_out,
+	uint16_t nb_pkts_out,
+	uint16_t mtu_size,
+	struct rte_mempool *pool_direct);
+
+/**
  * This function implements reassembly of fragmented IPv4 packets.
  * Incoming mbufs should have its l2_len/l3_len fields setup correctly.
  *
lib/ip_frag/rte_ipv4_fragmentation.c +174 -0

177 lines not on this page · raw message

lib/ip_frag/version.map +1 -0
index b9c1cca..8aad839 100644
@@ -17,4 +17,5 @@ EXPERIMENTAL {
 	global:
 
 	rte_ip_frag_table_del_expired_entries;
+	rte_ipv4_fragment_copy_nonseg_packet;
 };
Signature · 1 line
1.8.3.1
#18 lore raw

PATCH v6 1/1 ip_frag: add IPv4 fragment copy packet API

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool,has refcnt = 1,direct and non-segmented.)offload. In order to adapt to this offload function, add this API. Add some test data for this API.

---
 app/test/test_ipfrag.c               |   9 +-
 lib/ip_frag/rte_ip_frag.h            |  34 +++++++
 lib/ip_frag/rte_ipv4_fragmentation.c | 175 +++++++++++++++++++++++++++++++++++
 lib/ip_frag/version.map              |   1 +
 4 files changed, 218 insertions(+), 1 deletion(-)
app/test/test_ipfrag.c +8 -1
index ba0ffd0..88cc4cd 100644
@@ -418,10 +418,17 @@ static void ut_teardown(void)
 		}
 
 		if (tests[i].ipv == 4)
-			len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
+			if (i % 2)
+				len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
 						       tests[i].mtu_size,
 						       direct_pool,
 						       indirect_pool);
+			else
+				len = rte_ipv4_fragment_copy_nonseg_packet(b,
+						       pkts_out,
+						       BURST,
+						       tests[i].mtu_size,
+						       direct_pool);
 		else if (tests[i].ipv == 6)
 			len = rte_ipv6_fragment_packet(b, pkts_out, BURST,
 						       tests[i].mtu_size,
lib/ip_frag/rte_ip_frag.h +34 -0
index 7d2abe1..4a2b150 100644
@@ -179,6 +179,40 @@ int32_t rte_ipv4_fragment_packet(struct rte_mbuf *pkt_in,
 			struct rte_mempool *pool_indirect);
 
 /**
+ * IPv4 fragmentation by copy.
+ *
+ * This function implements the fragmentation of IPv4 packets by copy
+ * non-segmented mbuf.
+ * This function is mainly used to adapt TX MBUF_FAST_FREE offload.
+ * MBUF_FAST_FREE: Device supports optimization for fast release of mbufs.
+ * When set application must guarantee that per-queue all mbufs comes from
+ * the same mempool,has refcnt = 1,direct and non-segmented.
+ *
+ * @param pkt_in
+ *   The input packet.
+ * @param pkts_out
+ *   Array storing the output fragments.
+ * @param nb_pkts_out
+ *   Number of fragments.
+ * @param mtu_size
+ *   Size in bytes of the Maximum Transfer Unit (MTU) for the outgoing IPv4
+ *   datagrams. This value includes the size of the IPv4 header.
+ * @param pool_direct
+ *   MBUF pool used for allocating direct buffers for the output fragments.
+ * @return
+ *   Upon successful completion - number of output fragments placed
+ *   in the pkts_out array.
+ *   Otherwise - (-1) * errno.
+ */
+__rte_experimental
+int32_t
+rte_ipv4_fragment_copy_nonseg_packet(struct rte_mbuf *pkt_in,
+	struct rte_mbuf **pkts_out,
+	uint16_t nb_pkts_out,
+	uint16_t mtu_size,
+	struct rte_mempool *pool_direct);
+
+/**
  * This function implements reassembly of fragmented IPv4 packets.
  * Incoming mbufs should have its l2_len/l3_len fields setup correctly.
  *
lib/ip_frag/rte_ipv4_fragmentation.c +175 -0

178 lines not on this page · raw message

lib/ip_frag/version.map +1 -0
index b9c1cca..8aad839 100644
@@ -17,4 +17,5 @@ EXPERIMENTAL {
 	global:
 
 	rte_ip_frag_table_del_expired_entries;
+	rte_ipv4_fragment_copy_nonseg_packet;
 };
Signature · 1 line
1.8.3.1
#19 lore raw
Quoted 8 lines · On Sun, 24 Jul 2022 16:10:03 +0800 Huichao Cai <[email protected]> wrote:
lib/ip_frag/rte_ipv4_fragmentation.c +8 -0
+
+	/*
+	 * Formal parameter checking.
+	 */
+	if (unlikely(pkt_in == NULL) || unlikely(pkts_out == NULL) ||
+	    unlikely(nb_pkts_out == 0) || unlikely(pool_direct == NULL) ||
+	    unlikely(mtu_size < RTE_ETHER_MIN_MTU))
+		return -EINVAL;

lib/ip_frag/rte_ipv4_fragmentation.c static inline uint16_t __create_ipopt_frag_hdr(uint8_t *iph, · line 321

Quoted 10 lines
lib/ip_frag/rte_ipv4_fragmentation.c +10 -0
+
+	in_hdr = rte_pktmbuf_mtod(pkt_in, struct rte_ipv4_hdr *);
+	header_len = (in_hdr->version_ihl & RTE_IPV4_HDR_IHL_MASK) *
+	    RTE_IPV4_IHL_MULTIPLIER;
+
+	/* Check IP header length */
+	if (unlikely(pkt_in->data_len < header_len) ||
+	    unlikely(mtu_size < header_len))
+		return -EINVAL;
+

My suspicions are all this input parameter checking probably costs more than any performance gain of having a non-segmented fast path.

#20 lore raw
Quoted 24 lines · Stephen Hemminger
Quoted 8 lines · On Sun, 24 Jul 2022 16:10:03 +0800 Huichao Cai <[email protected]> wrote:
lib/ip_frag/rte_ipv4_fragmentation.c +8 -0
+
+	/*
+	 * Formal parameter checking.
+	 */
+	if (unlikely(pkt_in == NULL) || unlikely(pkts_out == NULL) ||
+	    unlikely(nb_pkts_out == 0) || unlikely(pool_direct == NULL) ||
+	    unlikely(mtu_size < RTE_ETHER_MIN_MTU))
+		return -EINVAL;

lib/ip_frag/rte_ipv4_fragmentation.c static inline uint16_t __create_ipopt_frag_hdr(uint8_t *iph, · line 321

Quoted 10 lines
lib/ip_frag/rte_ipv4_fragmentation.c +10 -0
+
+	in_hdr = rte_pktmbuf_mtod(pkt_in, struct rte_ipv4_hdr *);
+	header_len = (in_hdr->version_ihl & RTE_IPV4_HDR_IHL_MASK) *
+	    RTE_IPV4_IHL_MULTIPLIER;
+
+	/* Check IP header length */
+	if (unlikely(pkt_in->data_len < header_len) ||
+	    unlikely(mtu_size < header_len))
+		return -EINVAL;
+

My suspicions are all this input parameter checking probably costs more

Quoted 1 line

than any performance gain of having a non-segmented fast path.

These checks are consistent with the rte_ipv4_fragment_packet function. I think these have been tested for performance.If these checks do affect performance, perhaps the legitimacy of the variable is better guaranteed by the caller

#21 lore raw
Quoted 19 lines
Quoted 16 lines · Stephen Hemminger
Quoted 13 lines · On Sun, 24 Jul 2022 16:10:03 +0800 Huichao Cai <[email protected]> wrote:
lib/ip_frag/rte_ipv4_fragmentation.c +13 -0
+
+	/*
+	 * Formal parameter checking.
+	 */
+	if (unlikely(pkt_in == NULL) || unlikely(pkts_out == NULL) ||
+	    unlikely(nb_pkts_out == 0) || unlikely(pool_direct == NULL) ||
+	    unlikely(mtu_size < RTE_ETHER_MIN_MTU))
+		return -EINVAL;
+
+	in_hdr = rte_pktmbuf_mtod(pkt_in, struct rte_ipv4_hdr *);
+	header_len = (in_hdr->version_ihl & RTE_IPV4_HDR_IHL_MASK) *
+	    RTE_IPV4_IHL_MULTIPLIER;
+
Quoted 8 lines
Quoted 8 lines

lib/ip_frag/rte_ipv4_fragmentation.c static inline uint16_t __create_ipopt_frag_hdr(uint8_t *iph, · line 321

Quoted 5 lines
lib/ip_frag/rte_ipv4_fragmentation.c +5 -0
+	/* Check IP header length */
+	if (unlikely(pkt_in->data_len < header_len) ||
+	    unlikely(mtu_size < header_len))
+		return -EINVAL;
+

My suspicions are all this input parameter checking probably costs more than any performance gain of having a non-segmented fast path.

I think checks are not that expensive.
My guess - actual copying will be the main cycles eater here.
Though if percentage of packets that need to be fragmented is tiny,
might be it is still worth it.
Though yes, I still think better would be not to use MBUF_FAST_FREE at all, but we are where we are.

Quoted 3 lines

These checks are consistent with the rte_ipv4_fragment_packet function. I think these have been tested for performance.If these checks do affect performance, perhaps the legitimacy of the variable is better guaranteed by the caller

Acked-by on v6 1/1
#22 lore raw
In reply to Huichao Cai

24/07/2022 09:10, Huichao Cai пишет:

Quoted 181 lines

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool,has refcnt = 1,direct and non-segmented.)offload. In order to adapt to this offload function, add this API. Add some test data for this API.

---
  app/test/test_ipfrag.c               |   9 +-
  lib/ip_frag/rte_ip_frag.h            |  34 +++++++
  lib/ip_frag/rte_ipv4_fragmentation.c | 175 +++++++++++++++++++++++++++++++++++
  lib/ip_frag/version.map              |   1 +
  4 files changed, 218 insertions(+), 1 deletion(-)
app/test/test_ipfrag.c +8 -1
index ba0ffd0..88cc4cd 100644
@@ -418,10 +418,17 @@ static void ut_teardown(void)
  		}
 
  		if (tests[i].ipv == 4)
-			len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
+			if (i % 2)
+				len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
  						       tests[i].mtu_size,
  						       direct_pool,
  						       indirect_pool);
+			else
+				len = rte_ipv4_fragment_copy_nonseg_packet(b,
+						       pkts_out,
+						       BURST,
+						       tests[i].mtu_size,
+						       direct_pool);
  		else if (tests[i].ipv == 6)
  			len = rte_ipv6_fragment_packet(b, pkts_out, BURST,
  						       tests[i].mtu_size,
lib/ip_frag/rte_ip_frag.h +34 -0
index 7d2abe1..4a2b150 100644
@@ -179,6 +179,40 @@ int32_t rte_ipv4_fragment_packet(struct rte_mbuf *pkt_in,
  			struct rte_mempool *pool_indirect);
 
  /**
+ * IPv4 fragmentation by copy.
+ *
+ * This function implements the fragmentation of IPv4 packets by copy
+ * non-segmented mbuf.
+ * This function is mainly used to adapt TX MBUF_FAST_FREE offload.
+ * MBUF_FAST_FREE: Device supports optimization for fast release of mbufs.
+ * When set application must guarantee that per-queue all mbufs comes from
+ * the same mempool,has refcnt = 1,direct and non-segmented.
+ *
+ * @param pkt_in
+ *   The input packet.
+ * @param pkts_out
+ *   Array storing the output fragments.
+ * @param nb_pkts_out
+ *   Number of fragments.
+ * @param mtu_size
+ *   Size in bytes of the Maximum Transfer Unit (MTU) for the outgoing IPv4
+ *   datagrams. This value includes the size of the IPv4 header.
+ * @param pool_direct
+ *   MBUF pool used for allocating direct buffers for the output fragments.
+ * @return
+ *   Upon successful completion - number of output fragments placed
+ *   in the pkts_out array.
+ *   Otherwise - (-1) * errno.
+ */
+__rte_experimental
+int32_t
+rte_ipv4_fragment_copy_nonseg_packet(struct rte_mbuf *pkt_in,
+	struct rte_mbuf **pkts_out,
+	uint16_t nb_pkts_out,
+	uint16_t mtu_size,
+	struct rte_mempool *pool_direct);
+
+/**
   * This function implements reassembly of fragmented IPv4 packets.
   * Incoming mbufs should have its l2_len/l3_len fields setup correctly.
   *
lib/ip_frag/rte_ipv4_fragmentation.c +91 -0
index 27a8ad2..e6ec408 100644
@@ -259,3 +259,178 @@ static inline uint16_t __create_ipopt_frag_hdr(uint8_t *iph,
 
  	return out_pkt_pos;
  }
+
+/**
+ * IPv4 fragmentation by copy.
+ *
+ * This function implements the fragmentation of IPv4 packets by copy
+ * non-segmented mbuf.
+ * This function is mainly used to adapt TX MBUF_FAST_FREE offload.
+ * MBUF_FAST_FREE: Device supports optimization for fast release of mbufs.
+ * When set application must guarantee that per-queue all mbufs comes from
+ * the same mempool,has refcnt = 1,direct and non-segmented.
+ *
+ * @param pkt_in
+ *   The input packet.
+ * @param pkts_out
+ *   Array storing the output fragments.
+ * @param nb_pkts_out
+ *   Number of fragments.
+ * @param mtu_size
+ *   Size in bytes of the Maximum Transfer Unit (MTU) for the outgoing IPv4
+ *   datagrams. This value includes the size of the IPv4 header.
+ * @param pool_direct
+ *   MBUF pool used for allocating direct buffers for the output fragments.
+ * @return
+ *   Upon successful completion - number of output fragments placed
+ *   in the pkts_out array.
+ *   Otherwise - (-1) * errno.
+ */
+int32_t
+rte_ipv4_fragment_copy_nonseg_packet(struct rte_mbuf *pkt_in,
+	struct rte_mbuf **pkts_out,
+	uint16_t nb_pkts_out,
+	uint16_t mtu_size,
+	struct rte_mempool *pool_direct)
+{
+	struct rte_mbuf *in_seg = NULL;
+	struct rte_ipv4_hdr *in_hdr;
+	uint32_t out_pkt_pos, in_seg_data_pos;
+	uint32_t more_in_segs;
+	uint16_t fragment_offset, flag_offset, frag_size, header_len;
+	uint16_t frag_bytes_remaining;
+	uint8_t ipopt_frag_hdr[IPV4_HDR_MAX_LEN];
+	uint16_t ipopt_len;
+
+	/*
+	 * Formal parameter checking.
+	 */
+	if (unlikely(pkt_in == NULL) || unlikely(pkts_out == NULL) ||
+	    unlikely(nb_pkts_out == 0) || unlikely(pool_direct == NULL) ||
+	    unlikely(mtu_size < RTE_ETHER_MIN_MTU))
+		return -EINVAL;
+
+	in_hdr = rte_pktmbuf_mtod(pkt_in, struct rte_ipv4_hdr *);
+	header_len = (in_hdr->version_ihl & RTE_IPV4_HDR_IHL_MASK) *
+	    RTE_IPV4_IHL_MULTIPLIER;
+
+	/* Check IP header length */
+	if (unlikely(pkt_in->data_len < header_len) ||
+	    unlikely(mtu_size < header_len))
+		return -EINVAL;
+
+	/*
+	 * Ensure the IP payload length of all fragments is aligned to a
+	 * multiple of 8 bytes as per RFC791 section 2.3.
+	 */
+	frag_size = RTE_ALIGN_FLOOR((mtu_size - header_len),
+				    IPV4_HDR_FO_ALIGN);
+
+	flag_offset = rte_cpu_to_be_16(in_hdr->fragment_offset);
+
+	/* If Don't Fragment flag is set */
+	if (unlikely((flag_offset & IPV4_HDR_DF_MASK) != 0))
+		return -ENOTSUP;
+
+	/* Check that pkts_out is big enough to hold all fragments */
+	if (unlikely(frag_size * nb_pkts_out <
+	    (uint16_t)(pkt_in->pkt_len - header_len)))
+		return -EINVAL;
+
+	in_seg = pkt_in;
+	in_seg_data_pos = header_len;
+	out_pkt_pos = 0;
+	fragment_offset = 0;
+
+	ipopt_len = header_len - sizeof(struct rte_ipv4_hdr);
+	if (unlikely(ipopt_len > RTE_IPV4_HDR_OPT_MAX_LEN))
+		return -EINVAL;
+
+	more_in_segs = 1;
+	while (likely(more_in_segs)) {
+		struct rte_mbuf *out_pkt = NULL;
+		uint32_t more_out_segs;

lib/ip_frag/rte_ipv4_fragmentation.c static inline uint16_t __create_ipopt_frag_hdr(uint8_t *iph, · line 362

Quoted 10 lines
lib/ip_frag/rte_ipv4_fragmentation.c +10 -0
+		struct rte_ipv4_hdr *out_hdr;
+
+		/* Allocate direct buffer */
+		out_pkt = rte_pktmbuf_alloc(pool_direct);
+		if (unlikely(out_pkt == NULL)) {
+			__free_fragments(pkts_out, out_pkt_pos);
+			return -ENOMEM;
+		}
+		if (unlikely(out_pkt->buf_len - rte_pktmbuf_headroom(out_pkt) <
+				frag_size)) {

As a nit, might be better;
if (rte_pktmbuf_tailroom(out_pkt) < frag_size) {...}

Quoted 84 lines
lib/ip_frag/rte_ipv4_fragmentation.c +74 -0
+			rte_pktmbuf_free(out_pkt);
+			__free_fragments(pkts_out, out_pkt_pos);
+			return -EINVAL;
+		}
+
+		/* Reserve space for the IP header that will be built later */
+		out_pkt->data_len = header_len;
+		out_pkt->pkt_len = header_len;
+		frag_bytes_remaining = frag_size;
+
+		more_out_segs = 1;
+		while (likely(more_out_segs && more_in_segs)) {
+			uint32_t len;
+
+			len = frag_bytes_remaining;
+			if (len > (in_seg->data_len - in_seg_data_pos))
+				len = in_seg->data_len - in_seg_data_pos;
+
+			memcpy(rte_pktmbuf_mtod_offset(out_pkt, char *,
+					out_pkt->data_len),
+				rte_pktmbuf_mtod_offset(in_seg, char *,
+					in_seg_data_pos),
+				len);
+
+			in_seg_data_pos += len;
+			frag_bytes_remaining -= len;
+			out_pkt->data_len += len;
+
+			/* Current output packet (i.e. fragment) done ? */
+			if (unlikely(frag_bytes_remaining == 0))
+				more_out_segs = 0;
+
+			/* Current input segment done ? */
+			if (unlikely(in_seg_data_pos == in_seg->data_len)) {
+				in_seg = in_seg->next;
+				in_seg_data_pos = 0;
+
+				if (unlikely(in_seg == NULL))
+					more_in_segs = 0;
+			}
+		}
+
+		/* Build the IP header */
+
+		out_pkt->pkt_len = out_pkt->data_len;
+		out_hdr = rte_pktmbuf_mtod(out_pkt, struct rte_ipv4_hdr *);
+
+		__fill_ipv4hdr_frag(out_hdr, in_hdr, header_len,
+		    (uint16_t)out_pkt->pkt_len,
+		    flag_offset, fragment_offset, more_in_segs);
+
+		if (unlikely((fragment_offset == 0) && (ipopt_len) &&
+			    ((flag_offset & RTE_IPV4_HDR_OFFSET_MASK) == 0))) {
+			ipopt_len = __create_ipopt_frag_hdr((uint8_t *)in_hdr,
+				ipopt_len, ipopt_frag_hdr);
+			fragment_offset = (uint16_t)(fragment_offset +
+				out_pkt->pkt_len - header_len);
+			out_pkt->l3_len = header_len;
+
+			header_len = sizeof(struct rte_ipv4_hdr) + ipopt_len;
+			in_hdr = (struct rte_ipv4_hdr *)ipopt_frag_hdr;
+		} else {
+			fragment_offset = (uint16_t)(fragment_offset +
+				out_pkt->pkt_len - header_len);
+			out_pkt->l3_len = header_len;
+		}
+
+		/* Write the fragment to the output list */
+		pkts_out[out_pkt_pos] = out_pkt;
+		out_pkt_pos++;
+	}
+
+	return out_pkt_pos;
+}
lib/ip_frag/version.map +1 -0
index b9c1cca..8aad839 100644
@@ -17,4 +17,5 @@ EXPERIMENTAL {
  	global:
 
  	rte_ip_frag_table_del_expired_entries;
+	rte_ipv4_fragment_copy_nonseg_packet;
  };
#23 lore raw

PATCH v7 1/1 ip_frag: add IPv4 fragment copy packet API

In reply to Huichao Cai

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool,has refcnt = 1,direct and non-segmented.)offload. In order to adapt to this offload function, add this API. Add some test data for this API.

---
 app/test/test_ipfrag.c               |   9 +-
 lib/ip_frag/rte_ip_frag.h            |  34 +++++++
 lib/ip_frag/rte_ipv4_fragmentation.c | 174 +++++++++++++++++++++++++++++++++++
 lib/ip_frag/version.map              |   1 +
 4 files changed, 217 insertions(+), 1 deletion(-)
app/test/test_ipfrag.c +8 -1
index ba0ffd0..88cc4cd 100644
@@ -418,10 +418,17 @@ static void ut_teardown(void)
 		}
 
 		if (tests[i].ipv == 4)
-			len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
+			if (i % 2)
+				len = rte_ipv4_fragment_packet(b, pkts_out, BURST,
 						       tests[i].mtu_size,
 						       direct_pool,
 						       indirect_pool);
+			else
+				len = rte_ipv4_fragment_copy_nonseg_packet(b,
+						       pkts_out,
+						       BURST,
+						       tests[i].mtu_size,
+						       direct_pool);
 		else if (tests[i].ipv == 6)
 			len = rte_ipv6_fragment_packet(b, pkts_out, BURST,
 						       tests[i].mtu_size,
lib/ip_frag/rte_ip_frag.h +34 -0
index 7d2abe1..4a2b150 100644
@@ -179,6 +179,40 @@ int32_t rte_ipv4_fragment_packet(struct rte_mbuf *pkt_in,
 			struct rte_mempool *pool_indirect);
 
 /**
+ * IPv4 fragmentation by copy.
+ *
+ * This function implements the fragmentation of IPv4 packets by copy
+ * non-segmented mbuf.
+ * This function is mainly used to adapt TX MBUF_FAST_FREE offload.
+ * MBUF_FAST_FREE: Device supports optimization for fast release of mbufs.
+ * When set application must guarantee that per-queue all mbufs comes from
+ * the same mempool,has refcnt = 1,direct and non-segmented.
+ *
+ * @param pkt_in
+ *   The input packet.
+ * @param pkts_out
+ *   Array storing the output fragments.
+ * @param nb_pkts_out
+ *   Number of fragments.
+ * @param mtu_size
+ *   Size in bytes of the Maximum Transfer Unit (MTU) for the outgoing IPv4
+ *   datagrams. This value includes the size of the IPv4 header.
+ * @param pool_direct
+ *   MBUF pool used for allocating direct buffers for the output fragments.
+ * @return
+ *   Upon successful completion - number of output fragments placed
+ *   in the pkts_out array.
+ *   Otherwise - (-1) * errno.
+ */
+__rte_experimental
+int32_t
+rte_ipv4_fragment_copy_nonseg_packet(struct rte_mbuf *pkt_in,
+	struct rte_mbuf **pkts_out,
+	uint16_t nb_pkts_out,
+	uint16_t mtu_size,
+	struct rte_mempool *pool_direct);
+
+/**
  * This function implements reassembly of fragmented IPv4 packets.
  * Incoming mbufs should have its l2_len/l3_len fields setup correctly.
  *
lib/ip_frag/rte_ipv4_fragmentation.c +174 -0

177 lines not on this page · raw message

lib/ip_frag/version.map +1 -0
index b9c1cca..8aad839 100644
@@ -17,4 +17,5 @@ EXPERIMENTAL {
 	global:
 
 	rte_ip_frag_table_del_expired_entries;
+	rte_ipv4_fragment_copy_nonseg_packet;
 };
Signature · 1 line
1.8.3.1
Acked-by on v7 1/1
#24 lore raw

08/08/2022 02:48, Huichao Cai пишет:

Quoted 8 lines

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool,has refcnt = 1,direct and non-segmented.)offload. In order to adapt to this offload function, add this API. Add some test data for this API.

---

#25 lore raw

09/08/2022 00:29, Konstantin Ananyev:

Quoted 9 lines

08/08/2022 02:48, Huichao Cai пишет:

Quoted 7 lines

Some NIC drivers support MBUF_FAST_FREE(Device supports optimization for fast release of mbufs. When set application must guarantee that per-queue all mbufs comes from the same mempool,has refcnt = 1,direct and non-segmented.)offload. In order to adapt to this offload function, add this API. Add some test data for this API.

Applied, thanks.