Mbuf Structure Rework, part 2
cover Mbuf Structure Rework, part 2
This patch set continues on from the changes in part 1, and depends upon that patch set.
This patch set reorders the fields in the mbuf structure and splits the structure across two cache lines, given lots of new space for new fields to be added. This set uses some of that space by expanding the ol_flags field and by adding in a packet_type field. A part 3 patchset is planned to introduce other new fields into the new mbuf structure.
With the splitting of the mbuf across multiple cache lines, performance degradations are seen inside the drivers, both fast-path and slow path. For the fast-path, this patchset reworks the way in which the pool pointer is used to free packets post-TX, which removes the perf regression. For the slow path, an alternative approach is taken - a new scattered packets RX function is introduced into the vector PMD. Using this function, throughput for the slow path RX-TX using testpmd is increased by up to 20% over the original baseline.
Bruce Richardson (12):
mbuf: reorder fields by time of use
mbuf: add packet_type field
mbuf: expand ol_flags field to 64-bits
mbuf: introduce a flag to indicate a control mbuf
mbuf: minor changes for readability
mbuf: use macros only to access the mbuf metadata
mbuf: add named points inside the mbuf structure
ixgbe: rework vector pmd following mbuf changes
mbuf: split mbuf across two cache lines.
mbuf: move l2_len and l3_len to second cache line
ixgbe: Fix perf regression due to moved pool ptr
ixgbe: Improve slow-path perf: vector scattered RX
Olivier Matz (1):
mbuf: replace data pointer by an offset
app/test-pmd/csumonly.c | 2 +- app/test-pmd/flowgen.c | 2 +- app/test-pmd/icmpecho.c | 2 +- app/test-pmd/ieee1588fwd.c | 4 +- app/test-pmd/macfwd-retry.c | 2 +- app/test-pmd/macfwd.c | 2 +- app/test-pmd/macswap.c | 2 +- app/test-pmd/rxonly.c | 2 +- app/test-pmd/testpmd.c | 2 +- app/test-pmd/txonly.c | 7 +- app/test/packet_burst_generator.c | 7 +- app/test/test_mbuf.c | 8 +- app/test/test_table_acl.c | 7 +- app/test/test_table_pipeline.c | 8 +- examples/exception_path/main.c | 3 +- examples/vhost/main.c | 37 +-- examples/vhost_xen/main.c | 14 +- lib/librte_ip_frag/rte_ipv4_fragmentation.c | 6 +- lib/librte_ip_frag/rte_ipv6_fragmentation.c | 6 +- lib/librte_mbuf/rte_mbuf.c | 10 +- lib/librte_mbuf/rte_mbuf.h | 127 +++++---- lib/librte_pmd_bond/rte_eth_bond_pmd.c | 4 +- lib/librte_pmd_e1000/em_rxtx.c | 12 +- lib/librte_pmd_e1000/igb_rxtx.c | 13 +- lib/librte_pmd_i40e/i40e_rxtx.c | 17 +- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 56 ++-- lib/librte_pmd_ixgbe/ixgbe_rxtx.h | 22 +- lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c | 390 +++++++++++++++++----------- lib/librte_pmd_pcap/rte_eth_pcap.c | 9 +- lib/librte_pmd_virtio/virtio_rxtx.c | 10 +- lib/librte_pmd_virtio/virtqueue.h | 3 +- lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c | 5 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 33 files changed, 466 insertions(+), 337 deletions(-)
Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): 1409759378-10113-1-git-send-email-bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
PATCH 1/13 mbuf: replace data pointer by an offset
From: Olivier Matz <[email protected]>
Original patch:
The mbuf structure already contains a pointer to the beginning of the buffer (m->buf_addr). It is not needed to use 8 bytes again to store another pointer to the beginning of the data.
Using a 16 bits unsigned integer is enough as we know that a mbuf is never longer than 64KB. We gain 6 bytes in the structure thanks to this modification.
This version:
* Updated original patch to apply to latest on mainline.
* Disabled vector PMD in config as it relies heavily on the mbuf layout This will be re-enabled in a subsequent commit once vPMD has been reworked to take account of mbuf changes.
--- app/test-pmd/csumonly.c | 2 +- app/test-pmd/flowgen.c | 2 +- app/test-pmd/icmpecho.c | 2 +- app/test-pmd/ieee1588fwd.c | 4 +-- app/test-pmd/macfwd-retry.c | 2 +- app/test-pmd/macfwd.c | 2 +- app/test-pmd/macswap.c | 2 +- app/test-pmd/rxonly.c | 2 +- app/test-pmd/testpmd.c | 2 +- app/test-pmd/txonly.c | 7 ++--- app/test/packet_burst_generator.c | 7 ++--- app/test/test_mbuf.c | 6 ++--- app/test/test_table_acl.c | 7 ++--- app/test/test_table_pipeline.c | 8 ++---- config/common_linuxapp | 2 +- examples/exception_path/main.c | 3 ++- examples/vhost/main.c | 37 +++++++++++++------------ examples/vhost_xen/main.c | 14 +++++----- lib/librte_ip_frag/rte_ipv4_fragmentation.c | 6 ++--- lib/librte_ip_frag/rte_ipv6_fragmentation.c | 6 ++--- lib/librte_mbuf/rte_mbuf.c | 6 ++--- lib/librte_mbuf/rte_mbuf.h | 42 +++++++++++++---------------- lib/librte_pmd_bond/rte_eth_bond_pmd.c | 4 +-- lib/librte_pmd_e1000/em_rxtx.c | 12 ++++----- lib/librte_pmd_e1000/igb_rxtx.c | 13 +++++---- lib/librte_pmd_i40e/i40e_rxtx.c | 17 ++++++------ lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 13 ++++----- lib/librte_pmd_ixgbe/ixgbe_rxtx.h | 3 +-- lib/librte_pmd_pcap/rte_eth_pcap.c | 9 ++++--- lib/librte_pmd_virtio/virtio_rxtx.c | 10 +++---- lib/librte_pmd_virtio/virtqueue.h | 3 +-- lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c | 5 ++-- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 33 files changed, 130 insertions(+), 132 deletions(-)
app/test-pmd/csumonly.c +1 -1
@@ -263,7 +263,7 @@ pkt_burst_checksum_forward(struct fwd_stream *fs) pkt_ol_flags = mb->ol_flags; ol_flags = (uint16_t ) (pkt_ol_flags & (~PKT_TX_L4_MASK )); - eth_hdr = (struct ether_hdr *) mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *); eth_type =rte_be_to_cpu_16 (eth_hdr->ether_type);if (eth_type ==ETHER_TYPE_VLAN ) {/* Only allow single VLAN label here */
app/test-pmd/flowgen.c +1 -1
@@ -175,7 +175,7 @@ pkt_burst_flow_gen(struct fwd_stream *fs) pkt->next =NULL ;/* Initialize Ethernet header. */ - eth_hdr = (struct ether_hdr *)pkt->data; + eth_hdr = rte_pktmbuf_mtod(pkt, struct ether_hdr *);ether_addr_copy (&cfg_ether_dst, ð_hdr->d_addr);ether_addr_copy (&cfg_ether_src, ð_hdr->s_addr); eth_hdr->ether_type =rte_cpu_to_be_16 (ETHER_TYPE_IPv4);
app/test-pmd/icmpecho.c +1 -1
@@ -330,7 +330,7 @@ reply_to_icmp_echo_rqsts(struct fwd_stream *fs) nb_replies =0 ;for (i =0 ; i < nb_rx; i++) { pkt = pkts_burst[i]; - eth_h = (struct ether_hdr *) pkt->data; + eth_h = rte_pktmbuf_mtod(pkt, struct ether_hdr *); eth_type =RTE_BE_TO_CPU_16 (eth_h->ether_type); l2_len =sizeof (struct ether_hdr );if (verbose_level >0 ) {
app/test-pmd/ieee1588fwd.c +2 -2
@@ -546,7 +546,7 @@ ieee1588_packet_fwd(struct fwd_stream *fs)* Check that the received packet is a PTP packet that was detected * by the hardware. */ - eth_hdr = (struct ether_hdr *)mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *); eth_type =rte_be_to_cpu_16 (eth_hdr->ether_type);if (! (mb->ol_flags &PKT_RX_IEEE1588_PTP )) {if (eth_type ==ETHER_TYPE_1588 ) {
@@ -574,7 +574,7 @@ ieee1588_packet_fwd(struct fwd_stream *fs)* Check that the received PTP packet is a PTP V2 packet of type * PTP_SYNC_MESSAGE. */ - ptp_hdr = (struct ptpv2_msg *) ((char *) mb->data + + ptp_hdr = (struct ptpv2_msg *) (rte_pktmbuf_mtod(mb, char *) +sizeof (struct ether_hdr ));if (ptp_hdr->version !=0x02 ) { printf("Port %u Received PTP V2 Ethernet frame with wrong PTP"
app/test-pmd/macfwd-retry.c +1 -1
@@ -119,7 +119,7 @@ pkt_burst_mac_retry_forward(struct fwd_stream *fs) fs->rx_packets += nb_rx;for (i =0 ; i < nb_rx; i++) { mb = pkts_burst[i]; - eth_hdr = (struct ether_hdr *) mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *);ether_addr_copy (&peer_eth_addrs[fs->peer_addr], ð_hdr->d_addr); ether_addr_copy(&ports[fs->tx_port].eth_addr,
app/test-pmd/macfwd.c +1 -1
@@ -110,7 +110,7 @@ pkt_burst_mac_forward(struct fwd_stream *fs) txp = &ports[fs->tx_port];for (i =0 ; i < nb_rx; i++) { mb = pkts_burst[i]; - eth_hdr = (struct ether_hdr *) mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *);ether_addr_copy (&peer_eth_addrs[fs->peer_addr], ð_hdr->d_addr); ether_addr_copy(&ports[fs->tx_port].eth_addr,
app/test-pmd/macswap.c +1 -1
@@ -110,7 +110,7 @@ pkt_burst_mac_swap(struct fwd_stream *fs) txp = &ports[fs->tx_port];for (i =0 ; i < nb_rx; i++) { mb = pkts_burst[i]; - eth_hdr = (struct ether_hdr *) mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *);/* Swap dest and src mac addresses. */ ether_addr_copy (ð_hdr->d_addr, &addr);
app/test-pmd/rxonly.c +1 -1
@@ -149,7 +149,7 @@ pkt_burst_receive(struct fwd_stream *fs)rte_pktmbuf_free (mb);continue ; } - eth_hdr = (struct ether_hdr *) mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *); eth_type =RTE_BE_TO_CPU_16 (eth_hdr->ether_type); ol_flags = mb->ol_flags;print_ether_addr (" src=" , ð_hdr->s_addr);
app/test-pmd/testpmd.c +1 -1
@@ -404,7 +404,7 @@ testpmd_mbuf_ctor(struct rte_mempool *mp, mb_ctor_arg->seg_buf_offset); mb->buf_len = mb_ctor_arg->seg_buf_size; mb->ol_flags =0 ; - mb->data = (char *) mb->buf_addr +RTE_PKTMBUF_HEADROOM ; + mb->data_off =RTE_PKTMBUF_HEADROOM ; mb->nb_segs =1 ; mb->l2_len =0 ; mb->l3_len =0 ;
app/test-pmd/txonly.c +4 -3
@@ -111,13 +111,13 @@ copy_buf_to_pkt_segs(void* buf, unsigned len, struct rte_mbuf *pkt, seg = seg->next; } copy_len = seg->data_len - offset; - seg_buf = ((char *) seg->data + offset); + seg_buf = (rte_pktmbuf_mtod(seg, char *) + offset);while (len > copy_len) {rte_memcpy (seg_buf, buf, (size_t ) copy_len); len -= copy_len; buf = ((char *) buf + copy_len); seg = seg->next; - seg_buf = seg->data; + seg_buf = rte_pktmbuf_mtod(seg, char *); }rte_memcpy (seg_buf, buf, (size_t ) len); }
@@ -126,7 +126,8 @@ static inline void copy_buf_to_pkt(void* buf, unsigned len, struct rte_mbuf *pkt, unsigned offset) {if (offset + len <= pkt->data_len) { -rte_memcpy (((char *) pkt->data + offset), buf, (size_t ) len); + rte_memcpy((rte_pktmbuf_mtod(pkt, char *) + offset), + buf, (size_t ) len);return ; }copy_buf_to_pkt_segs (buf, len, pkt, offset);
app/test/packet_burst_generator.c +4 -3
@@ -59,13 +59,13 @@ copy_buf_to_pkt_segs(void *buf, unsigned len, struct rte_mbuf *pkt, seg = seg->next; } copy_len = seg->data_len - offset; - seg_buf = ((char *) seg->data + offset); + seg_buf = rte_pktmbuf_mtod(seg, char *) + offset;while (len > copy_len) {rte_memcpy (seg_buf, buf, (size_t ) copy_len); len -= copy_len; buf = ((char *) buf + copy_len); seg = seg->next; - seg_buf = seg->data; + seg_buf = rte_pktmbuf_mtod(seg, void *); }rte_memcpy (seg_buf, buf, (size_t ) len); }
@@ -74,7 +74,8 @@ static inline void copy_buf_to_pkt(void *buf, unsigned len, struct rte_mbuf *pkt, unsigned offset) {if (offset + len <= pkt->data_len) { -rte_memcpy (((char *) pkt->data + offset), buf, (size_t ) len); + rte_memcpy(rte_pktmbuf_mtod(pkt, char *) + offset, + buf, (size_t) len);return ; }copy_buf_to_pkt_segs (buf, len, pkt, offset);
app/test/test_mbuf.c +3 -3
@@ -432,7 +432,7 @@ test_pktmbuf_pool_ptr(void)printf ("rte_pktmbuf_alloc() failed (%u)\n" , i); ret =-1 ; } - m[i]->data =RTE_PTR_ADD (m[i]->data,64 ); + m[i]->data_off +=64 ; }/* free them */
@@ -451,8 +451,8 @@ test_pktmbuf_pool_ptr(void)printf ("rte_pktmbuf_alloc() failed (%u)\n" , i); ret =-1 ; } -if (m[i]->data !=RTE_PTR_ADD (m[i]->buf_addr,RTE_PKTMBUF_HEADROOM )) { -printf ("data pointer not set properly\n" ); +if (m[i]->data_off !=RTE_PKTMBUF_HEADROOM ) { +printf ("invalid data_off\n" ); ret =-1 ; } }
app/test/test_table_acl.c +4 -3
@@ -513,7 +513,7 @@ test_pipeline_single_filter(int expected_count)struct rte_mbuf *mbuf; mbuf =rte_pktmbuf_alloc (pool); -memset (mbuf->data,0x00 , + memset(rte_pktmbuf_mtod(mbuf, char *), 0x00, sizeof(struct ipv4_5tuple)); five_tuple.proto = j;
@@ -522,7 +522,7 @@ test_pipeline_single_filter(int expected_count) five_tuple.port_src =rte_bswap16 (100 + j); five_tuple.port_dst =rte_bswap16 (200 + j); -memcpy (mbuf->data, &five_tuple, + memcpy(rte_pktmbuf_mtod(mbuf, char *), &five_tuple, sizeof(struct ipv4_5tuple));RTE_LOG (INFO ,PIPELINE ,"%s: Enqueue onto ring %d\n" , __func__, i);
@@ -549,7 +549,8 @@ test_pipeline_single_filter(int expected_count)printf ("Got %d object(s) from ring %d!\n" , ret, i);for (j =0 ; j < ret; j++) { mbuf = (struct rte_mbuf *)objs[j]; -rte_hexdump (stdout,"mbuf" , mbuf->data,64 ); +rte_hexdump (stdout,"mbuf" , + rte_pktmbuf_mtod(mbuf, char *), 64);rte_pktmbuf_free (mbuf); } tx_count += ret;
app/test/test_table_pipeline.c +2 -6
@@ -39,11 +39,6 @@#include "test_table.h" #include "test_table_pipeline.h" -#define RTE_CBUF_UINT8_PTR (cbuf, offset) \ - (&cbuf->data[offset]) -#define RTE_CBUF_UINT32_PTR (cbuf, offset) \ - (&cbuf->data32[offset/sizeof(uint32_t)]) -#if 0 static rte_pipeline_port_out_action_handler port_action_0x00
@@ -498,7 +493,8 @@ test_pipeline_single_filter(int test_type, int expected_count)printf ("Got %d object(s) from ring %d!\n" , ret, i);for (j =0 ; j < ret; j++) { mbuf = (struct rte_mbuf *)objs[j]; -rte_hexdump (stdout,"Object:" , mbuf->data, +rte_hexdump (stdout,"Object:" , + rte_pktmbuf_mtod(mbuf, char *), mbuf->data_len);rte_pktmbuf_free (mbuf); }
config/common_linuxapp +1 -1
@@ -191,7 +191,7 @@ CONFIG_RTE_LIBRTE_IXGBE_DEBUG_DRIVER=n CONFIG_RTE_LIBRTE_IXGBE_PF_DISABLE_STRIP_CRC=n CONFIG_RTE_LIBRTE_IXGBE_RX_ALLOW_BULK_ALLOC=y CONFIG_RTE_LIBRTE_IXGBE_ALLOW_UNSUPPORTED_SFP=n -CONFIG_RTE_IXGBE_INC_VECTOR=y +CONFIG_RTE_IXGBE_INC_VECTOR=n CONFIG_RTE_IXGBE_RX_OLFLAGS_ENABLE=y #
examples/exception_path/main.c +2 -1
@@ -302,7 +302,8 @@ main_loop(__attribute__((unused)) void *arg)if (m ==NULL )continue ; - ret =read (tap_fd, m->data,MAX_PACKET_SZ ); + ret = read(tap_fd, rte_pktmbuf_mtod(m, void *), +MAX_PACKET_SZ ); lcore_stats[lcore_id].rx++;if (unlikely (ret <0 )) { FATAL_ERROR("Reading from %s interface failed",
examples/vhost/main.c +20 -17
@@ -1033,7 +1033,7 @@ virtio_dev_rx(struct virtio_net *dev, struct rte_mbuf **pkts, uint32_t count)/* Copy mbuf data to buffer */ rte_memcpy ((void *)(uintptr_t )buff_addr, - (const void *)buff->data, + rte_pktmbuf_mtod(buff, const void *),rte_pktmbuf_data_len (buff));PRINT_PACKET (dev, (uintptr_t )buff_addr,rte_pktmbuf_data_len (buff),0 );
@@ -1438,7 +1438,7 @@ link_vmdq(struct virtio_net *dev, struct rte_mbuf *m)int i, ret;/* Learn MAC address of guest device from packet */ - pkt_hdr = (struct ether_hdr *)m->data; + pkt_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *); dev_ll = ll_root_used;
@@ -1525,7 +1525,7 @@ virtio_tx_local(struct virtio_net *dev, struct rte_mbuf *m)struct ether_hdr *pkt_hdr;uint64_t ret =0 ; - pkt_hdr = (struct ether_hdr *)m->data; + pkt_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *);/*get the used devices list*/ dev_ll = ll_root_used;
@@ -1593,7 +1593,7 @@ virtio_tx_route(struct virtio_net* dev, struct rte_mbuf *m, struct rte_mempool *unsigned len, ret, offset =0 ;const uint16_t lcore_id =rte_lcore_id ();struct virtio_net_data_ll *dev_ll = ll_root_used; -struct ether_hdr *pkt_hdr = (struct ether_hdr *)m->data; + struct ether_hdr *pkt_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *);/*check if destination is local VM*/ if ((vm2vm_mode ==VM2VM_SOFTWARE ) && (virtio_tx_local (dev, m) ==0 ))
@@ -1652,18 +1652,21 @@ virtio_tx_route(struct virtio_net* dev, struct rte_mbuf *m, struct rte_mempool * mbuf->nb_segs = m->nb_segs;/* Copy ethernet header to mbuf. */ -rte_memcpy ((void *)mbuf->data, (const void *)m->data,ETH_HLEN ); + rte_memcpy(rte_pktmbuf_mtod(mbuf, void *), +rte_pktmbuf_mtod (m ,const void *), + ETH_HLEN);/* Setup vlan header. Bytes need to be re-ordered for network with htons()*/ - vlan_hdr = (struct vlan_ethhdr *) mbuf->data; + vlan_hdr = rte_pktmbuf_mtod(mbuf, struct vlan_ethhdr *); vlan_hdr->h_vlan_encapsulated_proto = vlan_hdr->h_vlan_proto; vlan_hdr->h_vlan_proto =htons (ETH_P_8021Q ); vlan_hdr->h_vlan_TCI =htons (vlan_tag);/* Copy the remaining packet contents to the mbuf. */ -rte_memcpy ((void *) ((uint8_t *)mbuf->data +VLAN_ETH_HLEN ), - (const void *) ((uint8_t *)m->data +ETH_HLEN ), (m->data_len -ETH_HLEN )); + rte_memcpy((void *)(rte_pktmbuf_mtod(mbuf, uint8_t *) + VLAN_ETH_HLEN), + (const void *)(rte_pktmbuf_mtod(m, uint8_t *) + ETH_HLEN), + (m->data_len -ETH_HLEN ));/* Copy the remaining segments for the whole packet. */ prev = mbuf;
@@ -1778,7 +1781,7 @@ virtio_dev_tx(struct virtio_net* dev, struct rte_mempool *mbuf_pool)/* Setup dummy mbuf. This is copied to a real mbuf if transmitted out the physical port. */ m.data_len = desc->len; m.pkt_len = desc->len; - m.data = (void *)(uintptr_t )buff_addr; + m.data_off =0 ;PRINT_PACKET (dev, (uintptr_t )buff_addr, desc->len,0 );
@@ -2333,7 +2336,7 @@ attach_rxmbuf_zcp(struct virtio_net *dev) } mbuf->buf_addr = (void *)(uintptr_t )(buff_addr -RTE_PKTMBUF_HEADROOM ); - mbuf->data = (void *)(uintptr_t )(buff_addr); + mbuf->data_off =RTE_PKTMBUF_HEADROOM ; mbuf->buf_physaddr = phys_addr -RTE_PKTMBUF_HEADROOM ; mbuf->data_len = desc->len;MBUF_HEADROOM_UINT32 (mbuf) = (uint32_t )desc_idx;
@@ -2370,7 +2373,7 @@ static inline void pktmbuf_detach_zcp(struct rte_mbuf *m) buf_ofs = (RTE_PKTMBUF_HEADROOM <= m->buf_len) ?RTE_PKTMBUF_HEADROOM : m->buf_len; - m->data = (char *) m->buf_addr + buf_ofs; + m->data_off = buf_ofs; m->data_len =0 ; }
@@ -2604,7 +2607,7 @@ virtio_tx_route_zcp(struct virtio_net *dev, struct rte_mbuf *m,unsigned len, ret, offset =0 ;struct vpool *vpool;struct virtio_net_data_ll *dev_ll = ll_root_used; -struct ether_hdr *pkt_hdr = (struct ether_hdr *)m->data; + struct ether_hdr *pkt_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *);uint16_t vlan_tag = (uint16_t )vlan_tags[(uint16_t )dev->device_fh];/*Add packet to the port tx queue*/
@@ -2681,11 +2684,11 @@ virtio_tx_route_zcp(struct virtio_net *dev, struct rte_mbuf *m, mbuf->pkt_len = mbuf->data_len;if (unlikely (need_copy)) {/* Copy the packet contents to the mbuf. */ -rte_memcpy ((void *)((uint8_t *)mbuf->data), - (const void *) ((uint8_t *)m->data), + rte_memcpy(rte_pktmbuf_mtod(mbuf, void *), +rte_pktmbuf_mtod (m ,void *), m->data_len); }else { - mbuf->data = m->data; + mbuf->data_off = m->data_off; mbuf->buf_physaddr = m->buf_physaddr; mbuf->buf_addr = m->buf_addr; }
@@ -2819,8 +2822,8 @@ virtio_dev_tx_zcp(struct virtio_net *dev) m.data_len = desc->len; m.nb_segs =1 ; m.next =NULL ; - m.data = (void *)(uintptr_t )buff_addr; - m.buf_addr = m.data; + m.data_off =0 ; + m.buf_addr = (void *)(uintptr_t )buff_addr; m.buf_physaddr = phys_addr;/*
examples/vhost_xen/main.c +8 -6
@@ -808,7 +808,7 @@ virtio_tx_local(struct virtio_net *dev, struct rte_mbuf *m)struct ether_hdr *pkt_hdr;uint64_t ret =0 ; - pkt_hdr = (struct ether_hdr *)m->data; + pkt_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *);/*get the used devices list*/ dev_ll = ll_root_used;
@@ -883,18 +883,20 @@ virtio_tx_route(struct virtio_net* dev, struct rte_mbuf *m, struct rte_mempool * mbuf->pkt_len = mbuf->data_len;/* Copy ethernet header to mbuf. */ -rte_memcpy ((void *)mbuf->data, (const void *)m->data,ETH_HLEN ); + rte_memcpy(rte_pktmbuf_mtod(mbuf, void*), + rte_pktmbuf_mtod(m, const void*), ETH_HLEN);/* Setup vlan header. Bytes need to be re-ordered for network with htons()*/ - vlan_hdr = (struct vlan_ethhdr *) mbuf->data; + vlan_hdr = rte_pktmbuf_mtod(mbuf, struct vlan_ethhdr *); vlan_hdr->h_vlan_encapsulated_proto = vlan_hdr->h_vlan_proto; vlan_hdr->h_vlan_proto =htons (ETH_P_8021Q ); vlan_hdr->h_vlan_TCI =htons (vlan_tag);/* Copy the remaining packet contents to the mbuf. */ -rte_memcpy ((void *) ((uint8_t *)mbuf->data +VLAN_ETH_HLEN ), - (const void *) ((uint8_t *)m->data +ETH_HLEN ), (m->data_len -ETH_HLEN )); + rte_memcpy((void *)(rte_pktmbuf_mtod(mbuf, uint8_t *) + VLAN_ETH_HLEN), + (const void *)(rte_pktmbuf_mtod(m, uint8_t *) + ETH_HLEN), + (m->data_len -ETH_HLEN )); tx_q->m_table[len] = mbuf; len++;if (enable_stats) {
@@ -981,7 +983,7 @@ virtio_dev_tx(struct virtio_net* dev, struct rte_mempool *mbuf_pool)/* Setup dummy mbuf. This is copied to a real mbuf if transmitted out the physical port. */ m.data_len = desc->len; - m.data = (void *)(uintptr_t )buff_addr; + m.data_off =0 ; m.nb_segs =1 ;virtio_tx_route (dev, &m, mbuf_pool,0 );
lib/librte_ip_frag/rte_ipv4_fragmentation.c +3 -3
@@ -109,7 +109,7 @@ rte_ipv4_fragment_packet(struct rte_mbuf *pkt_in,/* Fragment size should be a multiply of 8. */ IP_FRAG_ASSERT ((frag_size &IPV4_HDR_FO_MASK ) ==0 ); - in_hdr = (struct ipv4_hdr *) pkt_in->data; + in_hdr = rte_pktmbuf_mtod(pkt_in, struct ipv4_hdr *); flag_offset =rte_cpu_to_be_16 (in_hdr->fragment_offset);/* If Don't Fragment flag is set */
@@ -165,7 +165,7 @@ rte_ipv4_fragment_packet(struct rte_mbuf *pkt_in,if (len > (in_seg->data_len - in_seg_data_pos)) { len = in_seg->data_len - in_seg_data_pos; } - out_seg->data = (char *) in_seg->data + (uint16_t )in_seg_data_pos; + out_seg->data_off = in_seg->data_off + in_seg_data_pos; out_seg->data_len = (uint16_t )len; out_pkt->pkt_len = (uint16_t )(len + out_pkt->pkt_len);
@@ -188,7 +188,7 @@ rte_ipv4_fragment_packet(struct rte_mbuf *pkt_in,/* Build the IP header */ - out_hdr = (struct ipv4_hdr *) out_pkt->data; + out_hdr = rte_pktmbuf_mtod(out_pkt, struct ipv4_hdr *); __fill_ipv4hdr_frag(out_hdr, in_hdr, (uint16_t)out_pkt->pkt_len,
lib/librte_ip_frag/rte_ipv6_fragmentation.c +3 -3
@@ -125,7 +125,7 @@ rte_ipv6_fragment_packet(struct rte_mbuf *pkt_in, (uint16_t)(pkt_in->pkt_len - sizeof (struct ipv6_hdr)))) return (-EINVAL); - in_hdr = (struct ipv6_hdr *) pkt_in->data; + in_hdr = rte_pktmbuf_mtod(pkt_in, struct ipv6_hdr *); in_seg = pkt_in; in_seg_data_pos =sizeof (struct ipv6_hdr );
@@ -171,7 +171,7 @@ rte_ipv6_fragment_packet(struct rte_mbuf *pkt_in,if (len > (in_seg->data_len - in_seg_data_pos)) { len = in_seg->data_len - in_seg_data_pos; } - out_seg->data = (char *) in_seg->data + (uint16_t ) in_seg_data_pos; + out_seg->data_off = in_seg->data_off + in_seg_data_pos; out_seg->data_len = (uint16_t )len; out_pkt->pkt_len = (uint16_t )(len + out_pkt->pkt_len);
@@ -196,7 +196,7 @@ rte_ipv6_fragment_packet(struct rte_mbuf *pkt_in,/* Build the IP header */ - out_hdr = (struct ipv6_hdr *) out_pkt->data; + out_hdr = rte_pktmbuf_mtod(out_pkt, struct ipv6_hdr *); __fill_ipv6hdr_frag(out_hdr, in_hdr, (uint16_t) out_pkt->pkt_len - sizeof(struct ipv6_hdr),
lib/librte_mbuf/rte_mbuf.c +3 -3
@@ -117,7 +117,7 @@ rte_pktmbuf_init(struct rte_mempool *mp, m->buf_len = (uint16_t )buf_len;/* keep some headroom between start of buffer and data */ - m->data = (char *) m->buf_addr +RTE_MIN (RTE_PKTMBUF_HEADROOM , m->buf_len); + m->data_off =RTE_MIN (RTE_PKTMBUF_HEADROOM , (uint16_t )m->buf_len);/* init some constant fields */ m->pool = mp;
@@ -183,12 +183,12 @@ rte_pktmbuf_dump(FILE *f, const struct rte_mbuf *m, unsigned dump_len)__rte_mbuf_sanity_check (m,0 );fprintf (f," segment at 0x%p, data=0x%p, data_len=%u\n" , - m, m->data, (unsigned )m->data_len); + m, rte_pktmbuf_mtod(m, void *), (unsigned)m->data_len); len = dump_len;if (len > m->data_len) len = m->data_len;if (len !=0 ) -rte_hexdump (f,NULL , m->data, len); + rte_hexdump(f, NULL, rte_pktmbuf_mtod(m, void *), len); dump_len -= len; m = m->next; nb_segs --;
lib/librte_mbuf/rte_mbuf.h +19 -23
135 lines not on this page · raw message
lib/librte_pmd_bond/rte_eth_bond_pmd.c +2 -2
@@ -198,14 +198,14 @@ xmit_slave_hash(const struct rte_mbuf *buf, uint8_t slave_count, uint8_t policy)switch (policy) {case BALANCE_XMIT_POLICY_LAYER2 : - eth_hdr = (struct ether_hdr *)buf->data; + eth_hdr = rte_pktmbuf_mtod(buf, struct ether_hdr *); hash =ether_hash (eth_hdr); hash ^= hash >>8 ;return hash % slave_count;case BALANCE_XMIT_POLICY_LAYER23 : - eth_hdr = (struct ether_hdr *)buf->data; + eth_hdr = rte_pktmbuf_mtod(buf, struct ether_hdr *);if (buf->ol_flags &PKT_RX_VLAN_PKT ) eth_offset =sizeof (struct ether_hdr ) +sizeof (struct vlan_hdr );
lib/librte_pmd_e1000/em_rxtx.c +6 -6
@@ -90,8 +90,7 @@ rte_rxmbuf_alloc(struct rte_mempool *mp) }#define RTE_MBUF_DATA_DMA_ADDR (mb) \ - (uint64_t) ((mb)->buf_physaddr + \ - (uint64_t) ((char *)((mb)->data) - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off)#define RTE_MBUF_DATA_DMA_ADDR_DEFAULT (mb) \ (uint64_t) ((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM)
@@ -793,8 +792,8 @@ eth_em_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,*/ pkt_len = (uint16_t ) (rte_le_to_cpu_16 (rxd.length) - rxq->crc_len); - rxm->data = (char *) rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; -rte_packet_prefetch (rxm->data); + rxm->data_off =RTE_PKTMBUF_HEADROOM ; +rte_packet_prefetch ((char *)rxm->buf_addr + rxm->data_off); rxm->nb_segs =1 ; rxm->next =NULL ; rxm->pkt_len = pkt_len;
@@ -963,7 +962,7 @@ eth_em_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,*/ data_len =rte_le_to_cpu_16 (rxd.length); rxm->data_len = data_len; - rxm->data = (char *) rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ;/* * If this is the first buffer of the received packet,
@@ -1035,7 +1034,8 @@ eth_em_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rxm->vlan_tci =rte_le_to_cpu_16 (rxd.special);/* Prefetch data of first segment, if configured to do so. */ -rte_packet_prefetch (first_seg->data); +rte_packet_prefetch ((char *)first_seg->buf_addr + + first_seg->data_off);/* * Store the mbuf address into the next entry of the array
lib/librte_pmd_e1000/igb_rxtx.c +6 -7
@@ -95,9 +95,7 @@ rte_rxmbuf_alloc(struct rte_mempool *mp) }#define RTE_MBUF_DATA_DMA_ADDR (mb) \ - (uint64_t) ((mb)->buf_physaddr + \ - (uint64_t) ((char *)((mb)->data) - \ - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off)#define RTE_MBUF_DATA_DMA_ADDR_DEFAULT (mb) \ (uint64_t) ((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM)
@@ -776,8 +774,8 @@ eth_igb_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,*/ pkt_len = (uint16_t ) (rte_le_to_cpu_16 (rxd.wb.upper.length) - rxq->crc_len); - rxm->data = (char *) rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; -rte_packet_prefetch (rxm->data); + rxm->data_off =RTE_PKTMBUF_HEADROOM ; +rte_packet_prefetch ((char *)rxm->buf_addr + rxm->data_off); rxm->nb_segs =1 ; rxm->next =NULL ; rxm->pkt_len = pkt_len;
@@ -952,7 +950,7 @@ eth_igb_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,*/ data_len =rte_le_to_cpu_16 (rxd.wb.upper.length); rxm->data_len = data_len; - rxm->data = (char *) rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ;/* * If this is the first buffer of the received packet,
@@ -1033,7 +1031,8 @@ eth_igb_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, first_seg->ol_flags = pkt_flags;/* Prefetch data of first segment, if configured to do so. */ -rte_packet_prefetch (first_seg->data); +rte_packet_prefetch ((char *)first_seg->buf_addr + + first_seg->data_off);/* * Store the mbuf address into the next entry of the array
lib/librte_pmd_i40e/i40e_rxtx.c +8 -9
@@ -78,9 +78,7 @@ (uint64_t) ((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM)#define RTE_MBUF_DATA_DMA_ADDR (mb) \ - ((uint64_t)((mb)->buf_physaddr + \ - (uint64_t)((char *)((mb)->data) - \ - (char *)(mb)->buf_addr))) + ((uint64_t)((mb)->buf_physaddr + (mb)->data_off))static const struct rte_memzone * i40e_ring_dma_zone_reserve(struct rte_eth_dev *dev,
@@ -685,7 +683,7 @@ i40e_rx_alloc_bufs(struct i40e_rx_queue *rxq) mb = rxep[i].mbuf;rte_mbuf_refcnt_set (mb,1 ); mb->next =NULL ; - mb->data = (char *)mb->buf_addr +RTE_PKTMBUF_HEADROOM ; + mb->data_off =RTE_PKTMBUF_HEADROOM ; mb->nb_segs =1 ; mb->port = rxq->port_id; dma_addr = rte_cpu_to_le_64(\
@@ -842,8 +840,8 @@ i40e_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) rx_packet_len = ((qword1 &I40E_RXD_QW1_LENGTH_PBUF_MASK ) >>I40E_RXD_QW1_LENGTH_PBUF_SHIFT ) - rxq->crc_len; - rxm->data = (char *)rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; -rte_prefetch0 (rxm->data); + rxm->data_off =RTE_PKTMBUF_HEADROOM ; +rte_prefetch0 (RTE_PTR_ADD (rxm->buf_addr,RTE_PKTMBUF_HEADROOM )); rxm->nb_segs =1 ; rxm->next =NULL ; rxm->pkt_len = rx_packet_len;
@@ -946,7 +944,7 @@ i40e_recv_scattered_pkts(void *rx_queue, rx_packet_len = (qword1 &I40E_RXD_QW1_LENGTH_PBUF_MASK ) >>I40E_RXD_QW1_LENGTH_PBUF_SHIFT ; rxm->data_len = rx_packet_len; - rxm->data = (char *)rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ;/** * If this is the first buffer of the received packet, set the
@@ -1015,7 +1013,8 @@ i40e_recv_scattered_pkts(void *rx_queue,rte_le_to_cpu_32 (rxd.wb.qword0.hi_dword.rss);/* Prefetch data of first segment, if configured to do so. */ -rte_prefetch0 (first_seg->data); +rte_prefetch0 (RTE_PTR_ADD (first_seg->buf_addr, + first_seg->data_off)); rx_pkts[nb_rx++] = first_seg; first_seg =NULL ; }
@@ -2131,7 +2130,7 @@ i40e_alloc_rx_queue_mbufs(struct i40e_rx_queue *rxq)rte_mbuf_refcnt_set (mbuf,1 ); mbuf->next =NULL ; - mbuf->data = (char *)mbuf->buf_addr +RTE_PKTMBUF_HEADROOM ; + mbuf->data_off =RTE_PKTMBUF_HEADROOM ; mbuf->nb_segs =1 ; mbuf->port = rxq->port_id;
lib/librte_pmd_ixgbe/ixgbe_rxtx.c +7 -6
@@ -998,7 +998,7 @@ ixgbe_rx_alloc_bufs(struct igb_rx_queue *rxq) mb = rxep[i].mbuf;rte_mbuf_refcnt_set (mb,1 ); mb->next =NULL ; - mb->data = (char *)mb->buf_addr +RTE_PKTMBUF_HEADROOM ; + mb->data_off =RTE_PKTMBUF_HEADROOM ; mb->nb_segs =1 ; mb->port = rxq->port_id;
@@ -1249,8 +1249,8 @@ ixgbe_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,*/ pkt_len = (uint16_t ) (rte_le_to_cpu_16 (rxd.wb.upper.length) - rxq->crc_len); - rxm->data = (char *) rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; -rte_packet_prefetch (rxm->data); + rxm->data_off =RTE_PKTMBUF_HEADROOM ; +rte_packet_prefetch ((char *)rxm->buf_addr + rxm->data_off); rxm->nb_segs =1 ; rxm->next =NULL ; rxm->pkt_len = pkt_len;
@@ -1432,7 +1432,7 @@ ixgbe_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,*/ data_len =rte_le_to_cpu_16 (rxd.wb.upper.length); rxm->data_len = data_len; - rxm->data = (char *) rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ;/* * If this is the first buffer of the received packet,
@@ -1523,7 +1523,8 @@ ixgbe_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, }/* Prefetch data of first segment, if configured to do so. */ -rte_packet_prefetch (first_seg->data); +rte_packet_prefetch ((char *)first_seg->buf_addr + + first_seg->data_off);/* * Store the mbuf address into the next entry of the array
@@ -3213,7 +3214,7 @@ ixgbe_alloc_rx_queue_mbufs(struct igb_rx_queue *rxq)rte_mbuf_refcnt_set (mbuf,1 ); mbuf->next =NULL ; - mbuf->data = (char *)mbuf->buf_addr +RTE_PKTMBUF_HEADROOM ; + mbuf->data_off =RTE_PKTMBUF_HEADROOM ; mbuf->nb_segs =1 ; mbuf->port = rxq->port_id;
lib/librte_pmd_ixgbe/ixgbe_rxtx.h +1 -2
@@ -47,8 +47,7 @@#endif #define RTE_MBUF_DATA_DMA_ADDR (mb) \ - (uint64_t) ((mb)->buf_physaddr + (uint64_t)((char *)((mb)->data) - \ - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off)#define RTE_MBUF_DATA_DMA_ADDR_DEFAULT (mb) \ (uint64_t) ((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM)
lib/librte_pmd_pcap/rte_eth_pcap.c +6 -3
@@ -151,7 +151,8 @@ eth_pcap_rx(void *queue,if (header.len <= buf_size) {/* pcap packet will fit in the mbuf, go ahead and copy */ -rte_memcpy (mbuf->data, packet, header.len); + rte_memcpy(rte_pktmbuf_mtod(mbuf, void *), packet, + header.len); mbuf->data_len = (uint16_t )header.len; mbuf->pkt_len = mbuf->data_len; bufs[num_rx] = mbuf;
@@ -202,7 +203,8 @@ eth_pcap_tx_dumper(void *queue,calculate_timestamp (&header.ts); header.len = mbuf->data_len; header.caplen = header.len; -pcap_dump ((u_char *) dumper_q->dumper, &header, mbuf->data); +pcap_dump ((u_char *) dumper_q->dumper, &header, + rte_pktmbuf_mtod(mbuf, void*));rte_pktmbuf_free (mbuf); num_tx++; }
@@ -237,7 +239,8 @@ eth_pcap_tx(void *queue,for (i =0 ; i < nb_pkts; i++) { mbuf = bufs[i]; - ret =pcap_sendpacket (tx_queue->pcap, (u_char *) mbuf->data, + ret =pcap_sendpacket (tx_queue->pcap, + rte_pktmbuf_mtod(mbuf, u_char *), mbuf->data_len);if (unlikely (ret !=0 ))break ;
lib/librte_pmd_virtio/virtio_rxtx.c +4 -6
@@ -118,7 +118,7 @@ virtqueue_dequeue_burst_rx(struct virtqueue *vq, struct rte_mbuf **rx_pkts, }rte_prefetch0 (cookie); -rte_packet_prefetch (cookie->data); +rte_packet_prefetch (rte_pktmbuf_mtod(cookie ,void *)); rx_pkts[i] = cookie; vq->vq_used_cons_idx++;vq_ring_free_chain (vq, desc_idx);
@@ -480,7 +480,7 @@ virtio_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) } rxm->port = rxvq->port_id; - rxm->data = (char *)rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ; rxm->nb_segs =1 ; rxm->next =NULL ;
@@ -584,7 +584,7 @@ virtio_recv_mergeable_pkts(void *rx_queue,if (seg_num ==0 ) seg_num =1 ; - rxm->data = (char *)rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ; rxm->nb_segs = seg_num; rxm->next =NULL ; rxm->pkt_len = (uint32_t )(len[0 ] - hdr_size);
@@ -622,9 +622,7 @@ virtio_recv_mergeable_pkts(void *rx_queue,while (extra_idx < rcv_cnt) { rxm = rcv_pkts[extra_idx]; - rxm->data = - (char *)rxm->buf_addr + -RTE_PKTMBUF_HEADROOM - hdr_size; + rxm->data_off =RTE_PKTMBUF_HEADROOM - hdr_size; rxm->next =NULL ; rxm->pkt_len = (uint32_t )(len[extra_idx]); rxm->data_len = (uint16_t )(len[extra_idx]);
lib/librte_pmd_virtio/virtqueue.h +1 -2
@@ -59,8 +59,7 @@#define VIRTQUEUE_MAX_NAME_SZ 32#define RTE_MBUF_DATA_DMA_ADDR (mb) \ - (uint64_t) ((mb)->buf_physaddr + (uint64_t)((char *)((mb)->data) - \ - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off)#define VTNET_SQ_RQ_QUEUE_IDX 0#define VTNET_SQ_TQ_QUEUE_IDX 1
lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c +2 -3
@@ -79,8 +79,7 @@#define RTE_MBUF_DATA_DMA_ADDR (mb) \ - (uint64_t) ((mb)->buf_physaddr + (uint64_t)((char *)((mb)->data) - \ - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off)#define RTE_MBUF_DATA_DMA_ADDR_DEFAULT (mb) \ (uint64_t) ((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM)
@@ -565,7 +564,7 @@ vmxnet3_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) rxm->data_len = (uint16_t )rcd->len; rxm->port = rxq->port_id; rxm->vlan_tci =0 ; - rxm->data = (char *)rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ; rx_pkts[nb_rx++] = rxm;
lib/librte_pmd_xenvirt/rte_eth_xenvirt.c +1 -1
@@ -110,7 +110,7 @@ eth_xenvirt_rx(void *q, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) rxm = rx_pkts[i];PMD_RX_LOG (DEBUG ,"packet len:%d\n" , len[i]); rxm->next =NULL ; - rxm->data = (char *)rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ; rxm->data_len = (uint16_t )(len[i] -sizeof (struct virtio_net_hdr )); rxm->nb_segs =1 ; rxm->port = pi->port_id;
Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): 1409759378-10113-2-git-send-email-bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH 01/13] mbuf: replace data pointer by an offset
Hi Bruce,
Quoted 28 lines · Bruce Richardson
From: Olivier Matz <[email protected]>
Original patch:
The mbuf structure already contains a pointer to the beginning of the buffer (m->buf_addr). It is not needed to use 8 bytes again to store another pointer to the beginning of the data.Using a 16 bits unsigned integer is enough as we know that a mbuf is never longer than 64KB. We gain 6 bytes in the structure thanks to this modification.
Signed-off-by Olivier Matz <[email protected]>This version:
* Updated original patch to apply to latest on mainline.
* Disabled vector PMD in config as it relies heavily on the mbuf layout This will be re-enabled in a subsequent commit once vPMD has been reworked to take account of mbuf changes.Signed-off-by Bruce Richardson <[email protected]>[...]
lib/librte_mbuf/rte_mbuf.h +0
-0
lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 128
Quoted 10 lines
lib/librte_mbuf/rte_mbuf.h +7
-0@@ -119,6 +119,13 @@ struct rte_mbuf {void *buf_addr;/**< Virtual address of segment buffer. */ phys_addr_t buf_physaddr;/**< Physical address of segment buffer. */ uint16_t buf_len;/**< Length of segment buffer. */ + +/* valid for any segment */ +struct rte_mbuf *next;/**< Next segment of scattered packet. */ +uint16_t data_off; +uint16_t data_len;/**< Amount of data in segment buffer. */ +uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ +
Here, the compiler will add some padding between "buf_len" and "next" (6 bytes in 64bits, 2 in 32bits). Shouldn't we add "reserved" fields or at least a comment ? Another idea would be to reorganize the fields to avoid padding, if possible.
Except this remarq, Acked-by: Olivier Matz <[email protected]>
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 01/13] mbuf: replace data pointer by an offset
Quoted 13 lines
Quoted 13 lines
lib/librte_mbuf/rte_mbuf.h +4
-0@@ -119,6 +119,13 @@ struct rte_mbuf {void *buf_addr;/**< Virtual address of segment buffer. */ phys_addr_t buf_physaddr;/**< Physical address of segment buffer. */ uint16_t buf_len;/**< Length of segment buffer. */ + +/* valid for any segment */ +struct rte_mbuf *next;/**< Next segment of scattered packet. */ +uint16_t data_off;
Quoted 8 lines
lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 128
Quoted 3 lines
lib/librte_mbuf/rte_mbuf.h +3
-0+uint16_t data_len;/**< Amount of data in segment buffer. */ +uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ +Here, the compiler will add some padding between "
buf_len" and "next" (6 bytes in 64bits, 2 in 32bits). Shouldn't we add "reserved" fields or at least a comment ? Another idea would be to reorganize the fields to avoid padding, if possible.
I've just seen that patch 02/13 reorders the mbuf fields. So maybe this remarq is useless.
Olivier
PATCH 2/13 mbuf: reorder fields by time of use
In reply to a message not in the archive* Reorder the fields in the mbuf so that we have fields that are used$ together side-by-side in the structure. This means that we have a$ contiguous block of 8-bytes in the mbuf which are used to reset an mbuf$ of descriptor rearm, and a block of 16-bytes of data (excluding flags) which are set on RX from the received packet descriptor.
* Use dummy fields as appropraite to ensure alignment or to reserve gaps for later field additions.
* Place most items which are not used by fast-path RX separately at the end of the structure so they can later be moved to a separate cache line. [The l2/l3 length fields are not moved at this stage as doing so will cause overflow to the next cache line].
--- lib/librte_mbuf/rte_mbuf.h | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-)
lib/librte_mbuf/rte_mbuf.h +14 -11
@@ -115,16 +115,12 @@ extern "C" {* The generic rte_mbuf, containing a packet mbuf. */ struct rte_mbuf { -struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */ void *buf_addr;/**< Virtual address of segment buffer. */ phys_addr_t buf_physaddr;/**< Physical address of segment buffer. */ -uint16_t buf_len;/**< Length of segment buffer. */ -/* valid for any segment */ -struct rte_mbuf *next;/**< Next segment of scattered packet. */ +/* next 8 bytes are initialised on RX descriptor rearm */ +uint16_t buf_len;/**< Length of segment buffer. */ uint16_t data_off; -uint16_t data_len;/**< Amount of data in segment buffer. */ -uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ #ifdef RTE_MBUF_REFCNT /**
@@ -142,14 +138,17 @@ struct rte_mbuf {#else uint16_t refcnt_reserved;/**< Do not use this field */ #endif -uint16_t reserved;/**< Unused field. Required for padding */ -uint16_t ol_flags;/**< Offload features. */ - -/* these fields are valid for first segment only */ uint8_t nb_segs;/**< Number of segments. */ uint8_t port;/**< Input port. */ -/* offload features, valid for first segment only */ +uint16_t ol_flags;/**< Offload features. */ +uint16_t reserved0;/**< Unused field. Required for padding */ +uint32_t reserved1;/**< Unused field. Required for padding */ + +/* remaining bytes are set on RX when pulling packet from descriptor */ +uint16_t reserved2;/**< Unused field. Required for padding */ +uint16_t data_len;/**< Amount of data in segment buffer. */ +uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ uint16_t l3_len:9; /**< L3 (IP) Header Length. */ uint16_t l2_len:7; /**< L2 (MAC) Header Length. */uint16_t vlan_tci;/**< VLAN Tag Control Identifier (CPU order). */
@@ -162,6 +161,10 @@ struct rte_mbuf {uint32_t sched;/**< Hierarchical scheduler */ } hash;/**< hash information */ +/* fields only used in slow path or on TX */ +struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */ +struct rte_mbuf *next;/**< Next segment of scattered packet. */ +union {uint8_t metadata[0 ];uint16_t metadata16[0 ];
Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): 1409759378-10113-3-git-send-email-bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Olivier MATZ
Acked-byRe: [PATCH 02/13] mbuf: reorder fields by time of use
Hi Bruce,
Quoted 5 lines · Bruce Richardson
* Reorder the fields in the mbuf so that we have fields that are used$ together side-by-side in the structure. This means that we have a$ contiguous block of 8-bytes in the mbuf which are used to reset an mbuf$ of descriptor rearm, and a block of 16-bytes of data (excluding flags) which are set on RX from the received packet descriptor.
Some unwanted $ characters here.
Quoted 2 lines
* Use dummy fields as appropraite to ensure alignment or to reserve gaps for later field additions.
s/appropraite/appropriate
Quoted 6 lines
* Place most items which are not used by fast-path RX separately at the end of the structure so they can later be moved to a separate cache line. [The l2/l3 length fields are not moved at this stage as doing so will cause overflow to the next cache line].
Signed-off-by Bruce Richardson <[email protected]>
PATCH 3/13 mbuf: add packet_type field
In reply to a message not in the archiveReplace a reserved slot with the new packet type field used to identify the type of the packet, i.e. what protocols are used.
--- lib/librte_mbuf/rte_mbuf.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_mbuf/rte_mbuf.h +1 -1
@@ -146,7 +146,7 @@ struct rte_mbuf {uint32_t reserved1;/**< Unused field. Required for padding */ /* remaining bytes are set on RX when pulling packet from descriptor */ -uint16_t reserved2;/**< Unused field. Required for padding */ +uint16_t packet_type;/**< Type of packet, e.g. protocols used */ uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ uint16_t l3_len:9; /**< L3 (IP) Header Length. */
Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): 1409759378-10113-4-git-send-email-bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH 03/13] mbuf: add packet_type field
Hi Bruce,
Quoted 14 lines · Bruce Richardson
Replace a reserved slot with the new packet type field used to identify the type of the packet, i.e. what protocols are used.
Signed-off-by Bruce Richardson <[email protected]>--- lib/librte_mbuf/rte_mbuf.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/librte_mbuf/rte_mbuf.h +0
-0
lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 152
Quoted 8 lines
lib/librte_mbuf/rte_mbuf.h +1
-1@@ -146,7 +146,7 @@ struct rte_mbuf {uint32_t reserved1;/**< Unused field. Required for padding */ /* remaining bytes are set on RX when pulling packet from descriptor */ -uint16_t reserved2;/**< Unused field. Required for padding */ +uint16_t packet_type;/**< Type of packet, e.g. protocols used */ uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ uint16_t l3_len:9; /**< L3 (IP) Header Length. */
This patch adds a new fields that nobody uses. So why should we add it ?
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 03/13] mbuf: add packet_type field
I would use it :)
It's useful to store the IP protocol number (UDP, TCP etc) and version of IP (4, 6) and then relay packet to specific handler.
08.09.2014 16:17, Olivier MATZ пишет:
Quoted 26 lines
Hi Bruce,
Quoted 14 lines · Bruce Richardson
Replace a reserved slot with the new packet type field used to identify the type of the packet, i.e. what protocols are used.
Signed-off-by Bruce Richardson <[email protected]>--- lib/librte_mbuf/rte_mbuf.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/librte_mbuf/rte_mbuf.h +0
-0lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 152
Quoted 8 lines
lib/librte_mbuf/rte_mbuf.h +1
-1@@ -146,7 +146,7 @@ struct rte_mbuf {uint32_t reserved1;/**< Unused field. Required for padding */ /* remaining bytes are set on RX when pulling packet from descriptor */ -uint16_t reserved2;/**< Unused field. Required for padding */ +uint16_t packet_type;/**< Type of packet, e.g. protocols used */ uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ uint16_t l3_len:9; /**< L3 (IP) Header Length. */This patch adds a new fields that nobody uses. So why should we add it ?
Signature · 5 lines
Sincerely, Yerden Zhumabekov State Technical Service Astana, KZ
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 03/13] mbuf: add packet_type field
Hi Yerden,
Quoted 15 lines · Yerden Zhumabekov
08.09.2014 16:17, Olivier MATZ пишет:
Quoted 5 lines
Quoted 5 lines
lib/librte_mbuf/rte_mbuf.h +0
-0@@ -146,7 +146,7 @@ struct rte_mbuf {uint32_t reserved1;/**< Unused field. Required for padding */ Quoted 8 lines
lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 152
Quoted 7 lines
lib/librte_mbuf/rte_mbuf.h +1
-1/* remaining bytes are set on RX when pulling packet from descriptor */ -uint16_t reserved2;/**< Unused field. Required for padding */ +uint16_t packet_type;/**< Type of packet, e.g. protocols used */ uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ uint16_t l3_len:9; /**< L3 (IP) Header Length. */This patch adds a new fields that nobody uses. So why should we add it ?
Quoted 3 lines
I would use it :)
It's useful to store the IP protocol number (UDP, TCP etc) and version of IP (4, 6) and then relay packet to specific handler.
I'm not saying this field is useless. But even if it's useful for some applications like yours, it does not mean that it should go in the generic mbuf structure.
Also, for a new field, we should define who is in charge of filling it. Is is the driver? Does it mean that all drivers have to be modified to fill it? Or is it just a placeholder for applications? In this case, shouldn't we use application-specific metadata? In the other direction (TX), we would also need to define if this field must be filled by the application before transmitting a mbuf to a driver.
Regards,
Olivier
A message that is not in the archive (sent off-list or not yet received): CAD16F236028A64DBBC0158B1636EA4510F3E4F8@SHSMSX104.ccr.corp.intel.com
A message that is not in the archive (sent off-list or not yet received): CAD16F236028A64DBBC0158B1636EA4510F3E4F8-0J0gbvR4kTg/[email protected]
Re: [PATCH 03/13] mbuf: add packet_type field
Hi Olivier,
Quoted 35 lines
-----Original Message-----
From: Zhu, Heqing
Sent: Tuesday, September 09, 2014 9:48 AM
To: Wu, Jingjing; Liu, Jijiang; Zhang, Helin
Cc: Zhu, Heqing
Subject: FW: [dpdk-dev] [PATCH 03/13] mbuf: addpacket_typefieldOne of you need respond to this thread? Please make the answer generic and easy to understand/accept if possible.
-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Olivier MATZ
Sent: Monday, September 08, 2014 7:17 PM
To: Yerden Zhumabekov; Richardson, Bruce; [email protected]
Subject: Re: [dpdk-dev] [PATCH 03/13] mbuf: addpacket_typefieldHi Yerden,
Quoted 7 lines · Yerden Zhumabekov
08.09.2014 16:17, Olivier MATZ пишет:
Quoted 6 lines
lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 148
Quoted 6 lines
lib/librte_mbuf/rte_mbuf.h +0
-0@@ -146,7 +146,7 @@ struct rte_mbuf {uint32_t reserved1;/**< Unused field. Required for padding */ /* remaining bytes are set on RX when pulling packet from descriptor */
Quoted 8 lines
Quoted 7 lines
lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 152
Quoted 6 lines
lib/librte_mbuf/rte_mbuf.h +1
-1-uint16_t reserved2;/**< Unused field. Required for padding */ +uint16_t packet_type;/**< Type of packet, e.g. protocols used */ uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ uint16_t l3_len:9; /**< L3 (IP) Header Length. */This patch adds a new fields that nobody uses. So why should we add it ?
Quoted 6 lines
Quoted 3 lines
I would use it :)
It's useful to store the IP protocol number (UDP, TCP etc) and version of IP (4, 6) and then relay packet to specific handler.I'm not saying this field is useless. But even if it's useful for some applications like yours, it does not mean that it should go in the generic mbuf structure.
In some types of NIC, for an example, Intel Fortville, which supports various packet types, and packet type value is filled in a field of receive Descriptor by HW, Normally, application analyses easily what incoming packet format is if it know what packet type is. It is a common approach for analyzing packet format.
Quoted 4 lines
Also, for a new field, we should define who is in charge of filling it. Is is the driver? Does it mean that all drivers have to be modified to fill it? Or is it just a placeholder for applications? In this case, shouldn't we use application-specific metadata?
Yes, driver is in charge of filling it if this type of NIC has a packet type filed in receive Descriptor.
Quoted 3 lines
In the other direction (TX), we would also need
to define if this field must be filled by the application before transmitting a mbuf to a driver.
Normally, TX side don't care the packet type, instead, feature offload flag will be used in TX side.
In RX side, NIC HW analyses incoming packet and know what packet type is, and fill packet type value in Receive Descriptor.
Of course, I also don't object to add a packet type in TX side if mbuf TX space is enough, but from the present point of view, it is not necessary.
Quoted 2 lines
Regards,
Olivier
Regards,
Jijiang Liu
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 03/13] mbuf: add packet_type field
Quoted 11 lines
-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Olivier MATZ
Sent: Monday, September 8, 2014 7:17 PM
To: Yerden Zhumabekov; Richardson, Bruce; [email protected]
Subject: Re: [dpdk-dev] [PATCH 03/13] mbuf: addpacket_typefieldHi Yerden,
On 09/08/2014 12:33 PM, Yerden Zhumabekov wrote:
Quoted 19 lines
Quoted 7 lines
08.09.2014 16:17, Olivier MATZ пишет:
Quoted 6 lines
lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 148
Quoted 6 lines
lib/librte_mbuf/rte_mbuf.h +0
-0@@ -146,7 +146,7 @@ struct rte_mbuf {uint32_t reserved1;/**< Unused field. Required for padding */ /* remaining bytes are set on RX when pulling packet from descriptor */
Quoted 11 lines
Quoted 7 lines
lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 152
Quoted 6 lines
lib/librte_mbuf/rte_mbuf.h +1
-1-uint16_t reserved2;/**< Unused field. Required for padding */ +uint16_t packet_type;/**< Type of packet, e.g. protocols used */ uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ uint16_t l3_len:9; /**< L3 (IP) Header Length. */This patch adds a new fields that nobody uses. So why should we add it ?
I would use it :)
It's useful to store the IP protocol number (UDP, TCP etc) and version of IP (4, 6) and then relay packet to specific handler.
It is a common field which i40e PMD will use it to store the 'packet type ID'. i40e hardware can recognize more than a hundred of packet types of received packets, this is quite useful for upper layer stack or application. So this field is quite useful and will be filled by PMD.
In ixgbe/igb, it has less than 10 packet types which are marked in offload flags. From now on, it would be better to have new field here to put the hardware offloaded packet type in and it could be used for future NICs.
Quoted 10 lines
I'm not saying this field is useless. But even if it's useful for some applications like yours, it does not mean that it should go in the generic mbuf structure.
Also, for a new field, we should define who is in charge of filling it. Is is the driver? Does it mean that all drivers have to be modified to fill it? Or is it just a placeholder for applications? In this case, shouldn't we use application-specific metadata? In the other direction (TX), we would also need to define if this field must be filled by the application before transmitting a mbuf to a driver.
Yes, PMD will fill it. I40e PMD will be the first one, ixgbe/igb can be kept as it is, or modified to be consistent. It is used for RX side only, and for TX side, it can be investigated to see if it can be used also. I think some new features in development can think of that.
Anyway, it is a quite useful field for i40e and future generation of NICs.
Quoted 3 lines
Regards,
Olivier
A message that is not in the archive (sent off-list or not yet received): F35DEAC7BCE34641BA9FAC6BCA4A12E70A784E49-0J0gbvR4kTg/[email protected]
Re: [PATCH 03/13] mbuf: add packet_type field
Hello,
Quoted 18 lines · Helin
It is a common field which i40e PMD will use it to store the 'packet type ID'. i40e hardware can recognize more than a hundred of packet types of received packets, this is quite useful for upper layer stack or application. So this field is quite useful and will be filled by PMD.
In ixgbe/igb, it has less than 10 packet types which are marked in offload flags. From now on, it would be better to have new field here to put the hardware offloaded packet type in and it could be used for future NICs.Quoted 10 lines
I'm not saying this field is useless. But even if it's useful for some applications like yours, it does not mean that it should go in the generic mbuf structure.
Also, for a new field, we should define who is in charge of filling it. Is is the driver? Does it mean that all drivers have to be modified to fill it? Or is it just a placeholder for applications? In this case, shouldn't we use application-specific metadata? In the other direction (TX), we would also need to define if this field must be filled by the application before transmitting a mbuf to a driver.
Quoted 5 lines
Yes, PMD will fill it. I40e PMD will be the first one, ixgbe/igb can be kept as it is, or modified to be consistent. It is used for RX side only, and for TX side, it can be investigated to see if it can be used also. I think some new features in development can think of that.
Anyway, it is a quite useful field for i40e and future generation of NICs.
To me, having the support in a hardware for that feature is not a sufficient reason for adding this field. There are many hardware features that will never be integrated in dpdk.
This first version of the patch:
- just adds a field that is not used by any code, so it is useless. At least testpmd or an application example should show how to use it.
- does not describe what enhancement is provided by adding the field (performance? in this case, numbers + use case would help to convince people).
- does not describe what can be the content of the field. Is it a protocol number?
- does not explain if all drivers must fill this field. If yes, the patch has to update all drivers. If not, something must be done to mark the packet field as unknown by default.
Regards,
Olivier
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 03/13] mbuf: add packet_type field
Quoted 37 lines
-----Original Message-----
From: Olivier MATZ [mailto:[email protected]]
Sent: Tuesday, September 9, 2014 4:03 PM
To: Zhang, Helin; Yerden Zhumabekov; Richardson, Bruce; [email protected]
Subject: Re: [dpdk-dev] [PATCH 03/13] mbuf: addpacket_typefieldHello,
Quoted 7 lines · Helin
It is a common field which i40e PMD will use it to store the 'packet type ID'. i40e hardware can recognize more than a hundred of packet types of received packets, this is quite useful for upper layer stack or application. So this field is quite useful and will be filled by PMD. In ixgbe/igb, it has less than 10 packet types which are marked in offload flags. From now on, it would be better to have new field here to put the hardware offloaded packet type in and it could be used for future
NICs.
Quoted 4 lines
Quoted 4 lines
I'm not saying this field is useless. But even if it's useful for some applications like yours, it does not mean that it should go in the
generic mbuf structure.
Quoted 7 lines
Quoted 7 lines
Also, for a new field, we should define who is in charge of filling it. Is is the driver? Does it mean that all drivers have to be modified to fill it? Or is it just a placeholder for applications? In this case, shouldn't we use application-specific metadata? In the other direction (TX), we would also need to define if this field must be filled by the application before transmitting a mbuf to a driver.
Quoted 5 lines
Yes, PMD will fill it. I40e PMD will be the first one, ixgbe/igb can be kept as it is, or modified to be consistent. It is used for RX side only, and for TX side, it can be investigated to see if it can be used also. I think some new features in development can think of that. Anyway, it is a quite useful field for i40e and future generation of NICs.
Quoted 3 lines
To me, having the support in a hardware for that feature is not a sufficient reason for adding this field. There are many hardware features that will never be integrated in dpdk.
At least this field is quite important for i40e.
e.g. packet type=43 means that hardware recognize it as a VXLAN packet. To avoid checking what type of packet by software, hardware can offload that, and fill the packet type ID in that field. It cannot be put in ol_flags anymore, as it has more than 100 packet type can be recognized by hardware. Without it, vxlan feature cannot be implemented at all.
Quoted 6 lines
This first version of the patch:
- just adds a field that is not used by any code, so it is useless. At least testpmd or an application example should show how to use it.
It will be used at least in vxlan feature which is in development. Without it, vxlan cannot be completed. So this is a very important field i40e and future NICs.
Quoted 4 lines
- does not describe what enhancement is provided by adding the field (performance? in this case, numbers + use case would help to convince people).
I40e hardware can recognize received packets as different packet types, and there are about 256 packet types can be recognized by i40e hardware. It is not a enhancement, it is the key for at least i40e features.
Quoted 4 lines
- does not describe what can be the content of the field. Is it a protocol number?
The packet type is a offload feature of hardware, the value of it can mean one type of packet recognized by the i40e hardware. E.g.
| Packet type | Description | | 0 | Reserved | | 1 | MAC, PAY2 | | 2 | MAC, TimeSync, PAY2 |
...
| 43 | MAC, IPV4, GRENAT, PAY3 |
Quoted 4 lines
- does not explain if all drivers must fill this field. If yes, the patch has to update all drivers. If not, something must be done to mark the packet field as unknown by default.
I40e needs it.
igb/ixgbe can be changed to support it, but not mandatory, as ol_flags can represent it.
Actually ixgbe and igb has packet type also, but the number of those types is less than 10, so it can be put in ol_flags. For i40e and future NICs, the number of that could be 256 or more, ol_flags does not have that many bits for it. The best idea is to fill the packet type ID directly into a field.
Quoted 2 lines
Regards,
Olivier
Regards,
Helin
Re: [PATCH 03/13] mbuf: add packet_type field
In reply to a message not in the archiveQuoted 62 lines
-----Original Message-----
From: Olivier MATZ [mailto:[email protected]]
Sent: Tuesday, September 09, 2014 9:03 AM
To: Zhang, Helin; Yerden Zhumabekov; Richardson, Bruce; [email protected]
Subject: Re: [dpdk-dev] [PATCH 03/13] mbuf: addpacket_typefieldHello,
Quoted 1 line · Helin
It is a common field which i40e PMD will use it to store the 'packet type ID'.
i40e
Quoted 1 line
hardware can recognize more than a hundred of packet types of received
packets,
Quoted 1 line
this is quite useful for upper layer stack or application. So this field is quite
useful
Quoted 2 lines
and will be filled by PMD.
In ixgbe/igb, it has less than 10 packet types which are marked in offload flags.From
Quoted 1 line
now on, it would be better to have new field here to put the hardware
offloaded
Quoted 8 lines
packet type in and it could be used for future NICs.
Quoted 6 lines
I'm not saying this field is useless. But even if it's useful for some applications like yours, it does not mean that it should go in the generic mbuf structure.
Also, for a new field, we should define who is in charge of filling it. Is is the driver? Does it mean that all drivers have to be modified to fill it? Or
is
Quoted 2 lines
Quoted 2 lines
it just a placeholder for applications? In this case, shouldn't we use application-specific metadata? In the other direction (TX), we would also
need
Quoted 1 line
Quoted 1 line
to define if this field must be filled by the application before transmitting a
mbuf
Quoted 2 lines
Quoted 1 line
to a driver.
Yes, PMD will fill it. I40e PMD will be the first one, ixgbe/igb can be kept as it
is, or
Quoted 2 lines
modified to be consistent. It is used for RX side only, and for TX side, it can be investigated to see if it can be used also. I think some new features in
development
Quoted 2 lines
can think of that.
Anyway, it is a quite useful field for i40e and future generation of NICs.To me, having the support in a hardware for that feature is not a sufficient reason for adding this field. There are many hardware features that will never be integrated in dpdk.
This first version of the patch:
- just adds a field that is not used by any code, so it is useless. At least testpmd or an application example should show how to use it.
- does not describe what enhancement is provided by adding the field (performance? in this case, numbers + use case would help to convince people).
- does not describe what can be the content of the field. Is it a protocol number?
Quoted 6 lines
- does not explain if all drivers must fill this field. If yes, the patch has to update all drivers. If not, something must be done to mark the packet field as unknown by default.
Regards,
Olivier
Hi,
Points taken. Really, this patch doesn't belong in this set as I had planned things and better belongs in patch set 3 (coming soon, I hope) which should propose the new field additions. I simply put it here to avoid having to start renumbering and renaming reserved fields in the structure, but that is possibly the lesser of the two evils.
However, with regards to adding new fields in, I would like to have some agreement that I can add fields in without actually pushing in the patch to use them - so long as sufficient rational is provided for using the field and there is a soon pending change to actually use it. This patch did not meet the criteria for explanation, but if updated to do so, I would like to have this patch accepted on the basis of that explanation so as to enable those working on the drivers to make us of it.
Regards,
/Bruce
Re: [PATCH 03/13] mbuf: add packet_type field
In reply to a message not in the archiveQuoted 29 lines · Olivier MATZ
Hi Yerden,
Quoted 15 lines · Yerden Zhumabekov
08.09.2014 16:17, Olivier MATZ пишет:
Quoted 5 lines
Quoted 5 lines
lib/librte_mbuf/rte_mbuf.h +0
-0@@ -146,7 +146,7 @@ struct rte_mbuf {uint32_t reserved1;/**< Unused field. Required for padding */ Quoted 8 lines
lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 152
Quoted 7 lines
lib/librte_mbuf/rte_mbuf.h +1
-1/* remaining bytes are set on RX when pulling packet from descriptor */ -uint16_t reserved2;/**< Unused field. Required for padding */ +uint16_t packet_type;/**< Type of packet, e.g. protocols used */ uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ uint16_t l3_len:9; /**< L3 (IP) Header Length. */This patch adds a new fields that nobody uses. So why should we add it ?
Quoted 3 lines
I would use it :)
It's useful to store the IP protocol number (UDP, TCP etc) and version of IP (4, 6) and then relay packet to specific handler.I'm not saying this field is useless. But even if it's useful for some applications like yours, it does not mean that it should go in the generic mbuf structure.
Quoted 6 lines
Also, for a new field, we should define who is in charge of filling it. Is is the driver? Does it mean that all drivers have to be modified to fill it? Or is it just a placeholder for applications? In this case, shouldn't we use application-specific metadata? In the other direction (TX), we would also need to define if this field must be filled by the application before transmitting a mbuf to a driver.
Funny, but these new fields (and extended mbuf) were prominent during the dpdk summit.
I think it’s going to be quite useful.
PATCH 4/13 mbuf: expand ol_flags field to 64-bits
In reply to a message not in the archiveThe offload flags field (ol_flags) was 16-bits and had no further room for expansion. This patch increases the field size to 64-bits, using up the remaining reserved space in the single-cache-line mbuf.
NOTE: none of the values for existing flags have been changed, i.e. no new numbers have been explicitly reserved between existing flag definitions.
--- app/test-pmd/config.c | 8 ++--- app/test-pmd/csumonly.c | 8 ++--- app/test-pmd/rxonly.c | 2 +- app/test-pmd/testpmd.h | 4 +-- app/test-pmd/txonly.c | 2 +- lib/librte_mbuf/rte_mbuf.c | 2 +- lib/librte_mbuf/rte_mbuf.h | 4 +-- lib/librte_pmd_e1000/em_rxtx.c | 34 ++++++++++----------- lib/librte_pmd_e1000/igb_rxtx.c | 62 ++++++++++++++++++--------------------- lib/librte_pmd_i40e/i40e_rxtx.c | 31 ++++++++++---------- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 61 ++++++++++++++++++-------------------- lib/librte_pmd_ixgbe/ixgbe_rxtx.h | 2 +- 12 files changed, 104 insertions(+), 116 deletions(-)
app/test-pmd/config.c +4 -4
@@ -1714,14 +1714,14 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) }void -tx_cksum_set (portid_t port_id,uint8_t cksum_mask) +tx_cksum_set (portid_t port_id,uint64_t ol_flags) { -uint16_t tx_ol_flags; +uint64_t tx_ol_flags;if (port_id_is_invalid (port_id))return ;/* Clear last 4 bits and then set L3/4 checksum mask again */ - tx_ol_flags = (uint16_t ) (ports[port_id].tx_ol_flags &0xFFF0 ); - ports[port_id].tx_ol_flags = (uint16_t ) ((cksum_mask &0xf ) | tx_ol_flags); + tx_ol_flags = ports[port_id].tx_ol_flags & (~0x0Full ); + ports[port_id].tx_ol_flags = ((ol_flags &0xf ) | tx_ol_flags); }void
app/test-pmd/csumonly.c +4 -4
@@ -217,9 +217,9 @@ pkt_burst_checksum_forward(struct fwd_stream *fs)uint16_t nb_rx;uint16_t nb_tx;uint16_t i; -uint16_t ol_flags; -uint16_t pkt_ol_flags; -uint16_t tx_ol_flags; +uint64_t ol_flags; +uint64_t pkt_ol_flags; +uint64_t tx_ol_flags;uint16_t l4_proto;uint16_t eth_type;uint8_t l2_len;
@@ -261,7 +261,7 @@ pkt_burst_checksum_forward(struct fwd_stream *fs) mb = pkts_burst[i]; l2_len =sizeof (struct ether_hdr ); pkt_ol_flags = mb->ol_flags; - ol_flags = (uint16_t ) (pkt_ol_flags & (~PKT_TX_L4_MASK )); + ol_flags = (pkt_ol_flags & (~PKT_TX_L4_MASK )); eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *); eth_type =rte_be_to_cpu_16 (eth_hdr->ether_type);
app/test-pmd/rxonly.c +1 -1
@@ -109,7 +109,7 @@ pkt_burst_receive(struct fwd_stream *fs)struct rte_mbuf *mb;struct ether_hdr *eth_hdr;uint16_t eth_type; -uint16_t ol_flags; +uint64_t ol_flags;uint16_t nb_rx;uint16_t i;#ifdef RTE_TEST_PMD_RECORD_CORE_CYCLES
app/test-pmd/testpmd.h +2 -2
@@ -141,7 +141,7 @@ struct rte_port {struct fwd_stream *rx_stream;/**< Port RX stream, if unique */ struct fwd_stream *tx_stream;/**< Port TX stream, if unique */ unsigned int socket_id;/**< For NUMA support */ -uint16_t tx_ol_flags;/**< Offload Flags of TX packets. */ +uint64_t tx_ol_flags;/**< Offload Flags of TX packets. */ uint16_t tx_vlan_id;/**< Tag Id. in TX VLAN packets. */ void *fwd_ctx;/**< Forwarding mode context */ uint64_t rx_bad_ip_csum;/**< rx pkts with bad ip checksum */
@@ -494,7 +494,7 @@ void tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on);void set_qmap (portid_t port_id,uint8_t is_rx,uint16_t queue_id,uint8_t map_value); -void tx_cksum_set (portid_t port_id,uint8_t cksum_mask); +void tx_cksum_set (portid_t port_id,uint64_t ol_flags);void set_verbose_level (uint16_t vb_level);void set_tx_pkt_segments (unsigned *seg_lengths,unsigned nb_segs);
app/test-pmd/txonly.c +1 -1
@@ -203,7 +203,7 @@ pkt_burst_transmit(struct fwd_stream *fs)uint16_t nb_tx;uint16_t nb_pkt;uint16_t vlan_tci; -uint16_t ol_flags; +uint64_t ol_flags;uint8_t i;#ifdef RTE_TEST_PMD_RECORD_CORE_CYCLES uint64_t start_tsc;
lib/librte_mbuf/rte_mbuf.c +1 -1
@@ -174,7 +174,7 @@ rte_pktmbuf_dump(FILE *f, const struct rte_mbuf *m, unsigned dump_len)fprintf (f,"dump mbuf at 0x%p, phys=%" PRIx64", buf_len=%u\n" , m, (uint64_t )m->buf_physaddr, (unsigned )m->buf_len); -fprintf (f," pkt_len=%" PRIu32", ol_flags=%" PRIx16", nb_segs=%u, " +fprintf (f," pkt_len=%" PRIu32", ol_flags=%" PRIx64", nb_segs=%u, " "in_port=%u\n" , m->pkt_len, m->ol_flags, (unsigned )m->nb_segs, (unsigned )m->port); nb_segs = m->nb_segs;
lib/librte_mbuf/rte_mbuf.h +1 -3
@@ -141,9 +141,7 @@ struct rte_mbuf {uint8_t nb_segs;/**< Number of segments. */ uint8_t port;/**< Input port. */ -uint16_t ol_flags;/**< Offload features. */ -uint16_t reserved0;/**< Unused field. Required for padding */ -uint32_t reserved1;/**< Unused field. Required for padding */ +uint64_t ol_flags;/**< Offload features. */ /* remaining bytes are set on RX when pulling packet from descriptor */ uint16_t packet_type;/**< Type of packet, e.g. protocols used */
lib/librte_pmd_e1000/em_rxtx.c +16 -18
@@ -168,7 +168,7 @@ union em_vlan_macip {* Structure to check if new context need be built */ struct em_ctx_info { -uint16_t flags;/**< ol_flags related to context build. */ +uint64_t flags;/**< ol_flags related to context build. */ uint32_t cmp_mask;/**< compare mask */ union em_vlan_macip hdrlen;/**< L2 and L3 header lenghts */ };
@@ -238,7 +238,7 @@ struct em_tx_queue {static inline void em_set_xmit_ctx (struct em_tx_queue * txq,volatile struct e1000_context_desc *ctx_txd, -uint16_t flags, +uint64_t flags,union em_vlan_macip hdrlen) {uint32_t cmp_mask, cmd_len;
@@ -304,7 +304,7 @@ em_set_xmit_ctx(struct em_tx_queue* txq,* or create a new context descriptor. */ static inline uint32_t -what_ctx_update (struct em_tx_queue *txq,uint16_t flags, +what_ctx_update (struct em_tx_queue *txq,uint64_t flags,union em_vlan_macip hdrlen) {/* If match with the current context */
@@ -377,7 +377,7 @@ em_xmit_cleanup(struct em_tx_queue *txq) }static inline uint32_t -tx_desc_cksum_flags_to_upper (uint16_t ol_flags) +tx_desc_cksum_flags_to_upper (uint64_t ol_flags) {static const uint32_t l4_olinfo[2 ] = {0 ,E1000_TXD_POPTS_TXSM <<8 };static const uint32_t l3_olinfo[2 ] = {0 ,E1000_TXD_POPTS_IXSM <<8 };
@@ -403,12 +403,12 @@ eth_em_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,uint32_t popts_spec;uint32_t cmd_type_len;uint16_t slen; -uint16_t ol_flags; +uint64_t ol_flags;uint16_t tx_id;uint16_t tx_last;uint16_t nb_tx;uint16_t nb_used; -uint16_t tx_ol_req; +uint64_t tx_ol_req;uint32_t ctx;uint32_t new_ctx;union em_vlan_macip hdrlen;
@@ -438,8 +438,7 @@ eth_em_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, ol_flags = tx_pkt->ol_flags;/* If hardware offload required */ - tx_ol_req = (uint16_t )(ol_flags & (PKT_TX_IP_CKSUM | -PKT_TX_L4_MASK )); + tx_ol_req = (ol_flags & (PKT_TX_IP_CKSUM |PKT_TX_L4_MASK ));if (tx_ol_req) { hdrlen.f.vlan_tci = tx_pkt->vlan_tci; hdrlen.f.l2_len = tx_pkt->l2_len;
@@ -642,22 +641,21 @@ end_of_tx:* **********************************************************************/ -static inline uint16_t +static inline uint64_t rx_desc_status_to_pkt_flags (uint32_t rx_status) { -uint16_t pkt_flags; +uint64_t pkt_flags;/* Check if VLAN present */ - pkt_flags = (uint16_t )((rx_status &E1000_RXD_STAT_VP ) ? -PKT_RX_VLAN_PKT :0 ); + pkt_flags = ((rx_status &E1000_RXD_STAT_VP ) ?PKT_RX_VLAN_PKT :0 );return pkt_flags; } -static inline uint16_t +static inline uint64_t rx_desc_error_to_pkt_flags (uint32_t rx_error) { -uint16_t pkt_flags =0 ; +uint64_t pkt_flags =0 ;if (rx_error &E1000_RXD_ERR_IPE ) pkt_flags |=PKT_RX_IP_CKSUM_BAD ;
@@ -801,8 +799,8 @@ eth_em_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rxm->port = rxq->port_id; rxm->ol_flags =rx_desc_status_to_pkt_flags (status); - rxm->ol_flags = (uint16_t )(rxm->ol_flags | -rx_desc_error_to_pkt_flags (rxd.errors)); + rxm->ol_flags = rxm->ol_flags | +rx_desc_error_to_pkt_flags (rxd.errors);/* Only valid if PKT_RX_VLAN_PKT set in pkt_flags */ rxm->vlan_tci =rte_le_to_cpu_16 (rxd.special);
@@ -1027,8 +1025,8 @@ eth_em_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, first_seg->port = rxq->port_id; first_seg->ol_flags =rx_desc_status_to_pkt_flags (status); - first_seg->ol_flags = (uint16_t )(first_seg->ol_flags | -rx_desc_error_to_pkt_flags (rxd.errors)); + first_seg->ol_flags = first_seg->ol_flags | +rx_desc_error_to_pkt_flags (rxd.errors);/* Only valid if PKT_RX_VLAN_PKT set in pkt_flags */ rxm->vlan_tci =rte_le_to_cpu_16 (rxd.special);
lib/librte_pmd_e1000/igb_rxtx.c +28 -34
169 lines not on this page · raw message
lib/librte_pmd_i40e/i40e_rxtx.c +16 -15
@@ -91,27 +91,27 @@ static uint16_t i40e_xmit_pkts_simple(void *tx_queue, uint16_t nb_pkts);/* Translate the rx descriptor status to pkt flags */ -static inline uint16_t +static inline uint64_t i40e_rxd_status_to_pkt_flags (uint64_t qword) { -uint16_t flags; +uint64_t flags;/* Check if VLAN packet */ - flags = (uint16_t )(qword & (1 <<I40E_RX_DESC_STATUS_L2TAG1P_SHIFT ) ? -PKT_RX_VLAN_PKT :0 ); + flags = qword & (1 <<I40E_RX_DESC_STATUS_L2TAG1P_SHIFT ) ? +PKT_RX_VLAN_PKT :0 ;/* Check if RSS_HASH */ - flags |= (uint16_t )((((qword >>I40E_RX_DESC_STATUS_FLTSTAT_SHIFT ) & + flags |= (((qword >>I40E_RX_DESC_STATUS_FLTSTAT_SHIFT ) &I40E_RX_DESC_FLTSTAT_RSS_HASH ) == -I40E_RX_DESC_FLTSTAT_RSS_HASH ) ?PKT_RX_RSS_HASH :0 ); +I40E_RX_DESC_FLTSTAT_RSS_HASH ) ?PKT_RX_RSS_HASH :0 ;return flags; } -static inline uint16_t +static inline uint64_t i40e_rxd_error_to_pkt_flags (uint64_t qword) { -uint16_t flags =0 ; +uint64_t flags =0 ;uint64_t error_bits = (qword >>I40E_RXD_QW1_ERROR_SHIFT );#define I40E_RX_ERR_BITS 0x3f
@@ -143,12 +143,12 @@ i40e_rxd_error_to_pkt_flags(uint64_t qword) }/* Translate pkt types to pkt flags */ -static inline uint16_t +static inline uint64_t i40e_rxd_ptype_to_pkt_flags (uint64_t qword) {uint8_t ptype = (uint8_t )((qword &I40E_RXD_QW1_PTYPE_MASK ) >>I40E_RXD_QW1_PTYPE_SHIFT ); -static const uint16_t ip_ptype_map[I40E_MAX_PKT_TYPE ] = { +static const uint64_t ip_ptype_map[I40E_MAX_PKT_TYPE ] = {0 ,/* PTYPE 0 */ 0 ,/* PTYPE 1 */ 0 ,/* PTYPE 2 */
@@ -567,7 +567,7 @@ i40e_rx_scan_hw_ring(struct i40e_rx_queue *rxq)uint32_t rx_status;int32_t s[I40E_LOOK_AHEAD ], nb_dd;int32_t i, j, nb_rx =0 ; -uint16_t pkt_flags; +uint64_t pkt_flags; rxdp = &rxq->rx_ring[rxq->rx_tail]; rxep = &rxq->sw_ring[rxq->rx_tail];
@@ -789,7 +789,7 @@ i40e_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)uint16_t rx_packet_len;uint16_t rx_id, nb_hold;uint64_t dma_addr; -uint16_t pkt_flags; +uint64_t pkt_flags; nb_rx =0 ; nb_hold =0 ;
@@ -896,10 +896,11 @@ i40e_recv_scattered_pkts(void *rx_queue,struct rte_mbuf *last_seg = rxq->pkt_last_seg;struct rte_mbuf *nmb, *rxm;uint16_t rx_id = rxq->rx_tail; -uint16_t nb_rx =0 , nb_hold =0 , rx_packet_len, pkt_flags; +uint16_t nb_rx =0 , nb_hold =0 , rx_packet_len;uint32_t rx_status;uint64_t qword1;uint64_t dma_addr; +uint64_t pkt_flags;while (nb_rx < nb_pkts) { rxdp = &rx_ring[rx_id];
@@ -1046,7 +1047,7 @@ i40e_recv_scattered_pkts(void *rx_queue,/* Check if the context descriptor is needed for TX offloading */ static inline uint16_t -i40e_calc_context_desc (uint16_t flags) +i40e_calc_context_desc (uint64_t flags) {uint16_t mask =0 ;
@@ -1075,7 +1076,7 @@ i40e_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)uint32_t td_offset;uint32_t tx_flags;uint32_t td_tag; -uint16_t ol_flags; +uint64_t ol_flags;uint8_t l2_len;uint8_t l3_len;uint16_t nb_used;
lib/librte_pmd_ixgbe/ixgbe_rxtx.c +29 -32
167 lines not on this page · raw message
lib/librte_pmd_ixgbe/ixgbe_rxtx.h +1 -1
@@ -178,7 +178,7 @@ union ixgbe_vlan_macip {*/ struct ixgbe_advctx_info { -uint16_t flags;/**< ol_flags for context build. */ +uint64_t flags;/**< ol_flags for context build. */ uint32_t cmp_mask;/**< compare mask for vlan_macip_lens */ union ixgbe_vlan_macip vlan_macip_lens;/**< vlan, mac ip length. */ };
Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): 1409759378-10113-5-git-send-email-bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH 04/13] mbuf: expand ol_flags field to 64-bits
Hi Bruce,
Quoted 9 lines · Bruce Richardson
The offload flags field (
ol_flags) was 16-bits and had no further room for expansion. This patch increases the field size to 64-bits, using up the remaining reserved space in the single-cache-line mbuf.NOTE: none of the values for existing flags have been changed, i.e. no new numbers have been explicitly reserved between existing flag definitions.
Signed-off-by Bruce Richardson <[email protected]>
The initial series I've proposed [1][2] had on more enhancement: the first patch [1] allowed to remove the definition of flag names in testpmd. Indeed, this is not really good because they must be kept synchronized with the flags in rte_mbuf. What do you think about this patch? Should it be integrated in your series? Or later? Or never? ;)
The second patch [2] changes the value of the flags. This is not needed now, but if we do it in the future, we should not forget to change app/test-pmd/cmdline.c accordingly. Maybe this could go in your patch directly as it does not hurt?
Olivier
[1] http://dpdk.org/ml/archives/dev/2014-May/002545.html
[2] http://dpdk.org/ml/archives/dev/2014-May/002546.html
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 04/13] mbuf: expand ol_flags field to 64-bits
Quoted 17 lines
-----Original Message-----
From: Olivier MATZ [mailto:[email protected]]
Sent: Monday, September 08, 2014 11:26 AM
To: Richardson, Bruce; [email protected]
Subject: Re: [dpdk-dev] [PATCH 04/13] mbuf: expandol_flagsfield to 64-bitsHi Bruce,
Quoted 8 lines · Bruce Richardson
The offload flags field (
ol_flags) was 16-bits and had no further room for expansion. This patch increases the field size to 64-bits, using up the remaining reserved space in the single-cache-line mbuf.NOTE: none of the values for existing flags have been changed, i.e. no new numbers have been explicitly reserved between existing flag definitions.
Quoted 7 lines
Quoted 1 line
Signed-off-by Bruce Richardson <[email protected]>The initial series I've proposed [1][2] had on more enhancement: the first patch [1] allowed to remove the definition of flag names in testpmd. Indeed, this is not really good because they must be kept synchronized with the flags in
rte_mbuf. What do you think about this patch? Should it be integrated in your series? Or later? Or never? ;)
No, it is a good change - I've just keep it out of my series for simplicity as I'm largely trying to keep the scope as small as possible. I would love to see that go in as a separate patch maybe once the mbuf rework is finished.
Quoted 5 lines
The second patch [2] changes the value of the flags. This is not needed now, but if we do it in the future, we should not forget to change
app/test-pmd/cmdline.caccordingly. Maybe this could go in your patch directly as it does not hurt?
As above for now. Right now I'm just trying to get the structure worked out, and deal with any performance regressions that are found (such as what Pablo found last Friday :-( ).
/Bruce
Quoted 6 lines
Olivier
[1] http://dpdk.org/ml/archives/dev/2014-May/002545.html
[2] http://dpdk.org/ml/archives/dev/2014-May/002546.html
PATCH 5/13 mbuf: introduce a flag to indicate a control mbuf
In reply to a message not in the archiveSince the flags field is now 64-bits, we can allow one bit to be used to indicate a control i.e. non-packet mbuf. Dedicate the high bit (bit 63) for this purpose and add in a utility macro to test if a given mbuf has the bit set or not.
--- lib/librte_mbuf/rte_mbuf.c | 2 ++ lib/librte_mbuf/rte_mbuf.h | 19 +++++++++++++++++++ 2 files changed, 21 insertions(+)
lib/librte_mbuf/rte_mbuf.c +2 -0
@@ -69,7 +69,9 @@ rte_ctrlmbuf_init(struct rte_mempool *mp, void *_m, __attribute__((unused)) unsigned i) { +struct rte_mbuf *m = _m;rte_pktmbuf_init (mp, opaque_arg, _m, i); + m->ol_flags |=CTRL_MBUF_FLAG ; }/*
lib/librte_mbuf/rte_mbuf.h +19 -0
@@ -91,6 +91,7 @@ extern "C" {#define PKT_TX_IPV4_CSUM 0x1000/**< Alias of PKT_TX_IP_CKSUM. */ #define PKT_TX_IPV4 PKT_RX_IPV4_HDR/**< IPv4 with no IP checksum offload. */ #define PKT_TX_IPV6 PKT_RX_IPV6_HDR/**< IPv6 packet */ +/* * Bit 14~13 used for L4 packet type with checksum enabled. * 00: Reserved
@@ -106,6 +107,9 @@ extern "C" {/* Bit 15 */ #define PKT_TX_IEEE1588_TMST 0x8000/**< TX IEEE1588 packet to timestamp. */ +/* Use final bit of flags to indicate a control mbuf */ +#define CTRL_MBUF_FLAG (1ULL << 63) +/** * Bit Mask to indicate what bits required for building TX context */
@@ -466,6 +470,21 @@ void rte_ctrlmbuf_init(struct rte_mempool *mp, void *opaque_arg,*/ #define rte_ctrlmbuf_len (m) rte_pktmbuf_data_len(m) +/** +* Tests if an mbuf is a control mbuf +* +* @param m +* The mbuf.to be tested +* @return +* - True (1) if the mbuf is a control mbuf +* - False(0) otherwise +*/ +static inline int +rte_is_ctrlmbuf (struct rte_mbuf *m) +{ +return (!!(m->ol_flags &CTRL_MBUF_FLAG )); +} +/* Operations on pkt mbuf */ /**
Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): 1409759378-10113-6-git-send-email-bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Olivier MATZ
Acked-byRe: [PATCH 05/13] mbuf: introduce a flag to indicate a control mbuf
Hi Bruce,
Quoted 9 lines · Bruce Richardson
Since the flags field is now 64-bits, we can allow one bit to be used to indicate a control i.e. non-packet mbuf. Dedicate the high bit (bit 63) for this purpose and add in a utility macro to test if a given mbuf has the bit set or not.
Signed-off-by Bruce Richardson <[email protected]>[...]
Quoted 9 lines
+/**
+ * Tests if an mbuf is a control mbuf
+ *
+ * @param m
+ * The mbuf.to be tested
+ * @return
+ * - True (1) if the mbuf is a control mbuf
+ * - False(0) otherwise
+ */
Typo s/mbuf.to/mbuf to/
PATCH 6/13 mbuf: minor changes for readability
In reply to a message not in the archive* Ensure comments line up correctly
* Simplify the #ifdefs around the refcnt fields to make them clearer
--- lib/librte_mbuf/rte_mbuf.h | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-)
lib/librte_mbuf/rte_mbuf.h +8 -9
@@ -126,7 +126,6 @@ struct rte_mbuf {uint16_t buf_len;/**< Length of segment buffer. */ uint16_t data_off; -#ifdef RTE_MBUF_REFCNT /** * 16-bit Reference counter. * It should only be accessed using the following functions:
@@ -136,12 +135,12 @@ struct rte_mbuf {* config option. */ union { -rte_atomic16_t refcnt_atomic;/**< Atomically accessed refcnt */ -uint16_t refcnt;/**< Non-atomically accessed refcnt */ - }; -#else -uint16_t refcnt_reserved;/**< Do not use this field */ +#ifdef RTE_MBUF_REFCNT +rte_atomic16_t refcnt_atomic;/**< Atomically accessed refcnt */ +uint16_t refcnt;/**< Non-atomically accessed refcnt */ #endif +uint16_t refcnt_reserved;/**< Do not use this field */ + };uint8_t nb_segs;/**< Number of segments. */ uint8_t port;/**< Input port. */
@@ -155,12 +154,12 @@ struct rte_mbuf { uint16_t l2_len:7; /**< L2 (MAC) Header Length. */uint16_t vlan_tci;/**< VLAN Tag Control Identifier (CPU order). */ union { -uint32_t rss;/**< RSS hash result if RSS enabled */ +uint32_t rss;/**< RSS hash result if RSS enabled */ struct {uint16_t hash;uint16_t id; - } fdir;/**< Filter identifier if FDIR enabled */ -uint32_t sched;/**< Hierarchical scheduler */ + } fdir;/**< Filter identifier if FDIR enabled */ +uint32_t sched;/**< Hierarchical scheduler */ } hash;/**< hash information */ /* fields only used in slow path or on TX */
Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): 1409759378-10113-7-git-send-email-bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Olivier MATZ
Acked-byRe: [PATCH 06/13] mbuf: minor changes for readability
Quoted 4 lines · Bruce Richardson
* Ensure comments line up correctly
* Simplify the #ifdefs around the refcnt fields to make them clearerSigned-off-by Bruce Richardson <[email protected]>
PATCH 7/13 mbuf: use macros only to access the mbuf metadata
In reply to a message not in the archiveRemoved the explicit zero-sized metadata definition at the end of the mbuf data structure. Updated the metadata macros to take account of this change so that all existing code which uses those macros still works.
--- lib/librte_mbuf/rte_mbuf.h | 22 ++++++++-------------- 1 file changed, 8 insertions(+), 14 deletions(-)
lib/librte_mbuf/rte_mbuf.h +8 -14
@@ -166,31 +166,25 @@ struct rte_mbuf {struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */ struct rte_mbuf *next;/**< Next segment of scattered packet. */ -union { -uint8_t metadata[0 ]; -uint16_t metadata16[0 ]; -uint32_t metadata32[0 ]; -uint64_t metadata64[0 ]; - } __rte_cache_aligned; } __rte_cache_aligned;#define RTE_MBUF_METADATA_UINT8 (mbuf, offset) \ - (mbuf->metadata[offset]) + (((uint8_t *)&(mbuf)[1])[offset])#define RTE_MBUF_METADATA_UINT16 (mbuf, offset) \ - (mbuf->metadata16[offset/sizeof(uint16_t)]) + (((uint16_t *)&(mbuf)[1])[offset/sizeof(uint16_t)])#define RTE_MBUF_METADATA_UINT32 (mbuf, offset) \ - (mbuf->metadata32[offset/sizeof(uint32_t)]) + (((uint32_t *)&(mbuf)[1])[offset/sizeof(uint32_t)])#define RTE_MBUF_METADATA_UINT64 (mbuf, offset) \ - (mbuf->metadata64[offset/sizeof(uint64_t)]) + (((uint64_t *)&(mbuf)[1])[offset/sizeof(uint64_t)])#define RTE_MBUF_METADATA_UINT8_PTR (mbuf, offset) \ - (&mbuf->metadata[offset]) + (&RTE_MBUF_METADATA_UINT8(mbuf, offset))#define RTE_MBUF_METADATA_UINT16_PTR (mbuf, offset) \ - (&mbuf->metadata16[offset/sizeof(uint16_t)]) + (&RTE_MBUF_METADATA_UINT16(mbuf, offset))#define RTE_MBUF_METADATA_UINT32_PTR (mbuf, offset) \ - (&mbuf->metadata32[offset/sizeof(uint32_t)]) + (&RTE_MBUF_METADATA_UINT32(mbuf, offset))#define RTE_MBUF_METADATA_UINT64_PTR (mbuf, offset) \ - (&mbuf->metadata64[offset/sizeof(uint64_t)]) + (&RTE_MBUF_METADATA_UINT64(mbuf, offset))/** * Given the buf_addr returns the pointer to corresponding mbuf.
Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): 1409759378-10113-8-git-send-email-bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Olivier MATZ
Acked-byRe: [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
Hi Bruce,
Quoted 44 lines · Bruce Richardson
Removed the explicit zero-sized metadata definition at the end of the mbuf data structure. Updated the metadata macros to take account of this change so that all existing code which uses those macros still works.
Signed-off-by Bruce Richardson <[email protected]>--- lib/librte_mbuf/rte_mbuf.h | 22 ++++++++-------------- 1 file changed, 8 insertions(+), 14 deletions(-)lib/librte_mbuf/rte_mbuf.h +5
-12@@ -166,31 +166,25 @@ struct rte_mbuf {struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */ struct rte_mbuf *next;/**< Next segment of scattered packet. */ -union { -uint8_t metadata[0 ]; -uint16_t metadata16[0 ]; -uint32_t metadata32[0 ]; -uint64_t metadata64[0 ]; - } __rte_cache_aligned; } __rte_cache_aligned;#define RTE_MBUF_METADATA_UINT8 (mbuf, offset) \ - (mbuf->metadata[offset]) + (((uint8_t *)&(mbuf)[1])[offset])#define RTE_MBUF_METADATA_UINT16 (mbuf, offset) \ - (mbuf->metadata16[offset/sizeof(uint16_t)]) + (((uint16_t *)&(mbuf)[1])[offset/sizeof(uint16_t)])#define RTE_MBUF_METADATA_UINT32 (mbuf, offset) \ - (mbuf->metadata32[offset/sizeof(uint32_t)]) + (((uint32_t *)&(mbuf)[1])[offset/sizeof(uint32_t)])#define RTE_MBUF_METADATA_UINT64 (mbuf, offset) \ - (mbuf->metadata64[offset/sizeof(uint64_t)]) + (((uint64_t *)&(mbuf)[1])[offset/sizeof(uint64_t)])#define RTE_MBUF_METADATA_UINT8_PTR (mbuf, offset) \ - (&mbuf->metadata[offset]) + (&RTE_MBUF_METADATA_UINT8(mbuf, offset))#define RTE_MBUF_METADATA_UINT16_PTR (mbuf, offset) \ - (&mbuf->metadata16[offset/sizeof(uint16_t)])
lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 190
Quoted 10 lines
lib/librte_mbuf/rte_mbuf.h +3
-2+ (&RTE_MBUF_METADATA_UINT16(mbuf, offset))#define RTE_MBUF_METADATA_UINT32_PTR (mbuf, offset) \ - (&mbuf->metadata32[offset/sizeof(uint32_t)]) + (&RTE_MBUF_METADATA_UINT32(mbuf, offset))#define RTE_MBUF_METADATA_UINT64_PTR (mbuf, offset) \ - (&mbuf->metadata64[offset/sizeof(uint64_t)]) + (&RTE_MBUF_METADATA_UINT64(mbuf, offset))/** * Given the buf_addr returns the pointer to corresponding mbuf.
I think it goes in the good direction. So:
Just one question: why not removing RTE_MBUF_METADATA*() macros? I'd just provide one macro that gives a (void*) to the first byte after the mbuf structure.
The format of the metadata is up to the application, that usually casts (m + 1) into a private structure, making the macros not very useful. I suggest to move these macros outside rte_mbuf.h, in the application-specific or library-specific header, what do you think?
Regards,
Olivier
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
Quoted 73 lines
-----Original Message-----
From: Olivier MATZ [mailto:[email protected]]
Sent: Monday, September 08, 2014 1:06 PM
To: Richardson, Bruce; [email protected]
Subject: Re: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the mbuf metadataHi Bruce,
Quoted 14 lines · Bruce Richardson
Removed the explicit zero-sized metadata definition at the end of the mbuf data structure. Updated the metadata macros to take account of this change so that all existing code which uses those macros still works.
Signed-off-by Bruce Richardson <[email protected]>--- lib/librte_mbuf/rte_mbuf.h | 22 ++++++++-------------- 1 file changed, 8 insertions(+), 14 deletions(-)lib/librte_mbuf/rte_mbuf.h +0
-0lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 166
Quoted 1 line
lib/librte_mbuf/rte_mbuf.h +0
-0@@ -166,31 +166,25 @@ struct rte_mbuf {struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */
Quoted 29 lines
lib/librte_mbuf/rte_mbuf.h +5
-12struct rte_mbuf *next;/**< Next segment of scattered packet. */ -union { -uint8_t metadata[0 ]; -uint16_t metadata16[0 ]; -uint32_t metadata32[0 ]; -uint64_t metadata64[0 ]; - } __rte_cache_aligned; } __rte_cache_aligned;#define RTE_MBUF_METADATA_UINT8 (mbuf, offset) \ - (mbuf->metadata[offset]) + (((uint8_t *)&(mbuf)[1])[offset])#define RTE_MBUF_METADATA_UINT16 (mbuf, offset) \ - (mbuf->metadata16[offset/sizeof(uint16_t)]) + (((uint16_t *)&(mbuf)[1])[offset/sizeof(uint16_t)])#define RTE_MBUF_METADATA_UINT32 (mbuf, offset) \ - (mbuf->metadata32[offset/sizeof(uint32_t)]) + (((uint32_t *)&(mbuf)[1])[offset/sizeof(uint32_t)])#define RTE_MBUF_METADATA_UINT64 (mbuf, offset) \ - (mbuf->metadata64[offset/sizeof(uint64_t)]) + (((uint64_t *)&(mbuf)[1])[offset/sizeof(uint64_t)])#define RTE_MBUF_METADATA_UINT8_PTR (mbuf, offset) \ - (&mbuf->metadata[offset]) + (&RTE_MBUF_METADATA_UINT8(mbuf, offset))#define RTE_MBUF_METADATA_UINT16_PTR (mbuf, offset) \ - (&mbuf->metadata16[offset/sizeof(uint16_t)])lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 190
Quoted 10 lines
lib/librte_mbuf/rte_mbuf.h +3
-2+ (&RTE_MBUF_METADATA_UINT16(mbuf, offset))#define RTE_MBUF_METADATA_UINT32_PTR (mbuf, offset) \ - (&mbuf->metadata32[offset/sizeof(uint32_t)]) + (&RTE_MBUF_METADATA_UINT32(mbuf, offset))#define RTE_MBUF_METADATA_UINT64_PTR (mbuf, offset) \ - (&mbuf->metadata64[offset/sizeof(uint64_t)]) + (&RTE_MBUF_METADATA_UINT64(mbuf, offset))/** * Given the buf_addr returns the pointer to corresponding mbuf. I think it goes in the good direction. So:
Acked-by Olivier Matz <[email protected]>Just one question: why not removing RTE_MBUF_METADATA*() macros? I'd just provide one macro that gives a (void*) to the first byte after the mbuf structure.
Quoted 8 lines
The format of the metadata is up to the application, that usually casts (m + 1) into a private structure, making the macros not very useful. I suggest to move these macros outside
rte_mbuf.h, in the application-specific or library-specific header, what do you think?Regards,
Olivier
Yes, I'll look into that.
/Bruce
A message that is not in the archive (sent off-list or not yet received): 59AF69C657FD0841A61C55336867B5B0343EFAA3-kPTMFJFq+rELt2AQoY/[email protected]
Re: [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
Bruce, Olivier,
What is the reason to remove this field? Please explain the rationale of removing this field.
We previously agreed we need to provide an easy and standard mechanism for applications to extend the mandatory per buffer metadata (mbuf) with optional application-dependent metadata. This field just provides a clean way for the apps to know where is the end of the mandatory metadata, i.e. the first location in the packet buffer where the app can add its own metadata (of course, the app has to manage the headroom space before the first byte of packet data). A zero-size field is the standard mechanism that DPDK uses extensively in pretty much every library to access memory immediately after a header structure.
The impact of removing this field is that there is no standard way to identify where the end of the mandatory metadata is, so each application will have to reinvent this. With no clear convention, we will end up with a lot of non-standard ways. Every time the format of the mbuf structure is going to be changed, this can potentially break applications that use custom metadata, while using this simple standard mechanism would prevent this. So why remove this?
Having applications define their optional meta-data is a real need. Please take a look at the Service Chaining IEFT emerging protocols (https://datatracker.ietf.org/wg/sfc/documents/), which provide standard mechanisms for applications to define their own packet meta-data and share it between the elements of the processing pipeline (for Service Chaining, these are typically virtual machines scattered amongst the data center).
And, in my opinion, there is no negative impact/cost associated with keeping this field.
Regards,
Cristian
-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Richardson, Bruce
Sent: Tuesday, September 9, 2014 10:01 AM
To: Olivier MATZ; [email protected]
Subject: Re: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
Quoted 73 lines
-----Original Message-----
From: Olivier MATZ [mailto:[email protected]]
Sent: Monday, September 08, 2014 1:06 PM
To: Richardson, Bruce; [email protected]
Subject: Re: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the mbuf metadataHi Bruce,
Quoted 14 lines · Bruce Richardson
Removed the explicit zero-sized metadata definition at the end of the mbuf data structure. Updated the metadata macros to take account of this change so that all existing code which uses those macros still works.
Signed-off-by Bruce Richardson <[email protected]>--- lib/librte_mbuf/rte_mbuf.h | 22 ++++++++-------------- 1 file changed, 8 insertions(+), 14 deletions(-)lib/librte_mbuf/rte_mbuf.h +0
-0lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 166
Quoted 1 line
lib/librte_mbuf/rte_mbuf.h +0
-0@@ -166,31 +166,25 @@ struct rte_mbuf {struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */
Quoted 29 lines
lib/librte_mbuf/rte_mbuf.h +5
-12struct rte_mbuf *next;/**< Next segment of scattered packet. */ -union { -uint8_t metadata[0 ]; -uint16_t metadata16[0 ]; -uint32_t metadata32[0 ]; -uint64_t metadata64[0 ]; - } __rte_cache_aligned; } __rte_cache_aligned;#define RTE_MBUF_METADATA_UINT8 (mbuf, offset) \ - (mbuf->metadata[offset]) + (((uint8_t *)&(mbuf)[1])[offset])#define RTE_MBUF_METADATA_UINT16 (mbuf, offset) \ - (mbuf->metadata16[offset/sizeof(uint16_t)]) + (((uint16_t *)&(mbuf)[1])[offset/sizeof(uint16_t)])#define RTE_MBUF_METADATA_UINT32 (mbuf, offset) \ - (mbuf->metadata32[offset/sizeof(uint32_t)]) + (((uint32_t *)&(mbuf)[1])[offset/sizeof(uint32_t)])#define RTE_MBUF_METADATA_UINT64 (mbuf, offset) \ - (mbuf->metadata64[offset/sizeof(uint64_t)]) + (((uint64_t *)&(mbuf)[1])[offset/sizeof(uint64_t)])#define RTE_MBUF_METADATA_UINT8_PTR (mbuf, offset) \ - (&mbuf->metadata[offset]) + (&RTE_MBUF_METADATA_UINT8(mbuf, offset))#define RTE_MBUF_METADATA_UINT16_PTR (mbuf, offset) \ - (&mbuf->metadata16[offset/sizeof(uint16_t)])lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 190
Quoted 10 lines
lib/librte_mbuf/rte_mbuf.h +3
-2+ (&RTE_MBUF_METADATA_UINT16(mbuf, offset))#define RTE_MBUF_METADATA_UINT32_PTR (mbuf, offset) \ - (&mbuf->metadata32[offset/sizeof(uint32_t)]) + (&RTE_MBUF_METADATA_UINT32(mbuf, offset))#define RTE_MBUF_METADATA_UINT64_PTR (mbuf, offset) \ - (&mbuf->metadata64[offset/sizeof(uint64_t)]) + (&RTE_MBUF_METADATA_UINT64(mbuf, offset))/** * Given the buf_addr returns the pointer to corresponding mbuf. I think it goes in the good direction. So:
Acked-by Olivier Matz <[email protected]>Just one question: why not removing RTE_MBUF_METADATA*() macros? I'd just provide one macro that gives a (void*) to the first byte after the mbuf structure.
Quoted 8 lines
The format of the metadata is up to the application, that usually casts (m + 1) into a private structure, making the macros not very useful. I suggest to move these macros outside
rte_mbuf.h, in the application-specific or library-specific header, what do you think?Regards,
Olivier
Yes, I'll look into that.
/Bruce
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
A message that is not in the archive (sent off-list or not yet received): 3EB4FA525960D640B5BDFFD6A3D891262E070D42-kPTMFJFq+rEMvF1YICWikbfspsVTdybXVpNB7YpNyf8@public.gmane.org
Re: [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
Hello Cristian,
Quoted 2 lines
What is the reason to remove this field? Please explain the rationale of removing this field.
The rationale is explained in
http://dpdk.org/ml/archives/dev/2014-September/005232.html
"The format of the metadata is up to the application".
The type of data the application stores after the mbuf has not to be defined in the mbuf. These macros limits the types of metadata to uint8, uint16, uint32, uint64? What should I do if I need a void*, a struct foo ? Should we add a macro for each possible type?
Quoted 4 lines
We previously agreed we need to provide an easy and standard mechanism for applications to extend the mandatory per buffer metadata (mbuf) with optional application-dependent metadata.
Defining a structure in the application which does not pollute the rte_mbuf structure is "easy and standard(TM)" too.
Quoted 8 lines
This field just provides a clean way for the apps to know where is the end of the mandatory metadata, i.e. the first location in the packet buffer where the app can add its own metadata (of course, the app has to manage the headroom space before the first byte of packet data). A zero-size field is the standard mechanism that DPDK uses extensively in pretty much every library to access memory immediately after a header structure.
Having the following is clean too:
struct metadata {
...
};struct app_mbuf {
struct rte_mbuf mbuf;
struct metadata metadata;
};There is no need to define anything in the mbuf structure.
Quoted 9 lines
The impact of removing this field is that there is no standard way to identify where the end of the mandatory metadata is, so each application will have to reinvent this. With no clear convention, we will end up with a lot of non-standard ways. Every time the format of the mbuf structure is going to be changed, this can potentially break applications that use custom metadata, while using this simple standard mechanism would prevent this. So why remove this?
Waow. Five occurences of "standard" until now. Could you give a reference to the standard you're refering to? :)
Our application defines private metadata in mbufs in the way described above, we never changed that since we're supporting the dpdk. So I don't understand when you say that each time mbuf is reformatted it breaks the application.
Quoted 2 lines
Having applications define their optional meta-data is a real need.
Sure. This patch does not prevent this at all. You can continue to do exactly the same, but in the concerned application, not in the generic mbuf structure.
Quoted 3 lines
Please take a look at the Service Chaining IEFT emerging protocols (https://datatracker.ietf.org/wg/sfc/documents/), which provide standard mechanisms for applications to define their own
Six :)
I'm not sure these documents define the way to extend a packet structure with metadata in a C program. Again, Bruce's patch does not prevent to do what you need, it just moves it at the proper place.
Quoted 6 lines
packet meta-data and share it between the elements of the processing pipeline (for Service Chaining, these are typically virtual machines scattered amongst the data center).
And, in my opinion, there is no negative impact/cost associated with keeping this field.
To summarize what I think:
- this patch does not prevent to do what you want to do
- removing the macros help to have a shorter and more comprehensible mbuf structure
- the previous approach does not scale because it would require a macro per type
Quoted 8 lines
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. ClareThis e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
This is a public mailing list, this disclaimer is invalid.
Regards,
Olivier
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
Hi Olivier,
I agree that your suggested approach for application-dependent metadata makes sense, in fact the two approaches work in exactly the same way (packet metadata immediately after the regular mbuf), there is only a subtle difference, which is related to defining consistent DPDK usage guidelines.
1. Advertising the presence of application-dependent meta-data as supported mechanism
If we explicitly have a metadata zero-size field at the end of the mbuf, we basically tell people that adding their own application meta-data at the end of the mandatory meta-data (mbuf structure) is a mechanism that DPDK allows and supports, and will continue to do so for the foreseeable future. In other words, we guarantee that an application doing so will continue to build successfully with future releases of DPDK, and we will not introduce changes in DPDK that could potentially break this mechanism. It is also a hint to people of where to put their application dependent meta-data.
2. Defining a standard base address for the application-dependent metadata
- There are also libraries in DPDK that work with application dependent meta-data, currently these are the Packet Framework libraries: librte_port, librte_table, librte_pipeline. Of course, the library does not have the knowledge of the application dependent meta-data format, so they treat it as opaque array of bytes, with the offset and size of the array given as arguments. In my opinion, it is safer (and more elegant) if these libraries (and others) can rely on an mbuf API to access the application dependent meta-data (in an opaque way) rather than make an assumption about the mbuf (i.e. the location of custom metadata relative to the mbuf) that is not clearly supported/defined by the mbuf library.
- By having this API, we basically say: we define the custom meta-data base address (first location where custom metadata _could_ be placed) immediately after the mbuf, so libraries and apps accessing custom meta-data should do so by using a relative offset from this base rather than each application defining its own base: immediately after mbuf, or 128 bytes after mbuf, or 64 bytes before the end of the buffer, or other.
More (minor) comments inline below.
Thanks,
Cristian
-----Original Message-----
From: Olivier MATZ [mailto:[email protected]]
Sent: Friday, September 12, 2014 10:02 PM
To: Dumitrescu, Cristian; Richardson, Bruce; [email protected]
Subject: Re: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
Hello Cristian,
Quoted 2 lines
What is the reason to remove this field? Please explain the rationale of removing this field.
The rationale is explained in
http://dpdk.org/ml/archives/dev/2014-September/005232.html
"The format of the metadata is up to the application".
The type of data the application stores after the mbuf has not to be defined in the mbuf. These macros limits the types of metadata to uint8, uint16, uint32, uint64? What should I do if I need a void*, a struct foo ? Should we add a macro for each possible type?
[Cristian] Actually, this is not correct, as macros to access metadata through pointers (to void or scalar types) are provided as well. This pointer can be converted by the application to the format is defines.
Quoted 4 lines
We previously agreed we need to provide an easy and standard mechanism for applications to extend the mandatory per buffer metadata (mbuf) with optional application-dependent metadata.
Defining a structure in the application which does not pollute the rte_mbuf structure is "easy and standard(TM)" too.
[Cristian] I agree, both approaches work the same really, it is just the difference in advertising the presence of meta-data as supported mechanism and defining a standard base address for it.
Quoted 8 lines
This field just provides a clean way for the apps to know where is the end of the mandatory metadata, i.e. the first location in the packet buffer where the app can add its own metadata (of course, the app has to manage the headroom space before the first byte of packet data). A zero-size field is the standard mechanism that DPDK uses extensively in pretty much every library to access memory immediately after a header structure.
Having the following is clean too:
struct metadata {
...
};struct app_mbuf {
struct rte_mbuf mbuf;
struct metadata metadata;
};There is no need to define anything in the mbuf structure.
[Cristian] I agree, both approaches work the same really, it is just the difference in advertising the presence of meta-data as supported mechanism and defining a standard base address for it.
Quoted 9 lines
The impact of removing this field is that there is no standard way to identify where the end of the mandatory metadata is, so each application will have to reinvent this. With no clear convention, we will end up with a lot of non-standard ways. Every time the format of the mbuf structure is going to be changed, this can potentially break applications that use custom metadata, while using this simple standard mechanism would prevent this. So why remove this?
Waow. Five occurences of "standard" until now.
[Cristian] I am sorry :)
Could you give a
reference to the standard you're refering to? :)
[Cristian] See the IEFT Service Function Chaining link below, the environment is different (data center pipeline vs. CPU core-level pipeline), but the concepts are very similar.
Our application defines private metadata in mbufs in the way described above, we never changed that since we're supporting the dpdk. So I don't understand when you say that each time mbuf is reformatted it breaks the application.
Quoted 2 lines
Having applications define their optional meta-data is a real need.
Sure. This patch does not prevent this at all. You can continue to do exactly the same, but in the concerned application, not in the generic mbuf structure.
Quoted 3 lines
Please take a look at the Service Chaining IEFT emerging protocols (https://datatracker.ietf.org/wg/sfc/documents/), which provide standard mechanisms for applications to define their own
Six :)
I'm not sure these documents define the way to extend a packet structure with metadata in a C program. Again, Bruce's patch does not prevent to do what you need, it just moves it at the proper place.
Quoted 6 lines
packet meta-data and share it between the elements of the processing pipeline (for Service Chaining, these are typically virtual machines scattered amongst the data center).
And, in my opinion, there is no negative impact/cost associated with keeping this field.
To summarize what I think:
- this patch does not prevent to do what you want to do
- removing the macros help to have a shorter and more comprehensible mbuf structure
- the previous approach does not scale because it would require a macro per type
Quoted 8 lines
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. ClareThis e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
This is a public mailing list, this disclaimer is invalid.
Regards,
Olivier
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
A message that is not in the archive (sent off-list or not yet received): 3EB4FA525960D640B5BDFFD6A3D891262E071FE6-kPTMFJFq+rEMvF1YICWikbfspsVTdybXVpNB7YpNyf8@public.gmane.org
Re: [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
I completely agree with Cristian here, instead of leaving to applications where to place their meta data, we can provide a guidance by having this field about placement of application meta while maintaining transparency on the contents of application meta information.
Regards
Kannan Babu Ramia
Sr.System Architect
Communication Storage Infrastructure Group
DCG
-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Dumitrescu, Cristian
Sent: Wednesday, September 17, 2014 1:37 AM
To: Olivier MATZ; Richardson, Bruce; [email protected]
Subject: Re: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
Hi Olivier,
I agree that your suggested approach for application-dependent metadata makes sense, in fact the two approaches work in exactly the same way (packet metadata immediately after the regular mbuf), there is only a subtle difference, which is related to defining consistent DPDK usage guidelines.
1. Advertising the presence of application-dependent meta-data as supported mechanism If we explicitly have a metadata zero-size field at the end of the mbuf, we basically tell people that adding their own application meta-data at the end of the mandatory meta-data (mbuf structure) is a mechanism that DPDK allows and supports, and will continue to do so for the foreseeable future. In other words, we guarantee that an application doing so will continue to build successfully with future releases of DPDK, and we will not introduce changes in DPDK that could potentially break this mechanism. It is also a hint to people of where to put their application dependent meta-data.
2. Defining a standard base address for the application-dependent metadata
- There are also libraries in DPDK that work with application dependent meta-data, currently these are the Packet Framework libraries: librte_port, librte_table, librte_pipeline. Of course, the library does not have the knowledge of the application dependent meta-data format, so they treat it as opaque array of bytes, with the offset and size of the array given as arguments. In my opinion, it is safer (and more elegant) if these libraries (and others) can rely on an mbuf API to access the application dependent meta-data (in an opaque way) rather than make an assumption about the mbuf (i.e. the location of custom metadata relative to the mbuf) that is not clearly supported/defined by the mbuf library.
- By having this API, we basically say: we define the custom meta-data base address (first location where custom metadata _could_ be placed) immediately after the mbuf, so libraries and apps accessing custom meta-data should do so by using a relative offset from this base rather than each application defining its own base: immediately after mbuf, or 128 bytes after mbuf, or 64 bytes before the end of the buffer, or other.
More (minor) comments inline below.
Thanks,
Cristian
-----Original Message-----
From: Olivier MATZ [mailto:[email protected]]
Sent: Friday, September 12, 2014 10:02 PM
To: Dumitrescu, Cristian; Richardson, Bruce; [email protected]
Subject: Re: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
Hello Cristian,
Quoted 2 lines
What is the reason to remove this field? Please explain the rationale of removing this field.
The rationale is explained in
http://dpdk.org/ml/archives/dev/2014-September/005232.html
"The format of the metadata is up to the application".
The type of data the application stores after the mbuf has not to be defined in the mbuf. These macros limits the types of metadata to uint8, uint16, uint32, uint64? What should I do if I need a void*, a struct foo ? Should we add a macro for each possible type?
[Cristian] Actually, this is not correct, as macros to access metadata through pointers (to void or scalar types) are provided as well. This pointer can be converted by the application to the format is defines.
Quoted 3 lines
We previously agreed we need to provide an easy and standard mechanism for applications to extend the mandatory per buffer metadata (mbuf) with optional application-dependent metadata.
Defining a structure in the application which does not pollute the rte_mbuf structure is "easy and standard(TM)" too.
[Cristian] I agree, both approaches work the same really, it is just the difference in advertising the presence of meta-data as supported mechanism and defining a standard base address for it.
Quoted 7 lines
This field just provides a clean way for the apps to know where is the end of the mandatory metadata, i.e. the first location in the packet buffer where the app can add its own metadata (of course, the app has to manage the headroom space before the first byte of packet data). A zero-size field is the standard mechanism that DPDK uses extensively in pretty much every library to access memory immediately after a header structure.
Having the following is clean too:
struct metadata {
...
};struct app_mbuf {
struct rte_mbuf mbuf;
struct metadata metadata;
};There is no need to define anything in the mbuf structure.
[Cristian] I agree, both approaches work the same really, it is just the difference in advertising the presence of meta-data as supported mechanism and defining a standard base address for it.
Quoted 8 lines
The impact of removing this field is that there is no standard way to identify where the end of the mandatory metadata is, so each application will have to reinvent this. With no clear convention, we will end up with a lot of non-standard ways. Every time the format of the mbuf structure is going to be changed, this can potentially break applications that use custom metadata, while using this simple standard mechanism would prevent this. So why remove this?
Waow. Five occurences of "standard" until now.
[Cristian] I am sorry :)
Could you give a
reference to the standard you're refering to? :)
[Cristian] See the IEFT Service Function Chaining link below, the environment is different (data center pipeline vs. CPU core-level pipeline), but the concepts are very similar.
Our application defines private metadata in mbufs in the way described above, we never changed that since we're supporting the dpdk. So I don't understand when you say that each time mbuf is reformatted it breaks the application.
Quoted 1 line
Having applications define their optional meta-data is a real need.
Sure. This patch does not prevent this at all. You can continue to do exactly the same, but in the concerned application, not in the generic mbuf structure.
Quoted 3 lines
Please take a look at the Service Chaining IEFT emerging protocols
(https://datatracker.ietf.org/wg/sfc/documents/), which provide standard mechanisms for applications to define their own
Six :)
I'm not sure these documents define the way to extend a packet structure with metadata in a C program. Again, Bruce's patch does not prevent to do what you need, it just moves it at the proper place.
Quoted 6 lines
packet meta-data and share it between the elements of the processing pipeline (for Service Chaining, these are typically virtual machines scattered amongst the data center).
And, in my opinion, there is no negative impact/cost associated with keeping this field.
To summarize what I think:
- this patch does not prevent to do what you want to do
- removing the macros help to have a shorter and more comprehensible mbuf structure
- the previous approach does not scale because it would require a macro per type
Quoted 8 lines
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 Business address: Dromore House, East Park, Shannon, Co. ClareThis e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
This is a public mailing list, this disclaimer is invalid.
Regards,
Olivier
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 Business address: Dromore House, East Park, Shannon, Co. Clare
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
A message that is not in the archive (sent off-list or not yet received): 682698A055A0F44AA47192B20141149711B1FFE6-yHIBzpp8AekFyVwBAnZdSLfspsVTdybXVpNB7YpNyf8@public.gmane.org
Re: [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
Quoted 7 lines
-----Original Message-----
From: Ramia, Kannan Babu
Sent: Tuesday, September 16, 2014 11:06 PM
To: Dumitrescu, Cristian; Olivier MATZ; Richardson, Bruce; [email protected]
Subject: RE: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
Quoted 5 lines
I completely agree with Cristian here, instead of leaving to applications where to place their meta data, we can provide a guidance by having this field about placement of application meta while maintaining transparency on the contents of application meta information.
My opinion on this is that this is better served via documentation or a comment in the code. The reason is that this approach is not going to be suitable for all applications. The mbuf headroom being used by the metadata is actually designed to be used for any additional headers to be added to the packet - though other things can obviously be stored in it too. Therefore the amount of metadata that can be stored in it will depend from application to application, as any apps doing e.g. tunnelling will need the headroom for tunnelling headers and may only be able to store a small amount of metadata - potentially none. For larger amounts of metadata - I would feel that anything over 64-bytes or so - I have proposed adding in a separate userdata pointer in the mbuf structure so that apps have the option of storing the metadata externally e.g. pointing to a flow table entry or similar. [Please see mbuf rework patch set 3 proposal]. Because of this, I think it's better to put in a comment in the code indicating that metadata can go in the headroom, document this properly - including caveats and limitations - in the documentation, and provide an example of doing such - something we already have in the packet framework.
All that being said, and while I think this is a good patch, I don't feel too strongly about it. I'm happy enough if this particular patch does not get merged in for 1.8, as it's incidental to the overall mbuf changes.
Regards,
/Bruce
Quoted 197 lines
Regards
Kannan Babu Ramia
Sr.System Architect
Communication Storage Infrastructure Group
DCG
-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Dumitrescu, Cristian
Sent: Wednesday, September 17, 2014 1:37 AM
To: Olivier MATZ; Richardson, Bruce; [email protected]
Subject: Re: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the
mbuf metadataHi Olivier,
I agree that your suggested approach for application-dependent metadata makes sense, in fact the two approaches work in exactly the same way (packet metadata immediately after the regular mbuf), there is only a subtle difference, which is related to defining consistent DPDK usage guidelines.
1. Advertising the presence of application-dependent meta-data as supported mechanism If we explicitly have a metadata zero-size field at the end of the mbuf, we basically tell people that adding their own application meta-data at the end of the mandatory meta-data (mbuf structure) is a mechanism that DPDK allows and supports, and will continue to do so for the foreseeable future. In other words, we guarantee that an application doing so will continue to build successfully with future releases of DPDK, and we will not introduce changes in DPDK that could potentially break this mechanism. It is also a hint to people of where to put their application dependent meta-data.
2. Defining a standard base address for the application-dependent metadata
- There are also libraries in DPDK that work with application dependent meta- data, currently these are the Packet Framework libraries:librte_port,librte_table,librte_pipeline. Of course, the library does not have the knowledge of the application dependent meta-data format, so they treat it as opaque array of bytes, with the offset and size of the array given as arguments. In my opinion, it is safer (and more elegant) if these libraries (and others) can rely on an mbuf API to access the application dependent meta-data (in an opaque way) rather than make an assumption about the mbuf (i.e. the location of custom metadata relative to the mbuf) that is not clearly supported/defined by the mbuf library.
- By having this API, we basically say: we define the custom meta-data base address (first location where custom metadata _could_ be placed) immediately after the mbuf, so libraries and apps accessing custom meta-data should do so by using a relative offset from this base rather than each application defining its own base: immediately after mbuf, or 128 bytes after mbuf, or 64 bytes before the end of the buffer, or other.More (minor) comments inline below.
Thanks,
Cristian-----Original Message-----
From: Olivier MATZ [mailto:[email protected]]
Sent: Friday, September 12, 2014 10:02 PM
To: Dumitrescu, Cristian; Richardson, Bruce; [email protected]
Subject: Re: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the mbuf metadataHello Cristian,
Quoted 2 lines
What is the reason to remove this field? Please explain the rationale of removing this field.
The rationale is explained in
http://dpdk.org/ml/archives/dev/2014-September/005232.html"The format of the metadata is up to the application".
The type of data the application stores after the mbuf has not to be defined in the mbuf. These macros limits the types of metadata to uint8, uint16, uint32, uint64? What should I do if I need a void*, a struct foo ? Should we add a macro for each possible type?
[Cristian] Actually, this is not correct, as macros to access metadata through pointers (to void or scalar types) are provided as well. This pointer can be converted by the application to the format is defines.
Quoted 3 lines
We previously agreed we need to provide an easy and standard mechanism for applications to extend the mandatory per buffer metadata (mbuf) with optional application-dependent metadata.
Defining a structure in the application which does not pollute the
rte_mbufstructure is "easy and standard(TM)" too.[Cristian] I agree, both approaches work the same really, it is just the difference in advertising the presence of meta-data as supported mechanism and defining a standard base address for it.
Quoted 7 lines
This field just provides a clean way for the apps to know where is the end of the mandatory metadata, i.e. the first location in the packet buffer where the app can add its own metadata (of course, the app has to manage the headroom space before the first byte of packet data). A zero-size field is the standard mechanism that DPDK uses extensively in pretty much every library to access memory immediately after a header structure.
Having the following is clean too:
struct metadata { ... };struct app_mbuf { struct rte_mbuf mbuf; struct metadata metadata; };There is no need to define anything in the mbuf structure.
[Cristian] I agree, both approaches work the same really, it is just the difference in advertising the presence of meta-data as supported mechanism and defining a standard base address for it.
Quoted 8 lines
The impact of removing this field is that there is no standard way to identify where the end of the mandatory metadata is, so each application will have to reinvent this. With no clear convention, we will end up with a lot of non-standard ways. Every time the format of the mbuf structure is going to be changed, this can potentially break applications that use custom metadata, while using this simple standard mechanism would prevent this. So why remove this?
Waow. Five occurences of "standard" until now.
[Cristian] I am sorry :)Could you give a
reference to the standard you're refering to? :)[Cristian] See the IEFT Service Function Chaining link below, the environment is different (data center pipeline vs. CPU core-level pipeline), but the concepts are very similar.
Our application defines private metadata in mbufs in the way described above, we never changed that since we're supporting the dpdk. So I don't understand when you say that each time mbuf is reformatted it breaks the application.
Quoted 1 line
Having applications define their optional meta-data is a real need.
Sure. This patch does not prevent this at all. You can continue to do exactly the same, but in the concerned application, not in the generic mbuf structure.
Quoted 3 lines
Please take a look at the Service Chaining IEFT emerging protocols
(https://datatracker.ietf.org/wg/sfc/documents/), which provide standard mechanisms for applications to define their ownSix :)
I'm not sure these documents define the way to extend a packet structure with metadata in a C program. Again, Bruce's patch does not prevent to do what you need, it just moves it at the proper place.
Quoted 6 lines
packet meta-data and share it between the elements of the processing pipeline (for Service Chaining, these are typically virtual machines scattered amongst the data center).
And, in my opinion, there is no negative impact/cost associated with keeping this field.
To summarize what I think:
- this patch does not prevent to do what you want to do
- removing the macros help to have a shorter and more comprehensible mbuf structure
- the previous approach does not scale because it would require a macro per typeQuoted 8 lines
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 Business address: Dromore House, East Park, Shannon, Co. ClareThis e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
This is a public mailing list, this disclaimer is invalid.
Regards,
Olivier--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered
Number: 308263 Business address: Dromore House, East Park, Shannon, Co. ClareThis e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
A message that is not in the archive (sent off-list or not yet received): 59AF69C657FD0841A61C55336867B5B0343F2BD2-kPTMFJFq+rELt2AQoY/[email protected]
Re: [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
2014-09-17 10:31, Richardson, Bruce:
Quoted 16 lines
Quoted 6 lines
From: Ramia, Kannan Babu
I completely agree with Cristian here, instead of leaving to applications where to place their meta data, we can provide a guidance by having this field about placement of application meta while maintaining transparency on the contents of application meta information.
My opinion on this is that this is better served via documentation or a comment in the code. The reason is that this approach is not going to be suitable for all applications. The mbuf headroom being used by the metadata is actually designed to be used for any additional headers to be added to the packet - though other things can obviously be stored in it too. Therefore the amount of metadata that can be stored in it will depend from application to application, as any apps doing e.g. tunnelling will need the headroom for tunnelling headers and may only be able to store a small amount of metadata - potentially none. For larger amounts of metadata - I
Quoted 8 lines
would feel that anything over 64-bytes or so - I have proposed adding in a separate userdata pointer in the mbuf structure so that apps have the option of storing the metadata externally e.g. pointing to a flow table entry or similar. [Please see mbuf rework patch set 3 proposal]. Because of this, I think it's better to put in a comment in the code indicating that metadata can go in the headroom, document this properly - including caveats and limitations - in the documentation, and provide an example of doing such - something we already have in the packet framework.
I agree that replacing these markers by documentation give more accurate informations and (bonus) is simpler.
When documentation will be embedded in the git repository, I'd like to see a patch to document these mbuf usages.
Quoted 3 lines
All that being said, and while I think this is a good patch, I don't feel too strongly about it. I'm happy enough if this particular patch does not get merged in for 1.8, as it's incidental to the overall mbuf changes.
I think also it's a good patch so I keep it.
Signature · 1 line
Thomas
Re: [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
In reply to a message not in the archiveQuoted 61 lines · Olivier MATZ
Hi Bruce,
Quoted 44 lines · Bruce Richardson
Removed the explicit zero-sized metadata definition at the end of the mbuf data structure. Updated the metadata macros to take account of this change so that all existing code which uses those macros still works.
Signed-off-by Bruce Richardson <[email protected]>--- lib/librte_mbuf/rte_mbuf.h | 22 ++++++++-------------- 1 file changed, 8 insertions(+), 14 deletions(-)lib/librte_mbuf/rte_mbuf.h +5
-12@@ -166,31 +166,25 @@ struct rte_mbuf {struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */ struct rte_mbuf *next;/**< Next segment of scattered packet. */ -union { -uint8_t metadata[0 ]; -uint16_t metadata16[0 ]; -uint32_t metadata32[0 ]; -uint64_t metadata64[0 ]; - } __rte_cache_aligned; } __rte_cache_aligned;#define RTE_MBUF_METADATA_UINT8 (mbuf, offset) \ - (mbuf->metadata[offset]) + (((uint8_t *)&(mbuf)[1])[offset])#define RTE_MBUF_METADATA_UINT16 (mbuf, offset) \ - (mbuf->metadata16[offset/sizeof(uint16_t)]) + (((uint16_t *)&(mbuf)[1])[offset/sizeof(uint16_t)])#define RTE_MBUF_METADATA_UINT32 (mbuf, offset) \ - (mbuf->metadata32[offset/sizeof(uint32_t)]) + (((uint32_t *)&(mbuf)[1])[offset/sizeof(uint32_t)])#define RTE_MBUF_METADATA_UINT64 (mbuf, offset) \ - (mbuf->metadata64[offset/sizeof(uint64_t)]) + (((uint64_t *)&(mbuf)[1])[offset/sizeof(uint64_t)])#define RTE_MBUF_METADATA_UINT8_PTR (mbuf, offset) \ - (&mbuf->metadata[offset]) + (&RTE_MBUF_METADATA_UINT8(mbuf, offset))#define RTE_MBUF_METADATA_UINT16_PTR (mbuf, offset) \ - (&mbuf->metadata16[offset/sizeof(uint16_t)])lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 190
Quoted 10 lines
lib/librte_mbuf/rte_mbuf.h +3
-2+ (&RTE_MBUF_METADATA_UINT16(mbuf, offset))#define RTE_MBUF_METADATA_UINT32_PTR (mbuf, offset) \ - (&mbuf->metadata32[offset/sizeof(uint32_t)]) + (&RTE_MBUF_METADATA_UINT32(mbuf, offset))#define RTE_MBUF_METADATA_UINT64_PTR (mbuf, offset) \ - (&mbuf->metadata64[offset/sizeof(uint64_t)]) + (&RTE_MBUF_METADATA_UINT64(mbuf, offset))/** * Given the buf_addr returns the pointer to corresponding mbuf. I think it goes in the good direction. So:
Acked-by Olivier Matz <[email protected]>
Quoted 9 lines
Just one question: why not removing RTE_MBUF_METADATA*() macros? I'd just provide one macro that gives a (void*) to the first byte after the mbuf structure.
The format of the metadata is up to the application, that usually casts (m + 1) into a private structure, making the macros not very useful. I suggest to move these macros outside
rte_mbuf.h, in the application-specific or library-specific header, what do you think?
Things look to work if I just move the definitions en-masse into rte_port.h. Is that the sort of change you would be thinking of? I was wondering about replacing the typed macros here with a generic one to access just beyond the definition of the mbuf structure, but on further thought, I believe that using the buf_addr pointer in the mbuf data structure is probably enough for most applications. [An alternative to moving the definitions into rte_port.h is to move them into rte_table.h and having port_frag.c use the buf_addr pointer instead of a macro to get at the metadata. All other references to the macros apart from two in that port file, are in the tables or in apps that use the tables lib.] What do you think?
/Bruce
Re: [PATCH 07/13] mbuf: use macros only to access the mbuf metadata
Hi Bruce,
Quoted 10 lines · Bruce Richardson
Quoted 9 lines
Just one question: why not removing RTE_MBUF_METADATA*() macros? I'd just provide one macro that gives a (void*) to the first byte after the mbuf structure.
The format of the metadata is up to the application, that usually casts (m + 1) into a private structure, making the macros not very useful. I suggest to move these macros outside
rte_mbuf.h, in the application-specific or library-specific header, what do you think?
Quoted 13 lines
Things look to work if I just move the definitions en-masse into
rte_port.h. Is that the sort of change you would be thinking of? I was wondering about replacing the typed macros here with a generic one to access just beyond the definition of the mbuf structure, but on further thought, I believe that using thebuf_addrpointer in the mbuf data structure is probably enough for most applications. [An alternative to moving the definitions intorte_port.his to move them intorte_table.hand havingport_frag.cuse thebuf_addrpointer instead of a macro to get at the metadata. All other references to the macros apart from two in that port file, are in the tables or in apps that use the tables lib.]
What do you think?
Yes, moving the macros in rte_port.h looks good to me, since the libraries using rte_ports are the users of this specific metadata format.
Thanks!
Olivier
PATCH 8/13 mbuf: add named points inside the mbuf structure
In reply to a message not in the archiveAdd markers or "labels" at given points inside the mbuf which can be used instead of individual fields to identify the start of logical sections inside the mbuf.
The use of typedefs and dummy fields was chosen over using unions because of a couple reasons:
* unions cause an extra level of indentation (more likely two levels as a union containing a struct for multiple fields would be needed). This makes the lines longer than they need to be and increases the need for wrapping. [This was the main reason]
* with markers, you can apply multiple markers at the same point if wanted.
--- lib/librte_mbuf/rte_mbuf.h | 9 +++++++++ 1 file changed, 9 insertions(+)
lib/librte_mbuf/rte_mbuf.h +9 -0
@@ -115,14 +115,22 @@ extern "C" {*/ #define PKT_TX_OFFLOAD_MASK (PKT_TX_VLAN_PKT | PKT_TX_IP_CKSUM | PKT_TX_L4_MASK) +/* define a set of marker types that can be used to refer to set points in the +* mbuf */ +typedef void *MARKER [0 ];/**< generic marker for a point in a structure */ +typedef uint64_t MARKER64 [0 ];/**< marker that allows us to overwrite 8 bytes +* with a single assignment */ /** * The generic rte_mbuf, containing a packet mbuf. */ struct rte_mbuf { +MARKER cacheline0; +void *buf_addr;/**< Virtual address of segment buffer. */ phys_addr_t buf_physaddr;/**< Physical address of segment buffer. */ /* next 8 bytes are initialised on RX descriptor rearm */ +MARKER64 rearm_data;uint16_t buf_len;/**< Length of segment buffer. */ uint16_t data_off;
@@ -147,6 +155,7 @@ struct rte_mbuf {uint64_t ol_flags;/**< Offload features. */ /* remaining bytes are set on RX when pulling packet from descriptor */ +MARKER rx_descriptor_fields1;uint16_t packet_type;/**< Type of packet, e.g. protocols used */ uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segments. */
Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): 1409759378-10113-9-git-send-email-bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Olivier MATZ
Acked-byRe: [PATCH 08/13] mbuf: add named points inside the mbuf structure
Quoted 13 lines · Bruce Richardson
Add markers or "labels" at given points inside the mbuf which can be used instead of individual fields to identify the start of logical sections inside the mbuf.
The use of typedefs and dummy fields was chosen over using unions because of a couple reasons:
* unions cause an extra level of indentation (more likely two levels as a union containing a struct for multiple fields would be needed). This makes the lines longer than they need to be and increases the need for wrapping. [This was the main reason]
* with markers, you can apply multiple markers at the same point if wanted.
Quoted 1 line
Signed-off-by Bruce Richardson <[email protected]>
PATCH 9/13 ixgbe: rework vector pmd following mbuf changes
In reply to a message not in the archiveThe vector PMD expects fields to be in a specific order so that it can do vector operations on multiple fields at a time. Following mbuf rework, adjust driver to take account of the new layout and re-enable it in the config.
--- config/common_linuxapp | 2 +- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 2 +- lib/librte_pmd_ixgbe/ixgbe_rxtx.h | 4 +- lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c | 136 +++++++++++++--------------------- 4 files changed, 54 insertions(+), 90 deletions(-)
config/common_linuxapp +1 -1
@@ -191,7 +191,7 @@ CONFIG_RTE_LIBRTE_IXGBE_DEBUG_DRIVER=n CONFIG_RTE_LIBRTE_IXGBE_PF_DISABLE_STRIP_CRC=n CONFIG_RTE_LIBRTE_IXGBE_RX_ALLOW_BULK_ALLOC=y CONFIG_RTE_LIBRTE_IXGBE_ALLOW_UNSUPPORTED_SFP=n -CONFIG_RTE_IXGBE_INC_VECTOR=n +CONFIG_RTE_IXGBE_INC_VECTOR=y CONFIG_RTE_IXGBE_RX_OLFLAGS_ENABLE=y #
lib/librte_pmd_ixgbe/ixgbe_rxtx.c +1 -1
@@ -2167,7 +2167,7 @@ ixgbe_dev_rx_queue_setup(struct rte_eth_dev *dev,if (!ixgbe_rx_vec_condition_check (dev)) {PMD_INIT_LOG (INFO ,"Vector rx enabled, please make " "sure RX burst size no less than 32.\n" ); -ixgbe_rxq_vec_setup (rxq, socket_id); +ixgbe_rxq_vec_setup (rxq); dev->rx_pkt_burst = ixgbe_recv_pkts_vec; }#endif
lib/librte_pmd_ixgbe/ixgbe_rxtx.h +2 -2
@@ -115,6 +115,7 @@ struct igb_rx_queue {struct igb_rx_entry *sw_ring;/**< address of RX software ring. */ struct rte_mbuf *pkt_first_seg;/**< First segment of current packet. */ struct rte_mbuf *pkt_last_seg;/**< Last segment of current packet. */ +uint64_t mbuf_initializer;/**< value to init mbufs */ uint16_t nb_rx_desc;/**< number of RX descriptors. */ uint16_t rx_tail;/**< current value of RDT register. */ uint16_t nb_rx_hold;/**< number of held free RX desc. */
@@ -126,7 +127,6 @@ struct igb_rx_queue {#ifdef RTE_IXGBE_INC_VECTOR uint16_t rxrearm_nb;/**< the idx we start the re-arming from */ uint16_t rxrearm_start;/**< number of remaining to be re-armed */ -__m128i misc_info;/**< cache XMM combine port_id/crc/nb_segs */ #endif uint16_t rx_free_thresh;/**< max free RX desc to hold. */ uint16_t queue_id;/**< RX queue index. */
@@ -260,7 +260,7 @@ struct ixgbe_txq_ops {uint16_t ixgbe_recv_pkts_vec (void *rx_queue,struct rte_mbuf **rx_pkts,uint16_t nb_pkts);uint16_t ixgbe_xmit_pkts_vec (void *tx_queue,struct rte_mbuf **tx_pkts,uint16_t nb_pkts);int ixgbe_txq_vec_setup (struct igb_tx_queue *txq,unsigned int socket_id); -int ixgbe_rxq_vec_setup (struct igb_rx_queue *rxq,unsigned int socket_id); +int ixgbe_rxq_vec_setup (struct igb_rx_queue *rxq);int ixgbe_rx_vec_condition_check (struct rte_eth_dev *dev);#endif
lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +50 -86
246 lines not on this page · raw message
Signature · 1 line
1.9.3
PATCH 10/13 mbuf: split mbuf across two cache lines.
In reply to a message not in the archiveThis change splits the mbuf in two to move the pool and next pointers to the second cache line. This frees up 16 bytes in first cache line.
--- app/test/test_mbuf.c | 2 +- lib/librte_mbuf/rte_mbuf.h | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-)
app/test/test_mbuf.c +1 -1
@@ -782,7 +782,7 @@ test_failing_mbuf_sanity_check(void)static int test_mbuf (void ) { -RTE_BUILD_BUG_ON (sizeof (struct rte_mbuf ) !=64 ); +RTE_BUILD_BUG_ON (sizeof (struct rte_mbuf ) !=CACHE_LINE_SIZE *2 );/* create pktmbuf pool if it does not exist */ if (pktmbuf_pool ==NULL ) {
lib/librte_mbuf/rte_mbuf.h +2 -1
@@ -171,7 +171,8 @@ struct rte_mbuf {uint32_t sched;/**< Hierarchical scheduler */ } hash;/**< hash information */ -/* fields only used in slow path or on TX */ +/* second cache line - fields only used in slow path or on TX */ + MARKER cacheline1 __rte_cache_aligned;struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */ struct rte_mbuf *next;/**< Next segment of scattered packet. */
Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): 1409759378-10113-11-git-send-email-bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH 10/13] mbuf: split mbuf across two cache lines.
Hi Bruce,
Quoted 4 lines · Bruce Richardson
This change splits the mbuf in two to move the pool and next pointers to the second cache line. This frees up 16 bytes in first cache line.
Signed-off-by Bruce Richardson <[email protected]>
I think you should provide more explanation in the commit log about why this change is needed. I remeber you already provided the arguments on the mailing list, but it could be interesting to have them in the log.
Regards,
Olivier
PATCH 11/13 mbuf: move l2_len and l3_len to second cache line
In reply to a message not in the archiveThe l2_len and l3_len fields are used for TX offloads and so should be put on the second cache line, along with the other fields only used on TX.
--- lib/librte_mbuf/rte_mbuf.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_mbuf/rte_mbuf.h +4 -2
@@ -159,8 +159,7 @@ struct rte_mbuf {uint16_t packet_type;/**< Type of packet, e.g. protocols used */ uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ - uint16_t l3_len:9; /**< L3 (IP) Header Length. */ - uint16_t l2_len:7; /**< L2 (MAC) Header Length. */ +uint16_t reserved;uint16_t vlan_tci;/**< VLAN Tag Control Identifier (CPU order). */ union {uint32_t rss;/**< RSS hash result if RSS enabled */
@@ -176,6 +175,9 @@ struct rte_mbuf {struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */ struct rte_mbuf *next;/**< Next segment of scattered packet. */ +/* fields to support TX offloads */ + uint16_t l3_len:9; /**< L3 (IP) Header Length. */ + uint16_t l2_len:7; /**< L2 (MAC) Header Length. */ } __rte_cache_aligned;#define RTE_MBUF_METADATA_UINT8 (mbuf, offset) \
Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): 1409759378-10113-12-git-send-email-bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH 11/13] mbuf: move l2_len and l3_len to second cache line
Hello Bruce,
I'm a little bit concerned about performance issues that would arise if these fields would go to the 2nd cache line.
For exampe, l2_len and l3_len fields are used by librte_ip_frag to find L3 and L4 headers position inside mbuf data. Thus, these values should be calculated by NIC offload, or by user on RX leg.
Secondly, (I wouldn't say on behalf of everyone, but) we use these fields in our libraries as well for needs of classification. For instance, in case you try to support other ethertypes which are not supported by NIC offload (MPLS, IPX etc), but you still need to point out L3 and L3 headers.
If my concerns are consistent, what would be possible suggestions?
03.09.2014 21:49, Bruce Richardson пишет:
Quoted 5 lines
The
l2_lenandl3_lenfields are used for TX offloads and so should be put on the second cache line, along with the other fields only used on TX.Signed-off-by Bruce Richardson <[email protected]>
Signature · 5 lines
Sincerely, Yerden Zhumabekov STS, ACI Astana, KZ
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): 20140904102744.GA23231-IWE99D/oH1/[email protected]
Re: [PATCH 11/13] mbuf: move l2_len and l3_len to second cache line
I get your point. I've also read throught the code of various PMDs and have found no indication of setting l2_len/l3_len fields as well.
As for testing, we'd be happy to test the patchset but we are now in process of building our testing facilities so we are not ready to provide enough workload for the hardware/software. I was also wondering if anyone has run some test and can provide some numbers on that matter.
Personally, I don't think frag/reassemly app is a perfect example for evaluating 2nd cache line performance penalty. The offsets to L3 and L4 headers need to be calculated for all TCP/IP traffic and fragmented traffic is not representative in this case. Maybe it would be better to write an app which calculates these offsets for different set of mbufs and provides some stats. For example, l2fwd/l3fwd + additional l2_len and l3_len calculation.
And I'm also figuring out how to rewrite our app/libs (prefetch etc) to reflect the future changes in mbuf, hence my concerns :)
04.09.2014 16:27, Bruce Richardson пишет:
Quoted 40 lines
Hi Yerden,
I understand your concerns and it's good to have this discussion.
There are a number of reasons why I've moved these particular fields to the second cache line. Firstly, the main reason is that, obviously enough, not all fields will fit in cache line 0, and we need to prioritize what does get stored there. The guiding principle behind what fields get moved or not that I've chosen to use for this patch set is to move fields that are not used on the receive path (or the fastpath receive path, more specifically - so that we can move fields only used by jumbo frames that span mbufs) to the second cache line. From a search through the existing codebase, there are no drivers which set the l2/l3 length fields on RX, this is only used in reassembly libraries/apps and by the drivers on TX.
The other reason for moving it to the second cache line is that it logically belongs with all the other length fields that we need to add to enable tunneling support. [To get an idea of the extra fields that I propose adding to the mbuf, please see the RFC patchset I sent out previously as "[RFC PATCH 00/14] Extend the mbuf structure"]. While we probably can fit the 16-bits needed for l2/l3 length on the mbuf line 0, there is not enough room for all the lengths so we would end up splitting them with other fields in between.
So, in terms of what do to about this particular issue. I would hope that for applications that use these fields the impact should be small and/or possible to work around e.g. maybe prefetch second cache line on RX in driver. If not, then I'm happy to see about withdrawing this particular change and seeing if we can keep l2/l3 lengths on cache line zero, with other length fields being on cache line 1.
Question: would you consider the ip fragmentation and reassembly example apps in the Intel DPDK releases good examples to test to see the impacts of this change, or is there some other test you would prefer that I look to do? Can you perhaps test out the patch sets for the mbuf that I've upstreamed so far and let me know what regressions, if any, you see in your use-case scenarios?
Regards,
/Bruce
Signature · 5 lines
Sincerely, Yerden Zhumabekov STS, ACI Astana, KZ
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 11/13] mbuf: move l2_len and l3_len to second cache line
Quoted 16 lines · Yerden Zhumabekov
I get your point. I've also read throught the code of various PMDs and have found no indication of setting l2_len/l3_len fields as well.
As for testing, we'd be happy to test the patchset but we are now in process of building our testing facilities so we are not ready to provide enough workload for the hardware/software. I was also wondering if anyone has run some test and can provide some numbers on that matter.
Personally, I don't think frag/reassemly app is a perfect example for evaluating 2nd cache line performance penalty. The offsets to L3 and L4 headers need to be calculated for all TCP/IP traffic and fragmented traffic is not representative in this case. Maybe it would be better to write an app which calculates these offsets for different set of mbufs and provides some stats. For example, l2fwd/l3fwd + additional
l2_lenandl3_lencalculation.
Quoted 3 lines
And I'm also figuring out how to rewrite our app/libs (prefetch etc) to reflect the future changes in mbuf, hence my concerns :)
Just a final point on this. Note that the second cache line is always being read by the TX leg of the code to free back mbufs to their mbuf pool post- transmit. The overall fast-path RX+TX benchmarks show no performance degradation due to that access.
For sample apps, you make a good point indeed about the existing app not being very useful as they work on larger packets. I'll see what I can throw together here to make a more realistic test.
/Bruce
Quoted 49 lines
04.09.2014 16:27, Bruce Richardson ??????????:
Quoted 40 lines
Hi Yerden,
I understand your concerns and it's good to have this discussion.
There are a number of reasons why I've moved these particular fields to the second cache line. Firstly, the main reason is that, obviously enough, not all fields will fit in cache line 0, and we need to prioritize what does get stored there. The guiding principle behind what fields get moved or not that I've chosen to use for this patch set is to move fields that are not used on the receive path (or the fastpath receive path, more specifically - so that we can move fields only used by jumbo frames that span mbufs) to the second cache line. From a search through the existing codebase, there are no drivers which set the l2/l3 length fields on RX, this is only used in reassembly libraries/apps and by the drivers on TX.
The other reason for moving it to the second cache line is that it logically belongs with all the other length fields that we need to add to enable tunneling support. [To get an idea of the extra fields that I propose adding to the mbuf, please see the RFC patchset I sent out previously as "[RFC PATCH 00/14] Extend the mbuf structure"]. While we probably can fit the 16-bits needed for l2/l3 length on the mbuf line 0, there is not enough room for all the lengths so we would end up splitting them with other fields in between.
So, in terms of what do to about this particular issue. I would hope that for applications that use these fields the impact should be small and/or possible to work around e.g. maybe prefetch second cache line on RX in driver. If not, then I'm happy to see about withdrawing this particular change and seeing if we can keep l2/l3 lengths on cache line zero, with other length fields being on cache line 1.
Question: would you consider the ip fragmentation and reassembly example apps in the Intel DPDK releases good examples to test to see the impacts of this change, or is there some other test you would prefer that I look to do? Can you perhaps test out the patch sets for the mbuf that I've upstreamed so far and let me know what regressions, if any, you see in your use-case scenarios?
Regards,
/BruceSignature · 5 lines
Sincerely, Yerden Zhumabekov STS, ACI Astana, KZ
PATCH 12/13 ixgbe: Fix perf regression due to moved pool ptr
In reply to a message not in the archiveAdjust the fast-path code to fix the regression caused by the pool pointer moving to the second cache line. This change adjusts the prefetching and also the way in which the mbufs are freed back to the mempool.
Note: slow-path e.g. path supporting jumbo frames, is still slower, but is dealt with by a later commit
--- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 8 ++- lib/librte_pmd_ixgbe/ixgbe_rxtx.h | 14 +----- lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c | 93 +++++++++++++---------------------- 3 files changed, 39 insertions(+), 76 deletions(-)
lib/librte_pmd_ixgbe/ixgbe_rxtx.c +3 -5
@@ -142,10 +142,6 @@ ixgbe_tx_free_bufs(struct igb_tx_queue *txq)*/ txep = &(txq->sw_ring[txq->tx_next_dd - (txq->tx_rs_thresh -1 )]); -/* prefetch the mbufs that are about to be freed */ -for (i =0 ; i < txq->tx_rs_thresh; ++i) -rte_prefetch0 ((txep + i)->mbuf); -/* free buffers one at a time */ if ((txq->txq_flags & (uint32_t )ETH_TXQ_FLAGS_NOREFCOUNT ) !=0 ) {for (i =0 ; i < txq->tx_rs_thresh; ++i, ++txep) {
@@ -186,6 +182,7 @@ tx4(volatile union ixgbe_adv_tx_desc *txdp, struct rte_mbuf **pkts) ((uint32_t )DCMD_DTYP_FLAGS | pkt_len); txdp->read.olinfo_status = (pkt_len <<IXGBE_ADVTXD_PAYLEN_SHIFT ); +rte_prefetch0 (&(*pkts)->pool); } }
@@ -205,6 +202,7 @@ tx1(volatile union ixgbe_adv_tx_desc *txdp, struct rte_mbuf **pkts) ((uint32_t )DCMD_DTYP_FLAGS | pkt_len); txdp->read.olinfo_status = (pkt_len <<IXGBE_ADVTXD_PAYLEN_SHIFT ); +rte_prefetch0 (&(*pkts)->pool); }/*
@@ -1876,7 +1874,7 @@ ixgbe_dev_tx_queue_setup(struct rte_eth_dev *dev,PMD_INIT_LOG (INFO ,"Using simple tx code path\n" );#ifdef RTE_IXGBE_INC_VECTOR if (txq->tx_rs_thresh <=RTE_IXGBE_TX_MAX_FREE_BUF_SZ && -ixgbe_txq_vec_setup (txq, socket_id) ==0 ) { +ixgbe_txq_vec_setup (txq) ==0 ) {PMD_INIT_LOG (INFO ,"Vector tx enabled.\n" ); dev->tx_pkt_burst = ixgbe_xmit_pkts_vec; }
lib/librte_pmd_ixgbe/ixgbe_rxtx.h +1 -13
@@ -96,14 +96,6 @@ struct igb_tx_entry_v { };/** -* continuous entry sequence, gather by the same mempool -*/ -struct igb_tx_entry_seq { -const struct rte_mempool * pool; -uint32_t same_pool; -}; - -/** * Structure associated with each RX queue. */ struct igb_rx_queue {
@@ -191,10 +183,6 @@ struct igb_tx_queue {volatile union ixgbe_adv_tx_desc *tx_ring;uint64_t tx_ring_phys_addr;/**< TX ring DMA address. */ struct igb_tx_entry *sw_ring;/**< virtual address of SW ring. */ -#ifdef RTE_IXGBE_INC_VECTOR -/** continuous tx entry sequence within the same mempool */ -struct igb_tx_entry_seq *sw_ring_seq; -#endif volatile uint32_t *tdt_reg_addr;/**< Address of TDT register. */ uint16_t nb_tx_desc;/**< number of TX descriptors. */ uint16_t tx_tail;/**< current value of TDT reg. */
@@ -259,7 +247,7 @@ struct ixgbe_txq_ops {#ifdef RTE_IXGBE_INC_VECTOR uint16_t ixgbe_recv_pkts_vec (void *rx_queue,struct rte_mbuf **rx_pkts,uint16_t nb_pkts);uint16_t ixgbe_xmit_pkts_vec (void *tx_queue,struct rte_mbuf **tx_pkts,uint16_t nb_pkts); -int ixgbe_txq_vec_setup (struct igb_tx_queue *txq,unsigned int socket_id); +int ixgbe_txq_vec_setup (struct igb_tx_queue *txq);int ixgbe_rxq_vec_setup (struct igb_rx_queue *rxq);int ixgbe_rx_vec_condition_check (struct rte_eth_dev *dev);#endif
lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +35 -58
188 lines not on this page · raw message
Signature · 1 line
1.9.3
PATCH 13/13 ixgbe: Improve slow-path perf: vector scattered RX
In reply to a message not in the archiveProvide a wrapper routine to enable receive of scattered packets with a vector driver.
--- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 16 ++++ lib/librte_pmd_ixgbe/ixgbe_rxtx.h | 1 + lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c | 173 ++++++++++++++++++++++++++++++++-- 3 files changed, 183 insertions(+), 7 deletions(-)
lib/librte_pmd_ixgbe/ixgbe_rxtx.c +16 -0
@@ -3477,12 +3477,20 @@ ixgbe_dev_rx_init(struct rte_eth_dev *dev)if ((dev->data->dev_conf.rxmode.max_rx_pkt_len +2 *IXGBE_VLAN_TAG_SIZE ) > buf_size){ dev->data->scattered_rx =1 ; +#ifdef RTE_IXGBE_INC_VECTOR + dev->rx_pkt_burst = ixgbe_recv_scattered_pkts_vec; +#else dev->rx_pkt_burst = ixgbe_recv_scattered_pkts; +#endif } }if (dev->data->dev_conf.rxmode.enable_scatter) { +#ifdef RTE_IXGBE_INC_VECTOR + dev->rx_pkt_burst = ixgbe_recv_scattered_pkts_vec; +#else dev->rx_pkt_burst = ixgbe_recv_scattered_pkts; +#endif dev->data->scattered_rx =1 ; }
@@ -3970,12 +3978,20 @@ ixgbevf_dev_rx_init(struct rte_eth_dev *dev)if ((dev->data->dev_conf.rxmode.max_rx_pkt_len +2 *IXGBE_VLAN_TAG_SIZE ) > buf_size) { dev->data->scattered_rx =1 ; +#ifdef RTE_IXGBE_INC_VECTOR + dev->rx_pkt_burst = ixgbe_recv_scattered_pkts_vec; +#else dev->rx_pkt_burst = ixgbe_recv_scattered_pkts; +#endif } }if (dev->data->dev_conf.rxmode.enable_scatter) { +#ifdef RTE_IXGBE_INC_VECTOR + dev->rx_pkt_burst = ixgbe_recv_scattered_pkts_vec; +#else dev->rx_pkt_burst = ixgbe_recv_scattered_pkts; +#endif dev->data->scattered_rx =1 ; }
lib/librte_pmd_ixgbe/ixgbe_rxtx.h +1 -0
@@ -246,6 +246,7 @@ struct ixgbe_txq_ops {#ifdef RTE_IXGBE_INC_VECTOR uint16_t ixgbe_recv_pkts_vec (void *rx_queue,struct rte_mbuf **rx_pkts,uint16_t nb_pkts); +uint16_t ixgbe_recv_scattered_pkts_vec (void *rx_queue,struct rte_mbuf **rx_pkts,uint16_t nb_pkts);uint16_t ixgbe_xmit_pkts_vec (void *tx_queue,struct rte_mbuf **tx_pkts,uint16_t nb_pkts);int ixgbe_txq_vec_setup (struct igb_tx_queue *txq);int ixgbe_rxq_vec_setup (struct igb_rx_queue *rxq);
lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +166 -7
223 lines not on this page · raw message
Signature · 1 line
1.9.3
v2 cover Mbuf Structure Rework, part 2
In reply to a message not in the archiveThis patch set continues on from the changes in part 1, and depends upon that patch set.
This patch set reorders the fields in the mbuf structure and splits the structure across two cache lines, given lots of new space for new fields to be added. This set uses some of that space by expanding the ol_flags field. A part 3 patchset is planned to introduce some other new fields into the new mbuf structure.
With the splitting of the mbuf across multiple cache lines, performance degradations are seen inside the drivers, both fast-path and slow path. For the fast-path, this patchset reworks the way in which the pool pointer is used to free packets post-TX, which removes the perf regression. For the slow path, an alternative approach is taken - a new scattered packets RX function is introduced into the vector PMD. Using this function, throughput for the slow path RX-TX using testpmd is increased by up to 20% over the original baseline.
Changes in V2:
* General minor updates follow comments on V1 set
* Updated a number of patches to include KNI mbuf changes where needed
* Deferred the patch to add the packet type field to a future patch set
* After removing meta-data element from mbuf structure also added in patch to move the macros to rte_port/rte_port.h
Bruce Richardson (11):
mbuf: reorder fields by time of use
mbuf: expand ol_flags field to 64-bits
mbuf: introduce a flag to indicate a control mbuf
mbuf: minor changes for readability
mbuf: use macros only to access the mbuf metadata
mbuf: add named points inside the mbuf structure
ixgbe: rework vector pmd following mbuf changes
mbuf: split mbuf across two cache lines.
mbuf: move l2_len and l3_len to second cache line
ixgbe: Fix perf regression due to moved pool ptr
ixgbe: Improve slow-path perf: vector scattered RX
Olivier Matz (1):
mbuf: replace data pointer by an offset
app/test-pmd/config.c | 8 +- app/test-pmd/csumonly.c | 10 +- app/test-pmd/flowgen.c | 2 +- app/test-pmd/icmpecho.c | 2 +- app/test-pmd/ieee1588fwd.c | 4 +- app/test-pmd/macfwd-retry.c | 2 +- app/test-pmd/macfwd.c | 2 +- app/test-pmd/macswap.c | 2 +- app/test-pmd/rxonly.c | 4 +- app/test-pmd/testpmd.c | 2 +- app/test-pmd/testpmd.h | 4 +- app/test-pmd/txonly.c | 9 +- app/test/packet_burst_generator.c | 7 +- app/test/test_mbuf.c | 8 +- app/test/test_table_acl.c | 7 +- app/test/test_table_pipeline.c | 8 +- examples/exception_path/main.c | 3 +- examples/vhost/main.c | 37 +- examples/vhost_xen/main.c | 14 +- .../linuxapp/eal/include/exec-env/rte_kni_common.h | 20 +- lib/librte_eal/linuxapp/kni/kni_net.c | 18 +- lib/librte_ip_frag/rte_ipv4_fragmentation.c | 6 +- lib/librte_ip_frag/rte_ipv6_fragmentation.c | 6 +- lib/librte_mbuf/rte_mbuf.c | 10 +- lib/librte_mbuf/rte_mbuf.h | 139 ++++---- lib/librte_pmd_bond/rte_eth_bond_pmd.c | 4 +- lib/librte_pmd_e1000/em_rxtx.c | 46 ++- lib/librte_pmd_e1000/igb_rxtx.c | 75 ++-- lib/librte_pmd_i40e/i40e_rxtx.c | 48 +-- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 100 +++--- lib/librte_pmd_ixgbe/ixgbe_rxtx.h | 24 +- lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c | 389 +++++++++++++-------- lib/librte_pmd_pcap/rte_eth_pcap.c | 9 +- lib/librte_pmd_virtio/virtio_rxtx.c | 10 +- lib/librte_pmd_virtio/virtqueue.h | 3 +- lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c | 5 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 37 files changed, 586 insertions(+), 463 deletions(-)
Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): 1410441347-22840-1-git-send-email-bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
PATCH v2 1/13 mbuf: replace data pointer by an offset
From: Olivier Matz <[email protected]>
Original patch:
The mbuf structure already contains a pointer to the beginning of the buffer (m->buf_addr). It is not needed to use 8 bytes again to store another pointer to the beginning of the data.
Using a 16 bits unsigned integer is enough as we know that a mbuf is never longer than 64KB. We gain 6 bytes in the structure thanks to this modification.
This version:
* Updated original patch to apply to latest on mainline.
* Disabled vector PMD in config as it relies heavily on the mbuf layout This will be re-enabled in a subsequent commit once vPMD has been reworked to take account of mbuf changes.
Updated in V2:
* Updated the KNI buffer structure to match that of new mbuf
* fixed checkpatch errors.
--- app/test-pmd/csumonly.c | 2 +- app/test-pmd/flowgen.c | 2 +- app/test-pmd/icmpecho.c | 2 +- app/test-pmd/ieee1588fwd.c | 4 +-- app/test-pmd/macfwd-retry.c | 2 +- app/test-pmd/macfwd.c | 2 +- app/test-pmd/macswap.c | 2 +- app/test-pmd/rxonly.c | 2 +- app/test-pmd/testpmd.c | 2 +- app/test-pmd/txonly.c | 7 ++-- app/test/packet_burst_generator.c | 7 ++-- app/test/test_mbuf.c | 6 ++-- app/test/test_table_acl.c | 7 ++-- app/test/test_table_pipeline.c | 8 ++--- config/common_linuxapp | 2 +- examples/exception_path/main.c | 3 +- examples/vhost/main.c | 37 ++++++++++--------- examples/vhost_xen/main.c | 14 ++++---- .../linuxapp/eal/include/exec-env/rte_kni_common.h | 10 +++--- lib/librte_eal/linuxapp/kni/kni_net.c | 18 ++++++---- lib/librte_ip_frag/rte_ipv4_fragmentation.c | 6 ++-- lib/librte_ip_frag/rte_ipv6_fragmentation.c | 6 ++-- lib/librte_mbuf/rte_mbuf.c | 6 ++-- lib/librte_mbuf/rte_mbuf.h | 42 ++++++++++------------ lib/librte_pmd_bond/rte_eth_bond_pmd.c | 4 +-- lib/librte_pmd_e1000/em_rxtx.c | 12 +++---- lib/librte_pmd_e1000/igb_rxtx.c | 13 ++++--- lib/librte_pmd_i40e/i40e_rxtx.c | 17 +++++---- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 13 +++---- lib/librte_pmd_ixgbe/ixgbe_rxtx.h | 3 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 9 +++-- lib/librte_pmd_virtio/virtio_rxtx.c | 10 +++--- lib/librte_pmd_virtio/virtqueue.h | 3 +- lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c | 5 ++- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 35 files changed, 146 insertions(+), 144 deletions(-)
app/test-pmd/csumonly.c +1 -1
@@ -263,7 +263,7 @@ pkt_burst_checksum_forward(struct fwd_stream *fs) pkt_ol_flags = mb->ol_flags; ol_flags = (uint16_t ) (pkt_ol_flags & (~PKT_TX_L4_MASK )); - eth_hdr = (struct ether_hdr *) mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *); eth_type =rte_be_to_cpu_16 (eth_hdr->ether_type);if (eth_type ==ETHER_TYPE_VLAN ) {/* Only allow single VLAN label here */
app/test-pmd/flowgen.c +1 -1
@@ -175,7 +175,7 @@ pkt_burst_flow_gen(struct fwd_stream *fs) pkt->next =NULL ;/* Initialize Ethernet header. */ - eth_hdr = (struct ether_hdr *)pkt->data; + eth_hdr = rte_pktmbuf_mtod(pkt, struct ether_hdr *);ether_addr_copy (&cfg_ether_dst, ð_hdr->d_addr);ether_addr_copy (&cfg_ether_src, ð_hdr->s_addr); eth_hdr->ether_type =rte_cpu_to_be_16 (ETHER_TYPE_IPv4);
app/test-pmd/icmpecho.c +1 -1
@@ -330,7 +330,7 @@ reply_to_icmp_echo_rqsts(struct fwd_stream *fs) nb_replies =0 ;for (i =0 ; i < nb_rx; i++) { pkt = pkts_burst[i]; - eth_h = (struct ether_hdr *) pkt->data; + eth_h = rte_pktmbuf_mtod(pkt, struct ether_hdr *); eth_type =RTE_BE_TO_CPU_16 (eth_h->ether_type); l2_len =sizeof (struct ether_hdr );if (verbose_level >0 ) {
app/test-pmd/ieee1588fwd.c +2 -2
@@ -546,7 +546,7 @@ ieee1588_packet_fwd(struct fwd_stream *fs)* Check that the received packet is a PTP packet that was detected * by the hardware. */ - eth_hdr = (struct ether_hdr *)mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *); eth_type =rte_be_to_cpu_16 (eth_hdr->ether_type);if (! (mb->ol_flags &PKT_RX_IEEE1588_PTP )) {if (eth_type ==ETHER_TYPE_1588 ) {
@@ -574,7 +574,7 @@ ieee1588_packet_fwd(struct fwd_stream *fs)* Check that the received PTP packet is a PTP V2 packet of type * PTP_SYNC_MESSAGE. */ - ptp_hdr = (struct ptpv2_msg *) ((char *) mb->data + + ptp_hdr = (struct ptpv2_msg *) (rte_pktmbuf_mtod(mb, char *) +sizeof (struct ether_hdr ));if (ptp_hdr->version !=0x02 ) { printf("Port %u Received PTP V2 Ethernet frame with wrong PTP"
app/test-pmd/macfwd-retry.c +1 -1
@@ -119,7 +119,7 @@ pkt_burst_mac_retry_forward(struct fwd_stream *fs) fs->rx_packets += nb_rx;for (i =0 ; i < nb_rx; i++) { mb = pkts_burst[i]; - eth_hdr = (struct ether_hdr *) mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *);ether_addr_copy (&peer_eth_addrs[fs->peer_addr], ð_hdr->d_addr); ether_addr_copy(&ports[fs->tx_port].eth_addr,
app/test-pmd/macfwd.c +1 -1
@@ -110,7 +110,7 @@ pkt_burst_mac_forward(struct fwd_stream *fs) txp = &ports[fs->tx_port];for (i =0 ; i < nb_rx; i++) { mb = pkts_burst[i]; - eth_hdr = (struct ether_hdr *) mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *);ether_addr_copy (&peer_eth_addrs[fs->peer_addr], ð_hdr->d_addr); ether_addr_copy(&ports[fs->tx_port].eth_addr,
app/test-pmd/macswap.c +1 -1
@@ -110,7 +110,7 @@ pkt_burst_mac_swap(struct fwd_stream *fs) txp = &ports[fs->tx_port];for (i =0 ; i < nb_rx; i++) { mb = pkts_burst[i]; - eth_hdr = (struct ether_hdr *) mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *);/* Swap dest and src mac addresses. */ ether_addr_copy (ð_hdr->d_addr, &addr);
app/test-pmd/rxonly.c +1 -1
@@ -149,7 +149,7 @@ pkt_burst_receive(struct fwd_stream *fs)rte_pktmbuf_free (mb);continue ; } - eth_hdr = (struct ether_hdr *) mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *); eth_type =RTE_BE_TO_CPU_16 (eth_hdr->ether_type); ol_flags = mb->ol_flags;print_ether_addr (" src=" , ð_hdr->s_addr);
app/test-pmd/testpmd.c +1 -1
@@ -404,7 +404,7 @@ testpmd_mbuf_ctor(struct rte_mempool *mp, mb_ctor_arg->seg_buf_offset); mb->buf_len = mb_ctor_arg->seg_buf_size; mb->ol_flags =0 ; - mb->data = (char *) mb->buf_addr +RTE_PKTMBUF_HEADROOM ; + mb->data_off =RTE_PKTMBUF_HEADROOM ; mb->nb_segs =1 ; mb->l2_l3_len =0 ; mb->vlan_tci =0 ;
app/test-pmd/txonly.c +4 -3
@@ -111,13 +111,13 @@ copy_buf_to_pkt_segs(void* buf, unsigned len, struct rte_mbuf *pkt, seg = seg->next; } copy_len = seg->data_len - offset; - seg_buf = ((char *) seg->data + offset); + seg_buf = (rte_pktmbuf_mtod(seg, char *) + offset);while (len > copy_len) {rte_memcpy (seg_buf, buf, (size_t ) copy_len); len -= copy_len; buf = ((char *) buf + copy_len); seg = seg->next; - seg_buf = seg->data; + seg_buf = rte_pktmbuf_mtod(seg, char *); }rte_memcpy (seg_buf, buf, (size_t ) len); }
@@ -126,7 +126,8 @@ static inline void copy_buf_to_pkt(void* buf, unsigned len, struct rte_mbuf *pkt, unsigned offset) {if (offset + len <= pkt->data_len) { -rte_memcpy (((char *) pkt->data + offset), buf, (size_t ) len); + rte_memcpy((rte_pktmbuf_mtod(pkt, char *) + offset), + buf, (size_t ) len);return ; }copy_buf_to_pkt_segs (buf, len, pkt, offset);
app/test/packet_burst_generator.c +4 -3
@@ -59,13 +59,13 @@ copy_buf_to_pkt_segs(void *buf, unsigned len, struct rte_mbuf *pkt, seg = seg->next; } copy_len = seg->data_len - offset; - seg_buf = ((char *) seg->data + offset); + seg_buf = rte_pktmbuf_mtod(seg, char *) + offset;while (len > copy_len) {rte_memcpy (seg_buf, buf, (size_t ) copy_len); len -= copy_len; buf = ((char *) buf + copy_len); seg = seg->next; - seg_buf = seg->data; + seg_buf = rte_pktmbuf_mtod(seg, void *); }rte_memcpy (seg_buf, buf, (size_t ) len); }
@@ -74,7 +74,8 @@ static inline void copy_buf_to_pkt(void *buf, unsigned len, struct rte_mbuf *pkt, unsigned offset) {if (offset + len <= pkt->data_len) { -rte_memcpy (((char *) pkt->data + offset), buf, (size_t ) len); + rte_memcpy(rte_pktmbuf_mtod(pkt, char *) + offset, + buf, (size_t) len);return ; }copy_buf_to_pkt_segs (buf, len, pkt, offset);
app/test/test_mbuf.c +3 -3
@@ -432,7 +432,7 @@ test_pktmbuf_pool_ptr(void)printf ("rte_pktmbuf_alloc() failed (%u)\n" , i); ret =-1 ; } - m[i]->data =RTE_PTR_ADD (m[i]->data,64 ); + m[i]->data_off +=64 ; }/* free them */
@@ -451,8 +451,8 @@ test_pktmbuf_pool_ptr(void)printf ("rte_pktmbuf_alloc() failed (%u)\n" , i); ret =-1 ; } -if (m[i]->data !=RTE_PTR_ADD (m[i]->buf_addr,RTE_PKTMBUF_HEADROOM )) { -printf ("data pointer not set properly\n" ); +if (m[i]->data_off !=RTE_PKTMBUF_HEADROOM ) { +printf ("invalid data_off\n" ); ret =-1 ; } }
app/test/test_table_acl.c +4 -3
@@ -513,7 +513,7 @@ test_pipeline_single_filter(int expected_count)struct rte_mbuf *mbuf; mbuf =rte_pktmbuf_alloc (pool); -memset (mbuf->data,0x00 , + memset(rte_pktmbuf_mtod(mbuf, char *), 0x00, sizeof(struct ipv4_5tuple)); five_tuple.proto = j;
@@ -522,7 +522,7 @@ test_pipeline_single_filter(int expected_count) five_tuple.port_src =rte_bswap16 (100 + j); five_tuple.port_dst =rte_bswap16 (200 + j); -memcpy (mbuf->data, &five_tuple, + memcpy(rte_pktmbuf_mtod(mbuf, char *), &five_tuple, sizeof(struct ipv4_5tuple));RTE_LOG (INFO ,PIPELINE ,"%s: Enqueue onto ring %d\n" , __func__, i);
@@ -549,7 +549,8 @@ test_pipeline_single_filter(int expected_count)printf ("Got %d object(s) from ring %d!\n" , ret, i);for (j =0 ; j < ret; j++) { mbuf = (struct rte_mbuf *)objs[j]; -rte_hexdump (stdout,"mbuf" , mbuf->data,64 ); +rte_hexdump (stdout,"mbuf" , + rte_pktmbuf_mtod(mbuf, char *), 64);rte_pktmbuf_free (mbuf); } tx_count += ret;
app/test/test_table_pipeline.c +2 -6
@@ -39,11 +39,6 @@#include "test_table.h" #include "test_table_pipeline.h" -#define RTE_CBUF_UINT8_PTR (cbuf, offset) \ - (&cbuf->data[offset]) -#define RTE_CBUF_UINT32_PTR (cbuf, offset) \ - (&cbuf->data32[offset/sizeof(uint32_t)]) -#if 0 static rte_pipeline_port_out_action_handler port_action_0x00
@@ -498,7 +493,8 @@ test_pipeline_single_filter(int test_type, int expected_count)printf ("Got %d object(s) from ring %d!\n" , ret, i);for (j =0 ; j < ret; j++) { mbuf = (struct rte_mbuf *)objs[j]; -rte_hexdump (stdout,"Object:" , mbuf->data, +rte_hexdump (stdout,"Object:" , + rte_pktmbuf_mtod(mbuf, char *), mbuf->data_len);rte_pktmbuf_free (mbuf); }
config/common_linuxapp +1 -1
@@ -191,7 +191,7 @@ CONFIG_RTE_LIBRTE_IXGBE_DEBUG_DRIVER=n CONFIG_RTE_LIBRTE_IXGBE_PF_DISABLE_STRIP_CRC=n CONFIG_RTE_LIBRTE_IXGBE_RX_ALLOW_BULK_ALLOC=y CONFIG_RTE_LIBRTE_IXGBE_ALLOW_UNSUPPORTED_SFP=n -CONFIG_RTE_IXGBE_INC_VECTOR=y +CONFIG_RTE_IXGBE_INC_VECTOR=n CONFIG_RTE_IXGBE_RX_OLFLAGS_ENABLE=y #
examples/exception_path/main.c +2 -1
@@ -302,7 +302,8 @@ main_loop(__attribute__((unused)) void *arg)if (m ==NULL )continue ; - ret =read (tap_fd, m->data,MAX_PACKET_SZ ); + ret = read(tap_fd, rte_pktmbuf_mtod(m, void *), +MAX_PACKET_SZ ); lcore_stats[lcore_id].rx++;if (unlikely (ret <0 )) { FATAL_ERROR("Reading from %s interface failed",
examples/vhost/main.c +20 -17
@@ -1033,7 +1033,7 @@ virtio_dev_rx(struct virtio_net *dev, struct rte_mbuf **pkts, uint32_t count)/* Copy mbuf data to buffer */ rte_memcpy ((void *)(uintptr_t )buff_addr, - (const void *)buff->data, + rte_pktmbuf_mtod(buff, const void *),rte_pktmbuf_data_len (buff));PRINT_PACKET (dev, (uintptr_t )buff_addr,rte_pktmbuf_data_len (buff),0 );
@@ -1438,7 +1438,7 @@ link_vmdq(struct virtio_net *dev, struct rte_mbuf *m)int i, ret;/* Learn MAC address of guest device from packet */ - pkt_hdr = (struct ether_hdr *)m->data; + pkt_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *); dev_ll = ll_root_used;
@@ -1525,7 +1525,7 @@ virtio_tx_local(struct virtio_net *dev, struct rte_mbuf *m)struct ether_hdr *pkt_hdr;uint64_t ret =0 ; - pkt_hdr = (struct ether_hdr *)m->data; + pkt_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *);/*get the used devices list*/ dev_ll = ll_root_used;
@@ -1593,7 +1593,7 @@ virtio_tx_route(struct virtio_net* dev, struct rte_mbuf *m, struct rte_mempool *unsigned len, ret, offset =0 ;const uint16_t lcore_id =rte_lcore_id ();struct virtio_net_data_ll *dev_ll = ll_root_used; -struct ether_hdr *pkt_hdr = (struct ether_hdr *)m->data; + struct ether_hdr *pkt_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *);/*check if destination is local VM*/ if ((vm2vm_mode ==VM2VM_SOFTWARE ) && (virtio_tx_local (dev, m) ==0 ))
@@ -1652,18 +1652,21 @@ virtio_tx_route(struct virtio_net* dev, struct rte_mbuf *m, struct rte_mempool * mbuf->nb_segs = m->nb_segs;/* Copy ethernet header to mbuf. */ -rte_memcpy ((void *)mbuf->data, (const void *)m->data,ETH_HLEN ); + rte_memcpy(rte_pktmbuf_mtod(mbuf, void *), +rte_pktmbuf_mtod (m ,const void *), + ETH_HLEN);/* Setup vlan header. Bytes need to be re-ordered for network with htons()*/ - vlan_hdr = (struct vlan_ethhdr *) mbuf->data; + vlan_hdr = rte_pktmbuf_mtod(mbuf, struct vlan_ethhdr *); vlan_hdr->h_vlan_encapsulated_proto = vlan_hdr->h_vlan_proto; vlan_hdr->h_vlan_proto =htons (ETH_P_8021Q ); vlan_hdr->h_vlan_TCI =htons (vlan_tag);/* Copy the remaining packet contents to the mbuf. */ -rte_memcpy ((void *) ((uint8_t *)mbuf->data +VLAN_ETH_HLEN ), - (const void *) ((uint8_t *)m->data +ETH_HLEN ), (m->data_len -ETH_HLEN )); + rte_memcpy((void *)(rte_pktmbuf_mtod(mbuf, uint8_t *) + VLAN_ETH_HLEN), + (const void *)(rte_pktmbuf_mtod(m, uint8_t *) + ETH_HLEN), + (m->data_len -ETH_HLEN ));/* Copy the remaining segments for the whole packet. */ prev = mbuf;
@@ -1778,7 +1781,7 @@ virtio_dev_tx(struct virtio_net* dev, struct rte_mempool *mbuf_pool)/* Setup dummy mbuf. This is copied to a real mbuf if transmitted out the physical port. */ m.data_len = desc->len; m.pkt_len = desc->len; - m.data = (void *)(uintptr_t )buff_addr; + m.data_off =0 ;PRINT_PACKET (dev, (uintptr_t )buff_addr, desc->len,0 );
@@ -2333,7 +2336,7 @@ attach_rxmbuf_zcp(struct virtio_net *dev) } mbuf->buf_addr = (void *)(uintptr_t )(buff_addr -RTE_PKTMBUF_HEADROOM ); - mbuf->data = (void *)(uintptr_t )(buff_addr); + mbuf->data_off =RTE_PKTMBUF_HEADROOM ; mbuf->buf_physaddr = phys_addr -RTE_PKTMBUF_HEADROOM ; mbuf->data_len = desc->len;MBUF_HEADROOM_UINT32 (mbuf) = (uint32_t )desc_idx;
@@ -2370,7 +2373,7 @@ static inline void pktmbuf_detach_zcp(struct rte_mbuf *m) buf_ofs = (RTE_PKTMBUF_HEADROOM <= m->buf_len) ?RTE_PKTMBUF_HEADROOM : m->buf_len; - m->data = (char *) m->buf_addr + buf_ofs; + m->data_off = buf_ofs; m->data_len =0 ; }
@@ -2604,7 +2607,7 @@ virtio_tx_route_zcp(struct virtio_net *dev, struct rte_mbuf *m,unsigned len, ret, offset =0 ;struct vpool *vpool;struct virtio_net_data_ll *dev_ll = ll_root_used; -struct ether_hdr *pkt_hdr = (struct ether_hdr *)m->data; + struct ether_hdr *pkt_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *);uint16_t vlan_tag = (uint16_t )vlan_tags[(uint16_t )dev->device_fh];/*Add packet to the port tx queue*/
@@ -2681,11 +2684,11 @@ virtio_tx_route_zcp(struct virtio_net *dev, struct rte_mbuf *m, mbuf->pkt_len = mbuf->data_len;if (unlikely (need_copy)) {/* Copy the packet contents to the mbuf. */ -rte_memcpy ((void *)((uint8_t *)mbuf->data), - (const void *) ((uint8_t *)m->data), + rte_memcpy(rte_pktmbuf_mtod(mbuf, void *), +rte_pktmbuf_mtod (m ,void *), m->data_len); }else { - mbuf->data = m->data; + mbuf->data_off = m->data_off; mbuf->buf_physaddr = m->buf_physaddr; mbuf->buf_addr = m->buf_addr; }
@@ -2819,8 +2822,8 @@ virtio_dev_tx_zcp(struct virtio_net *dev) m.data_len = desc->len; m.nb_segs =1 ; m.next =NULL ; - m.data = (void *)(uintptr_t )buff_addr; - m.buf_addr = m.data; + m.data_off =0 ; + m.buf_addr = (void *)(uintptr_t )buff_addr; m.buf_physaddr = phys_addr;/*
examples/vhost_xen/main.c +8 -6
@@ -808,7 +808,7 @@ virtio_tx_local(struct virtio_net *dev, struct rte_mbuf *m)struct ether_hdr *pkt_hdr;uint64_t ret =0 ; - pkt_hdr = (struct ether_hdr *)m->data; + pkt_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *);/*get the used devices list*/ dev_ll = ll_root_used;
@@ -883,18 +883,20 @@ virtio_tx_route(struct virtio_net* dev, struct rte_mbuf *m, struct rte_mempool * mbuf->pkt_len = mbuf->data_len;/* Copy ethernet header to mbuf. */ -rte_memcpy ((void *)mbuf->data, (const void *)m->data,ETH_HLEN ); + rte_memcpy(rte_pktmbuf_mtod(mbuf, void*), + rte_pktmbuf_mtod(m, const void*), ETH_HLEN);/* Setup vlan header. Bytes need to be re-ordered for network with htons()*/ - vlan_hdr = (struct vlan_ethhdr *) mbuf->data; + vlan_hdr = rte_pktmbuf_mtod(mbuf, struct vlan_ethhdr *); vlan_hdr->h_vlan_encapsulated_proto = vlan_hdr->h_vlan_proto; vlan_hdr->h_vlan_proto =htons (ETH_P_8021Q ); vlan_hdr->h_vlan_TCI =htons (vlan_tag);/* Copy the remaining packet contents to the mbuf. */ -rte_memcpy ((void *) ((uint8_t *)mbuf->data +VLAN_ETH_HLEN ), - (const void *) ((uint8_t *)m->data +ETH_HLEN ), (m->data_len -ETH_HLEN )); + rte_memcpy((void *)(rte_pktmbuf_mtod(mbuf, uint8_t *) + VLAN_ETH_HLEN), + (const void *)(rte_pktmbuf_mtod(m, uint8_t *) + ETH_HLEN), + (m->data_len -ETH_HLEN )); tx_q->m_table[len] = mbuf; len++;if (enable_stats) {
@@ -981,7 +983,7 @@ virtio_dev_tx(struct virtio_net* dev, struct rte_mempool *mbuf_pool)/* Setup dummy mbuf. This is copied to a real mbuf if transmitted out the physical port. */ m.data_len = desc->len; - m.data = (void *)(uintptr_t )buff_addr; + m.data_off =0 ; m.nb_segs =1 ;virtio_tx_route (dev, &m, mbuf_pool,0 );
lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h +5 -5
@@ -110,13 +110,13 @@ struct rte_kni_fifo {struct rte_kni_mbuf {void *pool;void *buf_addr; -char pad0[14 ]; -uint16_t ol_flags;/**< Offload features. */ +char pad0[16 ];void *next; -void *data;/**< Start address of data in segment buffer. */ +uint16_t data_off;/**< Start address of data in segment buffer. */ uint16_t data_len;/**< Amount of data in segment buffer. */ -char pad2[2 ]; -uint16_t pkt_len;/**< Total pkt len: sum of all segment data_len. */ +uint32_t pkt_len;/**< Total pkt len: sum of all segment data_len. */ +char pad2[4 ]; +uint16_t ol_flags;/**< Offload features. */ } __attribute__((__aligned__ (64 )));/*
lib/librte_eal/linuxapp/kni/kni_net.c +11 -7
@@ -162,7 +162,8 @@ kni_net_rx_normal(struct kni_dev *kni)for (i =0 ; i < num; i++) { kva = (void *)va[i] - kni->mbuf_va + kni->mbuf_kva; len = kva->data_len; - data_kva = kva->data - kni->mbuf_va + kni->mbuf_kva; + data_kva = kva->buf_addr + kva->data_off - kni->mbuf_va + + kni->mbuf_kva; skb =dev_alloc_skb (len +2 );if (!skb) {
@@ -246,13 +247,14 @@ kni_net_rx_lo_fifo(struct kni_dev *kni)for (i =0 ; i < num; i++) { kva = (void *)va[i] - kni->mbuf_va + kni->mbuf_kva; len = kva->pkt_len; - data_kva = kva->data - kni->mbuf_va + - kni->mbuf_kva; + data_kva = kva->buf_addr + kva->data_off - + kni->mbuf_va + kni->mbuf_kva; alloc_kva = (void *)alloc_va[i] - kni->mbuf_va + kni->mbuf_kva; - alloc_data_kva = alloc_kva->data - kni->mbuf_va + - kni->mbuf_kva; + alloc_data_kva = alloc_kva->buf_addr + + alloc_kva->data_off - kni->mbuf_va + + kni->mbuf_kva;memcpy (alloc_data_kva, data_kva, len); alloc_kva->pkt_len = len; alloc_kva->data_len = len;
@@ -321,7 +323,8 @@ kni_net_rx_lo_fifo_skb(struct kni_dev *kni)for (i =0 ; i < num; i++) { kva = (void *)va[i] - kni->mbuf_va + kni->mbuf_kva; len = kva->data_len; - data_kva = kva->data - kni->mbuf_va + kni->mbuf_kva; + data_kva = kva->buf_addr + kva->data_off - kni->mbuf_va + + kni->mbuf_kva; skb =dev_alloc_skb (len +2 );if (skb ==NULL )
@@ -423,7 +426,8 @@ kni_net_tx(struct sk_buff *skb, struct net_device *dev)void *data_kva; pkt_kva = (void *)pkt_va - kni->mbuf_va + kni->mbuf_kva; - data_kva = pkt_kva->data - kni->mbuf_va + kni->mbuf_kva; + data_kva = pkt_kva->buf_addr + pkt_kva->data_off - kni->mbuf_va + + kni->mbuf_kva; len = skb->len;memcpy (data_kva, skb->data, len);
lib/librte_ip_frag/rte_ipv4_fragmentation.c +3 -3
@@ -109,7 +109,7 @@ rte_ipv4_fragment_packet(struct rte_mbuf *pkt_in,/* Fragment size should be a multiply of 8. */ IP_FRAG_ASSERT ((frag_size &IPV4_HDR_FO_MASK ) ==0 ); - in_hdr = (struct ipv4_hdr *) pkt_in->data; + in_hdr = rte_pktmbuf_mtod(pkt_in, struct ipv4_hdr *); flag_offset =rte_cpu_to_be_16 (in_hdr->fragment_offset);/* If Don't Fragment flag is set */
@@ -165,7 +165,7 @@ rte_ipv4_fragment_packet(struct rte_mbuf *pkt_in,if (len > (in_seg->data_len - in_seg_data_pos)) { len = in_seg->data_len - in_seg_data_pos; } - out_seg->data = (char *) in_seg->data + (uint16_t )in_seg_data_pos; + out_seg->data_off = in_seg->data_off + in_seg_data_pos; out_seg->data_len = (uint16_t )len; out_pkt->pkt_len = (uint16_t )(len + out_pkt->pkt_len);
@@ -188,7 +188,7 @@ rte_ipv4_fragment_packet(struct rte_mbuf *pkt_in,/* Build the IP header */ - out_hdr = (struct ipv4_hdr *) out_pkt->data; + out_hdr = rte_pktmbuf_mtod(out_pkt, struct ipv4_hdr *); __fill_ipv4hdr_frag(out_hdr, in_hdr, (uint16_t)out_pkt->pkt_len,
lib/librte_ip_frag/rte_ipv6_fragmentation.c +3 -3
@@ -125,7 +125,7 @@ rte_ipv6_fragment_packet(struct rte_mbuf *pkt_in, (uint16_t)(pkt_in->pkt_len - sizeof (struct ipv6_hdr)))) return (-EINVAL); - in_hdr = (struct ipv6_hdr *) pkt_in->data; + in_hdr = rte_pktmbuf_mtod(pkt_in, struct ipv6_hdr *); in_seg = pkt_in; in_seg_data_pos =sizeof (struct ipv6_hdr );
@@ -171,7 +171,7 @@ rte_ipv6_fragment_packet(struct rte_mbuf *pkt_in,if (len > (in_seg->data_len - in_seg_data_pos)) { len = in_seg->data_len - in_seg_data_pos; } - out_seg->data = (char *) in_seg->data + (uint16_t ) in_seg_data_pos; + out_seg->data_off = in_seg->data_off + in_seg_data_pos; out_seg->data_len = (uint16_t )len; out_pkt->pkt_len = (uint16_t )(len + out_pkt->pkt_len);
@@ -196,7 +196,7 @@ rte_ipv6_fragment_packet(struct rte_mbuf *pkt_in,/* Build the IP header */ - out_hdr = (struct ipv6_hdr *) out_pkt->data; + out_hdr = rte_pktmbuf_mtod(out_pkt, struct ipv6_hdr *); __fill_ipv6hdr_frag(out_hdr, in_hdr, (uint16_t) out_pkt->pkt_len - sizeof(struct ipv6_hdr),
lib/librte_mbuf/rte_mbuf.c +3 -3
@@ -117,7 +117,7 @@ rte_pktmbuf_init(struct rte_mempool *mp, m->buf_len = (uint16_t )buf_len;/* keep some headroom between start of buffer and data */ - m->data = (char *) m->buf_addr +RTE_MIN (RTE_PKTMBUF_HEADROOM , m->buf_len); + m->data_off =RTE_MIN (RTE_PKTMBUF_HEADROOM , (uint16_t )m->buf_len);/* init some constant fields */ m->pool = mp;
@@ -183,12 +183,12 @@ rte_pktmbuf_dump(FILE *f, const struct rte_mbuf *m, unsigned dump_len)__rte_mbuf_sanity_check (m,0 );fprintf (f," segment at 0x%p, data=0x%p, data_len=%u\n" , - m, m->data, (unsigned )m->data_len); + m, rte_pktmbuf_mtod(m, void *), (unsigned)m->data_len); len = dump_len;if (len > m->data_len) len = m->data_len;if (len !=0 ) -rte_hexdump (f,NULL , m->data, len); + rte_hexdump(f, NULL, rte_pktmbuf_mtod(m, void *), len); dump_len -= len; m = m->next; nb_segs --;
lib/librte_mbuf/rte_mbuf.h +19 -23
135 lines not on this page · raw message
lib/librte_pmd_bond/rte_eth_bond_pmd.c +2 -2
@@ -198,14 +198,14 @@ xmit_slave_hash(const struct rte_mbuf *buf, uint8_t slave_count, uint8_t policy)switch (policy) {case BALANCE_XMIT_POLICY_LAYER2 : - eth_hdr = (struct ether_hdr *)buf->data; + eth_hdr = rte_pktmbuf_mtod(buf, struct ether_hdr *); hash =ether_hash (eth_hdr); hash ^= hash >>8 ;return hash % slave_count;case BALANCE_XMIT_POLICY_LAYER23 : - eth_hdr = (struct ether_hdr *)buf->data; + eth_hdr = rte_pktmbuf_mtod(buf, struct ether_hdr *);if (buf->ol_flags &PKT_RX_VLAN_PKT ) eth_offset =sizeof (struct ether_hdr ) +sizeof (struct vlan_hdr );
lib/librte_pmd_e1000/em_rxtx.c +6 -6
@@ -90,8 +90,7 @@ rte_rxmbuf_alloc(struct rte_mempool *mp) }#define RTE_MBUF_DATA_DMA_ADDR (mb) \ - (uint64_t) ((mb)->buf_physaddr + \ - (uint64_t) ((char *)((mb)->data) - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off)#define RTE_MBUF_DATA_DMA_ADDR_DEFAULT (mb) \ (uint64_t) ((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM)
@@ -793,8 +792,8 @@ eth_em_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,*/ pkt_len = (uint16_t ) (rte_le_to_cpu_16 (rxd.length) - rxq->crc_len); - rxm->data = (char *) rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; -rte_packet_prefetch (rxm->data); + rxm->data_off =RTE_PKTMBUF_HEADROOM ; +rte_packet_prefetch ((char *)rxm->buf_addr + rxm->data_off); rxm->nb_segs =1 ; rxm->next =NULL ; rxm->pkt_len = pkt_len;
@@ -963,7 +962,7 @@ eth_em_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,*/ data_len =rte_le_to_cpu_16 (rxd.length); rxm->data_len = data_len; - rxm->data = (char *) rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ;/* * If this is the first buffer of the received packet,
@@ -1035,7 +1034,8 @@ eth_em_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rxm->vlan_tci =rte_le_to_cpu_16 (rxd.special);/* Prefetch data of first segment, if configured to do so. */ -rte_packet_prefetch (first_seg->data); +rte_packet_prefetch ((char *)first_seg->buf_addr + + first_seg->data_off);/* * Store the mbuf address into the next entry of the array
lib/librte_pmd_e1000/igb_rxtx.c +6 -7
@@ -95,9 +95,7 @@ rte_rxmbuf_alloc(struct rte_mempool *mp) }#define RTE_MBUF_DATA_DMA_ADDR (mb) \ - (uint64_t) ((mb)->buf_physaddr + \ - (uint64_t) ((char *)((mb)->data) - \ - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off)#define RTE_MBUF_DATA_DMA_ADDR_DEFAULT (mb) \ (uint64_t) ((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM)
@@ -774,8 +772,8 @@ eth_igb_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,*/ pkt_len = (uint16_t ) (rte_le_to_cpu_16 (rxd.wb.upper.length) - rxq->crc_len); - rxm->data = (char *) rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; -rte_packet_prefetch (rxm->data); + rxm->data_off =RTE_PKTMBUF_HEADROOM ; +rte_packet_prefetch ((char *)rxm->buf_addr + rxm->data_off); rxm->nb_segs =1 ; rxm->next =NULL ; rxm->pkt_len = pkt_len;
@@ -950,7 +948,7 @@ eth_igb_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,*/ data_len =rte_le_to_cpu_16 (rxd.wb.upper.length); rxm->data_len = data_len; - rxm->data = (char *) rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ;/* * If this is the first buffer of the received packet,
@@ -1031,7 +1029,8 @@ eth_igb_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, first_seg->ol_flags = pkt_flags;/* Prefetch data of first segment, if configured to do so. */ -rte_packet_prefetch (first_seg->data); +rte_packet_prefetch ((char *)first_seg->buf_addr + + first_seg->data_off);/* * Store the mbuf address into the next entry of the array
lib/librte_pmd_i40e/i40e_rxtx.c +8 -9
@@ -78,9 +78,7 @@ (uint64_t) ((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM)#define RTE_MBUF_DATA_DMA_ADDR (mb) \ - ((uint64_t)((mb)->buf_physaddr + \ - (uint64_t)((char *)((mb)->data) - \ - (char *)(mb)->buf_addr))) + ((uint64_t)((mb)->buf_physaddr + (mb)->data_off))static const struct rte_memzone * i40e_ring_dma_zone_reserve(struct rte_eth_dev *dev,
@@ -685,7 +683,7 @@ i40e_rx_alloc_bufs(struct i40e_rx_queue *rxq) mb = rxep[i].mbuf;rte_mbuf_refcnt_set (mb,1 ); mb->next =NULL ; - mb->data = (char *)mb->buf_addr +RTE_PKTMBUF_HEADROOM ; + mb->data_off =RTE_PKTMBUF_HEADROOM ; mb->nb_segs =1 ; mb->port = rxq->port_id; dma_addr = rte_cpu_to_le_64(\
@@ -842,8 +840,8 @@ i40e_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) rx_packet_len = ((qword1 &I40E_RXD_QW1_LENGTH_PBUF_MASK ) >>I40E_RXD_QW1_LENGTH_PBUF_SHIFT ) - rxq->crc_len; - rxm->data = (char *)rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; -rte_prefetch0 (rxm->data); + rxm->data_off =RTE_PKTMBUF_HEADROOM ; +rte_prefetch0 (RTE_PTR_ADD (rxm->buf_addr,RTE_PKTMBUF_HEADROOM )); rxm->nb_segs =1 ; rxm->next =NULL ; rxm->pkt_len = rx_packet_len;
@@ -946,7 +944,7 @@ i40e_recv_scattered_pkts(void *rx_queue, rx_packet_len = (qword1 &I40E_RXD_QW1_LENGTH_PBUF_MASK ) >>I40E_RXD_QW1_LENGTH_PBUF_SHIFT ; rxm->data_len = rx_packet_len; - rxm->data = (char *)rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ;/** * If this is the first buffer of the received packet, set the
@@ -1015,7 +1013,8 @@ i40e_recv_scattered_pkts(void *rx_queue,rte_le_to_cpu_32 (rxd.wb.qword0.hi_dword.rss);/* Prefetch data of first segment, if configured to do so. */ -rte_prefetch0 (first_seg->data); +rte_prefetch0 (RTE_PTR_ADD (first_seg->buf_addr, + first_seg->data_off)); rx_pkts[nb_rx++] = first_seg; first_seg =NULL ; }
@@ -2131,7 +2130,7 @@ i40e_alloc_rx_queue_mbufs(struct i40e_rx_queue *rxq)rte_mbuf_refcnt_set (mbuf,1 ); mbuf->next =NULL ; - mbuf->data = (char *)mbuf->buf_addr +RTE_PKTMBUF_HEADROOM ; + mbuf->data_off =RTE_PKTMBUF_HEADROOM ; mbuf->nb_segs =1 ; mbuf->port = rxq->port_id;
lib/librte_pmd_ixgbe/ixgbe_rxtx.c +7 -6
@@ -997,7 +997,7 @@ ixgbe_rx_alloc_bufs(struct igb_rx_queue *rxq) mb = rxep[i].mbuf;rte_mbuf_refcnt_set (mb,1 ); mb->next =NULL ; - mb->data = (char *)mb->buf_addr +RTE_PKTMBUF_HEADROOM ; + mb->data_off =RTE_PKTMBUF_HEADROOM ; mb->nb_segs =1 ; mb->port = rxq->port_id;
@@ -1248,8 +1248,8 @@ ixgbe_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,*/ pkt_len = (uint16_t ) (rte_le_to_cpu_16 (rxd.wb.upper.length) - rxq->crc_len); - rxm->data = (char *) rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; -rte_packet_prefetch (rxm->data); + rxm->data_off =RTE_PKTMBUF_HEADROOM ; +rte_packet_prefetch ((char *)rxm->buf_addr + rxm->data_off); rxm->nb_segs =1 ; rxm->next =NULL ; rxm->pkt_len = pkt_len;
@@ -1431,7 +1431,7 @@ ixgbe_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,*/ data_len =rte_le_to_cpu_16 (rxd.wb.upper.length); rxm->data_len = data_len; - rxm->data = (char *) rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ;/* * If this is the first buffer of the received packet,
@@ -1522,7 +1522,8 @@ ixgbe_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, }/* Prefetch data of first segment, if configured to do so. */ -rte_packet_prefetch (first_seg->data); +rte_packet_prefetch ((char *)first_seg->buf_addr + + first_seg->data_off);/* * Store the mbuf address into the next entry of the array
@@ -3212,7 +3213,7 @@ ixgbe_alloc_rx_queue_mbufs(struct igb_rx_queue *rxq)rte_mbuf_refcnt_set (mbuf,1 ); mbuf->next =NULL ; - mbuf->data = (char *)mbuf->buf_addr +RTE_PKTMBUF_HEADROOM ; + mbuf->data_off =RTE_PKTMBUF_HEADROOM ; mbuf->nb_segs =1 ; mbuf->port = rxq->port_id;
lib/librte_pmd_ixgbe/ixgbe_rxtx.h +1 -2
@@ -47,8 +47,7 @@#endif #define RTE_MBUF_DATA_DMA_ADDR (mb) \ - (uint64_t) ((mb)->buf_physaddr + (uint64_t)((char *)((mb)->data) - \ - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off)#define RTE_MBUF_DATA_DMA_ADDR_DEFAULT (mb) \ (uint64_t) ((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM)
lib/librte_pmd_pcap/rte_eth_pcap.c +6 -3
@@ -151,7 +151,8 @@ eth_pcap_rx(void *queue,if (header.len <= buf_size) {/* pcap packet will fit in the mbuf, go ahead and copy */ -rte_memcpy (mbuf->data, packet, header.len); + rte_memcpy(rte_pktmbuf_mtod(mbuf, void *), packet, + header.len); mbuf->data_len = (uint16_t )header.len; mbuf->pkt_len = mbuf->data_len; bufs[num_rx] = mbuf;
@@ -202,7 +203,8 @@ eth_pcap_tx_dumper(void *queue,calculate_timestamp (&header.ts); header.len = mbuf->data_len; header.caplen = header.len; -pcap_dump ((u_char *) dumper_q->dumper, &header, mbuf->data); +pcap_dump ((u_char *)dumper_q->dumper, &header, + rte_pktmbuf_mtod(mbuf, void*));rte_pktmbuf_free (mbuf); num_tx++; }
@@ -237,7 +239,8 @@ eth_pcap_tx(void *queue,for (i =0 ; i < nb_pkts; i++) { mbuf = bufs[i]; - ret =pcap_sendpacket (tx_queue->pcap, (u_char *) mbuf->data, + ret =pcap_sendpacket (tx_queue->pcap, + rte_pktmbuf_mtod(mbuf, u_char *), mbuf->data_len);if (unlikely (ret !=0 ))break ;
lib/librte_pmd_virtio/virtio_rxtx.c +4 -6
@@ -118,7 +118,7 @@ virtqueue_dequeue_burst_rx(struct virtqueue *vq, struct rte_mbuf **rx_pkts, }rte_prefetch0 (cookie); -rte_packet_prefetch (cookie->data); +rte_packet_prefetch (rte_pktmbuf_mtod(cookie ,void *)); rx_pkts[i] = cookie; vq->vq_used_cons_idx++;vq_ring_free_chain (vq, desc_idx);
@@ -480,7 +480,7 @@ virtio_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) } rxm->port = rxvq->port_id; - rxm->data = (char *)rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ; rxm->nb_segs =1 ; rxm->next =NULL ;
@@ -584,7 +584,7 @@ virtio_recv_mergeable_pkts(void *rx_queue,if (seg_num ==0 ) seg_num =1 ; - rxm->data = (char *)rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ; rxm->nb_segs = seg_num; rxm->next =NULL ; rxm->pkt_len = (uint32_t )(len[0 ] - hdr_size);
@@ -622,9 +622,7 @@ virtio_recv_mergeable_pkts(void *rx_queue,while (extra_idx < rcv_cnt) { rxm = rcv_pkts[extra_idx]; - rxm->data = - (char *)rxm->buf_addr + -RTE_PKTMBUF_HEADROOM - hdr_size; + rxm->data_off =RTE_PKTMBUF_HEADROOM - hdr_size; rxm->next =NULL ; rxm->pkt_len = (uint32_t )(len[extra_idx]); rxm->data_len = (uint16_t )(len[extra_idx]);
lib/librte_pmd_virtio/virtqueue.h +1 -2
@@ -59,8 +59,7 @@#define VIRTQUEUE_MAX_NAME_SZ 32#define RTE_MBUF_DATA_DMA_ADDR (mb) \ - (uint64_t) ((mb)->buf_physaddr + (uint64_t)((char *)((mb)->data) - \ - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off)#define VTNET_SQ_RQ_QUEUE_IDX 0#define VTNET_SQ_TQ_QUEUE_IDX 1
lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c +2 -3
@@ -79,8 +79,7 @@#define RTE_MBUF_DATA_DMA_ADDR (mb) \ - (uint64_t) ((mb)->buf_physaddr + (uint64_t)((char *)((mb)->data) - \ - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off)#define RTE_MBUF_DATA_DMA_ADDR_DEFAULT (mb) \ (uint64_t) ((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM)
@@ -565,7 +564,7 @@ vmxnet3_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) rxm->data_len = (uint16_t )rcd->len; rxm->port = rxq->port_id; rxm->vlan_tci =0 ; - rxm->data = (char *)rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ; rx_pkts[nb_rx++] = rxm;
lib/librte_pmd_xenvirt/rte_eth_xenvirt.c +1 -1
@@ -110,7 +110,7 @@ eth_xenvirt_rx(void *q, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) rxm = rx_pkts[i];PMD_RX_LOG (DEBUG ,"packet len:%d\n" , len[i]); rxm->next =NULL ; - rxm->data = (char *)rxm->buf_addr +RTE_PKTMBUF_HEADROOM ; + rxm->data_off =RTE_PKTMBUF_HEADROOM ; rxm->data_len = (uint16_t )(len[i] -sizeof (struct virtio_net_hdr )); rxm->nb_segs =1 ; rxm->port = pi->port_id;
Signature · 1 line
1.9.3
PATCH v2 2/13 mbuf: reorder fields by time of use
In reply to a message not in the archive* Reorder the fields in the mbuf so that we have fields that are used together side-by-side in the structure. This means that we have a contiguous block of 8-bytes in the mbuf which are used to reset an mbuf of descriptor rearm, and a block of 16-bytes of data (excluding flags) which are set on RX from the received packet descriptor.
* Use dummy fields as appropriate to ensure alignment or to reserve gaps for later field additions.
* Place most items which are not used by fast-path RX separately at the end of the structure so they can later be moved to a separate cache line. [The l2/l3 length fields are not moved at this stage as doing so will cause overflow to the next cache line].
Updated in V2:
* Updated the KNI buffer structure to match that of new mbuf
* Cleaned up commit message typos.
--- .../linuxapp/eal/include/exec-env/rte_kni_common.h | 12 ++++++----- lib/librte_mbuf/rte_mbuf.h | 25 ++++++++++++---------- 2 files changed, 21 insertions(+), 16 deletions(-)
lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h +7 -5
@@ -108,15 +108,17 @@ struct rte_kni_fifo {* Padding is necessary to assure the offsets of these fields */ struct rte_kni_mbuf { -void *pool;void *buf_addr; -char pad0[16 ]; -void *next; +char pad0[10 ];uint16_t data_off;/**< Start address of data in segment buffer. */ +char pad1[4 ]; +uint16_t ol_flags;/**< Offload features. */ +char pad2[8 ];uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segment data_len. */ -char pad2[4 ]; -uint16_t ol_flags;/**< Offload features. */ +char pad3[8 ]; +void *pool; +void *next; } __attribute__((__aligned__ (64 )));/*
lib/librte_mbuf/rte_mbuf.h +14 -11
@@ -115,16 +115,12 @@ extern "C" {* The generic rte_mbuf, containing a packet mbuf. */ struct rte_mbuf { -struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */ void *buf_addr;/**< Virtual address of segment buffer. */ phys_addr_t buf_physaddr;/**< Physical address of segment buffer. */ -uint16_t buf_len;/**< Length of segment buffer. */ -/* valid for any segment */ -struct rte_mbuf *next;/**< Next segment of scattered packet. */ +/* next 8 bytes are initialised on RX descriptor rearm */ +uint16_t buf_len;/**< Length of segment buffer. */ uint16_t data_off; -uint16_t data_len;/**< Amount of data in segment buffer. */ -uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ #ifdef RTE_MBUF_REFCNT /**
@@ -142,14 +138,17 @@ struct rte_mbuf {#else uint16_t refcnt_reserved;/**< Do not use this field */ #endif -uint16_t reserved;/**< Unused field. Required for padding */ -uint16_t ol_flags;/**< Offload features. */ - -/* these fields are valid for first segment only */ uint8_t nb_segs;/**< Number of segments. */ uint8_t port;/**< Input port. */ -/* offload features, valid for first segment only */ +uint16_t ol_flags;/**< Offload features. */ +uint16_t reserved0;/**< Unused field. Required for padding */ +uint32_t reserved1;/**< Unused field. Required for padding */ + +/* remaining bytes are set on RX when pulling packet from descriptor */ +uint16_t reserved2;/**< Unused field. Required for padding */ +uint16_t data_len;/**< Amount of data in segment buffer. */ +uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ union {uint16_t l2_l3_len;/**< combined l2/l3 lengths as single var */ struct {
@@ -167,6 +166,10 @@ struct rte_mbuf {uint32_t sched;/**< Hierarchical scheduler */ } hash;/**< hash information */ +/* fields only used in slow path or on TX */ +struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */ +struct rte_mbuf *next;/**< Next segment of scattered packet. */ +union {uint8_t metadata[0 ];uint16_t metadata16[0 ];
Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): 1410441347-22840-3-git-send-email-bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v2 02/13] mbuf: reorder fields by time of use
Hi Bruce,
Quoted 117 lines
-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Bruce Richardson
Sent: Thursday, September 11, 2014 9:16 PM
To: [email protected]
Subject: [dpdk-dev] [PATCH v2 02/13] mbuf: reorder fields by time of use* Reorder the fields in the mbuf so that we have fields that are used together side-by-side in the structure. This means that we have a contiguous block of 8-bytes in the mbuf which are used to reset an mbuf of descriptor rearm, and a block of 16-bytes of data (excluding flags) which are set on RX from the received packet descriptor.
* Use dummy fields as appropriate to ensure alignment or to reserve gaps for later field additions.
* Place most items which are not used by fast-path RX separately at the end of the structure so they can later be moved to a separate cache line. [The l2/l3 length fields are not moved at this stage as doing so will cause overflow to the next cache line].Updated in V2:
* Updated the KNI buffer structure to match that of new mbuf
* Cleaned up commit message typos.Signed-off-by Bruce Richardson <[email protected]>--- .../linuxapp/eal/include/exec-env/rte_kni_common.h | 12 ++++++----- lib/librte_mbuf/rte_mbuf.h | 25 ++++++++++++---------- 2 files changed, 21 insertions(+), 16 deletions(-)lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h +7
-5@@ -108,15 +108,17 @@ struct rte_kni_fifo {* Padding is necessary to assure the offsets of these fields */ struct rte_kni_mbuf { -void *pool;void *buf_addr; -char pad0[16 ]; -void *next; +char pad0[10 ];uint16_t data_off;/**< Start address of data in segment buffer. */ +char pad1[4 ]; +uint16_t ol_flags;/**< Offload features. */ +char pad2[8 ];uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segment data_len. */ -char pad2[4 ]; -uint16_t ol_flags;/**< Offload features. */ +char pad3[8 ]; +void *pool; +void *next; } __attribute__((__aligned__ (64 )));/*
lib/librte_mbuf/rte_mbuf.h index renamed from lib/librte_mbuf/rte_mbuf.h +0
-071080e5..413a5a1 100644
lib/librte_mbuf/rte_mbuf.h +10
-11@@ -115,16 +115,12 @@ extern "C" {* The generic rte_mbuf, containing a packet mbuf. */ struct rte_mbuf { -struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */ void *buf_addr;/**< Virtual address of segment buffer. */ phys_addr_t buf_physaddr;/**< Physical address of segment buffer. */ -uint16_t buf_len;/**< Length of segment buffer. */ -/* valid for any segment */ -struct rte_mbuf *next;/**< Next segment of scattered packet. */ +/* next 8 bytes are initialised on RX descriptor rearm */ +uint16_t buf_len;/**< Length of segment buffer. */ uint16_t data_off; -uint16_t data_len;/**< Amount of data in segment buffer. */ -uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ #ifdef RTE_MBUF_REFCNT /** @@ -142,14 +138,17 @@ struct rte_mbuf {#else uint16_t refcnt_reserved; /**< Do not use this field */ #endif -uint16_t reserved;/**< Unused field. Required for padding */ -uint16_t ol_flags;/**< Offload features. */ - -/* these fields are valid for first segment only */ uint8_t nb_segs;/**< Number of segments. */ uint8_t port;/**< Input port. */ -/* offload features, valid for first segment only */ +uint16_t ol_flags;/**< Offload features. */ +uint16_t reserved0;/**< Unused field. Required for padding */ +uint32_t reserved1;/**< Unused field. Required for padding */ + +/* remaining bytes are set on RX when pulling packet from descriptor */ +uint16_t reserved2;/**< Unused field. Required for padding */ +uint16_t data_len;/**< Amount of data in segment buffer. */ +uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ union {uint16_t l2_l3_len;/**< combined l2/l3 lengths as single var */ lib/librte_mbuf/rte_mbuf.h +4
-1struct {@@ -167,6 +166,10 @@ struct rte_mbuf {uint32_t sched;/**< Hierarchical scheduler */ } hash;/**< hash information */ +/* fields only used in slow path or on TX */ +struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */ +struct rte_mbuf *next;/**< Next segment of scattered packet. */ +union {uint8_t metadata[0 ];uint16_t metadata16[0 ]; --
Quoted 1 line
1.9.3
It seems that you removed the inner_l3_len and inner_l2_len fields from the rte_mbuf.h structure, what is the reason?
/Jijiang Liu
A message that is not in the archive (sent off-list or not yet received): 1ED644BD7E0A5F4091CF203DAFB8E4CC01D701BA-0J0gbvR4kThpB2pF5aRoyrfspsVTdybXVpNB7YpNyf8@public.gmane.org
Re: [PATCH v2 02/13] mbuf: reorder fields by time of use
Quoted 132 lines
-----Original Message-----
From: Liu, Jijiang
Sent: Monday, September 15, 2014 8:12 AM
To: Richardson, Bruce; [email protected]
Subject: RE: [dpdk-dev] [PATCH v2 02/13] mbuf: reorder fields by time of useHi Bruce,
Quoted 63 lines
-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Bruce Richardson
Sent: Thursday, September 11, 2014 9:16 PM
To: [email protected]
Subject: [dpdk-dev] [PATCH v2 02/13] mbuf: reorder fields by time of use* Reorder the fields in the mbuf so that we have fields that are used together side-by-side in the structure. This means that we have a contiguous block of 8-bytes in the mbuf which are used to reset an mbuf of descriptor rearm, and a block of 16-bytes of data (excluding flags) which are set on RX from the received packet descriptor.
* Use dummy fields as appropriate to ensure alignment or to reserve gaps for later field additions.
* Place most items which are not used by fast-path RX separately at the end of the structure so they can later be moved to a separate cache line. [The l2/l3 length fields are not moved at this stage as doing so will cause overflow to the next cache line].Updated in V2:
* Updated the KNI buffer structure to match that of new mbuf
* Cleaned up commit message typos.Signed-off-by Bruce Richardson <[email protected]>--- .../linuxapp/eal/include/exec-env/rte_kni_common.h | 12 ++++++----- lib/librte_mbuf/rte_mbuf.h | 25 ++++++++++++---------- 2 files changed, 21 insertions(+), 16 deletions(-)lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h +7
-5@@ -108,15 +108,17 @@ struct rte_kni_fifo {* Padding is necessary to assure the offsets of these fields */ struct rte_kni_mbuf { -void *pool;void *buf_addr; -char pad0[16 ]; -void *next; +char pad0[10 ];uint16_t data_off;/**< Start address of data in segment buffer. */ +char pad1[4 ]; +uint16_t ol_flags;/**< Offload features. */ +char pad2[8 ];uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segment data_len. */ -char pad2[4 ]; -uint16_t ol_flags;/**< Offload features. */ +char pad3[8 ]; +void *pool; +void *next; } __attribute__((__aligned__ (64 )));/*
lib/librte_mbuf/rte_mbuf.h index renamed from lib/librte_mbuf/rte_mbuf.h +0
-071080e5..413a5a1 100644
lib/librte_mbuf/rte_mbuf.h extern "C" { · line 118
Quoted 4 lines
lib/librte_mbuf/rte_mbuf.h +0
-1@@ -115,16 +115,12 @@ extern "C" {* The generic rte_mbuf, containing a packet mbuf. */ struct rte_mbuf { -struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */
Quoted 39 lines
lib/librte_mbuf/rte_mbuf.h +10
-10void *buf_addr;/**< Virtual address of segment buffer. */ phys_addr_t buf_physaddr;/**< Physical address of segment buffer. */ -uint16_t buf_len;/**< Length of segment buffer. */ -/* valid for any segment */ -struct rte_mbuf *next;/**< Next segment of scattered packet. */ +/* next 8 bytes are initialised on RX descriptor rearm */ +uint16_t buf_len;/**< Length of segment buffer. */ uint16_t data_off; -uint16_t data_len;/**< Amount of data in segment buffer. */ -uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ #ifdef RTE_MBUF_REFCNT /** @@ -142,14 +138,17 @@ struct rte_mbuf {#else uint16_t refcnt_reserved; /**< Do not use this field */ #endif -uint16_t reserved;/**< Unused field. Required for padding */ -uint16_t ol_flags;/**< Offload features. */ - -/* these fields are valid for first segment only */ uint8_t nb_segs;/**< Number of segments. */ uint8_t port;/**< Input port. */ -/* offload features, valid for first segment only */ +uint16_t ol_flags;/**< Offload features. */ +uint16_t reserved0;/**< Unused field. Required for padding */ +uint32_t reserved1;/**< Unused field. Required for padding */ + +/* remaining bytes are set on RX when pulling packet from descriptor */ +uint16_t reserved2;/**< Unused field. Required for padding */ +uint16_t data_len;/**< Amount of data in segment buffer. */ +uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ union {uint16_t l2_l3_len;/**< combined l2/l3 lengths as single var */ lib/librte_mbuf/rte_mbuf.h +0
-0struct {lib/librte_mbuf/rte_mbuf.h struct rte_mbuf { · line 170
Quoted 5 lines
lib/librte_mbuf/rte_mbuf.h +2
-0@@ -167,6 +166,10 @@ struct rte_mbuf {uint32_t sched;/**< Hierarchical scheduler */ } hash;/**< hash information */ +/* fields only used in slow path or on TX */ +struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */
Quoted 7 lines
lib/librte_mbuf/rte_mbuf.h +2
-1+struct rte_mbuf *next;/**< Next segment of scattered packet. */ +union {uint8_t metadata[0 ];uint16_t metadata16[0 ]; --1.9.3
It seems that you removed the
inner_l3_lenandinner_l2_lenfields from therte_mbuf.hstructure, what is the reason?
Quoted 1 line
/Jijiang Liu
No, those fields were never added to the mbuf, they were simply proposed as a new field addition in an RFC patch set. The adding of new fields to the mbuf is beyond the scope of the two patch sets already submitted. Those patch sets merely reorder the existing l2 and l3 length fields without changing their size or name.
/Bruce
PATCH v2 3/13 mbuf: expand ol_flags field to 64-bits
In reply to a message not in the archiveThe offload flags field (ol_flags) was 16-bits and had no further room for expansion. This patch increases the field size to 64-bits, using up the remaining reserved space in the single-cache-line mbuf.
NOTE: none of the values for existing flags have been changed, i.e. no new numbers have been explicitly reserved between existing flag definitions.
Updates in V2:
* Modified ol_flags field in kni mbuf structure
Updated kni data structure
--- app/test-pmd/config.c | 8 +-- app/test-pmd/csumonly.c | 8 +-- app/test-pmd/rxonly.c | 2 +- app/test-pmd/testpmd.h | 4 +- app/test-pmd/txonly.c | 2 +- .../linuxapp/eal/include/exec-env/rte_kni_common.h | 4 +- lib/librte_mbuf/rte_mbuf.c | 2 +- lib/librte_mbuf/rte_mbuf.h | 4 +- lib/librte_pmd_e1000/em_rxtx.c | 34 ++++++------ lib/librte_pmd_e1000/igb_rxtx.c | 62 ++++++++++------------ lib/librte_pmd_i40e/i40e_rxtx.c | 31 +++++------ lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 61 ++++++++++----------- lib/librte_pmd_ixgbe/ixgbe_rxtx.h | 2 +- 13 files changed, 106 insertions(+), 118 deletions(-)
app/test-pmd/config.c +4 -4
@@ -1714,14 +1714,14 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) }void -tx_cksum_set (portid_t port_id,uint8_t cksum_mask) +tx_cksum_set (portid_t port_id,uint64_t ol_flags) { -uint16_t tx_ol_flags; +uint64_t tx_ol_flags;if (port_id_is_invalid (port_id))return ;/* Clear last 4 bits and then set L3/4 checksum mask again */ - tx_ol_flags = (uint16_t ) (ports[port_id].tx_ol_flags &0xFFF0 ); - ports[port_id].tx_ol_flags = (uint16_t ) ((cksum_mask &0xf ) | tx_ol_flags); + tx_ol_flags = ports[port_id].tx_ol_flags & (~0x0Full ); + ports[port_id].tx_ol_flags = ((ol_flags &0xf ) | tx_ol_flags); }void
app/test-pmd/csumonly.c +4 -4
@@ -217,9 +217,9 @@ pkt_burst_checksum_forward(struct fwd_stream *fs)uint16_t nb_rx;uint16_t nb_tx;uint16_t i; -uint16_t ol_flags; -uint16_t pkt_ol_flags; -uint16_t tx_ol_flags; +uint64_t ol_flags; +uint64_t pkt_ol_flags; +uint64_t tx_ol_flags;uint16_t l4_proto;uint16_t eth_type;uint8_t l2_len;
@@ -261,7 +261,7 @@ pkt_burst_checksum_forward(struct fwd_stream *fs) mb = pkts_burst[i]; l2_len =sizeof (struct ether_hdr ); pkt_ol_flags = mb->ol_flags; - ol_flags = (uint16_t ) (pkt_ol_flags & (~PKT_TX_L4_MASK )); + ol_flags = (pkt_ol_flags & (~PKT_TX_L4_MASK )); eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *); eth_type =rte_be_to_cpu_16 (eth_hdr->ether_type);
app/test-pmd/rxonly.c +1 -1
@@ -109,7 +109,7 @@ pkt_burst_receive(struct fwd_stream *fs)struct rte_mbuf *mb;struct ether_hdr *eth_hdr;uint16_t eth_type; -uint16_t ol_flags; +uint64_t ol_flags;uint16_t nb_rx;uint16_t i;#ifdef RTE_TEST_PMD_RECORD_CORE_CYCLES
app/test-pmd/testpmd.h +2 -2
@@ -141,7 +141,7 @@ struct rte_port {struct fwd_stream *rx_stream;/**< Port RX stream, if unique */ struct fwd_stream *tx_stream;/**< Port TX stream, if unique */ unsigned int socket_id;/**< For NUMA support */ -uint16_t tx_ol_flags;/**< Offload Flags of TX packets. */ +uint64_t tx_ol_flags;/**< Offload Flags of TX packets. */ uint16_t tx_vlan_id;/**< Tag Id. in TX VLAN packets. */ void *fwd_ctx;/**< Forwarding mode context */ uint64_t rx_bad_ip_csum;/**< rx pkts with bad ip checksum */
@@ -494,7 +494,7 @@ void tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on);void set_qmap (portid_t port_id,uint8_t is_rx,uint16_t queue_id,uint8_t map_value); -void tx_cksum_set (portid_t port_id,uint8_t cksum_mask); +void tx_cksum_set (portid_t port_id,uint64_t ol_flags);void set_verbose_level (uint16_t vb_level);void set_tx_pkt_segments (unsigned *seg_lengths,unsigned nb_segs);
app/test-pmd/txonly.c +1 -1
@@ -203,7 +203,7 @@ pkt_burst_transmit(struct fwd_stream *fs)uint16_t nb_tx;uint16_t nb_pkt;uint16_t vlan_tci; -uint16_t ol_flags; +uint64_t ol_flags;uint8_t i;#ifdef RTE_TEST_PMD_RECORD_CORE_CYCLES uint64_t start_tsc;
lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h +2 -2
@@ -112,8 +112,8 @@ struct rte_kni_mbuf {char pad0[10 ];uint16_t data_off;/**< Start address of data in segment buffer. */ char pad1[4 ]; -uint16_t ol_flags;/**< Offload features. */ -char pad2[8 ]; +uint64_t ol_flags;/**< Offload features. */ +char pad2[2 ];uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segment data_len. */ char pad3[8 ];
lib/librte_mbuf/rte_mbuf.c +1 -1
@@ -174,7 +174,7 @@ rte_pktmbuf_dump(FILE *f, const struct rte_mbuf *m, unsigned dump_len)fprintf (f,"dump mbuf at 0x%p, phys=%" PRIx64", buf_len=%u\n" , m, (uint64_t )m->buf_physaddr, (unsigned )m->buf_len); -fprintf (f," pkt_len=%" PRIu32", ol_flags=%" PRIx16", nb_segs=%u, " +fprintf (f," pkt_len=%" PRIu32", ol_flags=%" PRIx64", nb_segs=%u, " "in_port=%u\n" , m->pkt_len, m->ol_flags, (unsigned )m->nb_segs, (unsigned )m->port); nb_segs = m->nb_segs;
lib/librte_mbuf/rte_mbuf.h +1 -3
@@ -141,9 +141,7 @@ struct rte_mbuf {uint8_t nb_segs;/**< Number of segments. */ uint8_t port;/**< Input port. */ -uint16_t ol_flags;/**< Offload features. */ -uint16_t reserved0;/**< Unused field. Required for padding */ -uint32_t reserved1;/**< Unused field. Required for padding */ +uint64_t ol_flags;/**< Offload features. */ /* remaining bytes are set on RX when pulling packet from descriptor */ uint16_t reserved2;/**< Unused field. Required for padding */
lib/librte_pmd_e1000/em_rxtx.c +16 -18
@@ -168,7 +168,7 @@ union em_vlan_macip {* Structure to check if new context need be built */ struct em_ctx_info { -uint16_t flags;/**< ol_flags related to context build. */ +uint64_t flags;/**< ol_flags related to context build. */ uint32_t cmp_mask;/**< compare mask */ union em_vlan_macip hdrlen;/**< L2 and L3 header lenghts */ };
@@ -238,7 +238,7 @@ struct em_tx_queue {static inline void em_set_xmit_ctx (struct em_tx_queue * txq,volatile struct e1000_context_desc *ctx_txd, -uint16_t flags, +uint64_t flags,union em_vlan_macip hdrlen) {uint32_t cmp_mask, cmd_len;
@@ -304,7 +304,7 @@ em_set_xmit_ctx(struct em_tx_queue* txq,* or create a new context descriptor. */ static inline uint32_t -what_ctx_update (struct em_tx_queue *txq,uint16_t flags, +what_ctx_update (struct em_tx_queue *txq,uint64_t flags,union em_vlan_macip hdrlen) {/* If match with the current context */
@@ -377,7 +377,7 @@ em_xmit_cleanup(struct em_tx_queue *txq) }static inline uint32_t -tx_desc_cksum_flags_to_upper (uint16_t ol_flags) +tx_desc_cksum_flags_to_upper (uint64_t ol_flags) {static const uint32_t l4_olinfo[2 ] = {0 ,E1000_TXD_POPTS_TXSM <<8 };static const uint32_t l3_olinfo[2 ] = {0 ,E1000_TXD_POPTS_IXSM <<8 };
@@ -403,12 +403,12 @@ eth_em_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,uint32_t popts_spec;uint32_t cmd_type_len;uint16_t slen; -uint16_t ol_flags; +uint64_t ol_flags;uint16_t tx_id;uint16_t tx_last;uint16_t nb_tx;uint16_t nb_used; -uint16_t tx_ol_req; +uint64_t tx_ol_req;uint32_t ctx;uint32_t new_ctx;union em_vlan_macip hdrlen;
@@ -438,8 +438,7 @@ eth_em_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, ol_flags = tx_pkt->ol_flags;/* If hardware offload required */ - tx_ol_req = (uint16_t )(ol_flags & (PKT_TX_IP_CKSUM | -PKT_TX_L4_MASK )); + tx_ol_req = (ol_flags & (PKT_TX_IP_CKSUM |PKT_TX_L4_MASK ));if (tx_ol_req) { hdrlen.f.vlan_tci = tx_pkt->vlan_tci; hdrlen.f.l2_len = tx_pkt->l2_len;
@@ -642,22 +641,21 @@ end_of_tx:* **********************************************************************/ -static inline uint16_t +static inline uint64_t rx_desc_status_to_pkt_flags (uint32_t rx_status) { -uint16_t pkt_flags; +uint64_t pkt_flags;/* Check if VLAN present */ - pkt_flags = (uint16_t )((rx_status &E1000_RXD_STAT_VP ) ? -PKT_RX_VLAN_PKT :0 ); + pkt_flags = ((rx_status &E1000_RXD_STAT_VP ) ?PKT_RX_VLAN_PKT :0 );return pkt_flags; } -static inline uint16_t +static inline uint64_t rx_desc_error_to_pkt_flags (uint32_t rx_error) { -uint16_t pkt_flags =0 ; +uint64_t pkt_flags =0 ;if (rx_error &E1000_RXD_ERR_IPE ) pkt_flags |=PKT_RX_IP_CKSUM_BAD ;
@@ -801,8 +799,8 @@ eth_em_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rxm->port = rxq->port_id; rxm->ol_flags =rx_desc_status_to_pkt_flags (status); - rxm->ol_flags = (uint16_t )(rxm->ol_flags | -rx_desc_error_to_pkt_flags (rxd.errors)); + rxm->ol_flags = rxm->ol_flags | +rx_desc_error_to_pkt_flags (rxd.errors);/* Only valid if PKT_RX_VLAN_PKT set in pkt_flags */ rxm->vlan_tci =rte_le_to_cpu_16 (rxd.special);
@@ -1027,8 +1025,8 @@ eth_em_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, first_seg->port = rxq->port_id; first_seg->ol_flags =rx_desc_status_to_pkt_flags (status); - first_seg->ol_flags = (uint16_t )(first_seg->ol_flags | -rx_desc_error_to_pkt_flags (rxd.errors)); + first_seg->ol_flags = first_seg->ol_flags | +rx_desc_error_to_pkt_flags (rxd.errors);/* Only valid if PKT_RX_VLAN_PKT set in pkt_flags */ rxm->vlan_tci =rte_le_to_cpu_16 (rxd.special);
lib/librte_pmd_e1000/igb_rxtx.c +28 -34
169 lines not on this page · raw message
lib/librte_pmd_i40e/i40e_rxtx.c +16 -15
@@ -91,27 +91,27 @@ static uint16_t i40e_xmit_pkts_simple(void *tx_queue, uint16_t nb_pkts);/* Translate the rx descriptor status to pkt flags */ -static inline uint16_t +static inline uint64_t i40e_rxd_status_to_pkt_flags (uint64_t qword) { -uint16_t flags; +uint64_t flags;/* Check if VLAN packet */ - flags = (uint16_t )(qword & (1 <<I40E_RX_DESC_STATUS_L2TAG1P_SHIFT ) ? -PKT_RX_VLAN_PKT :0 ); + flags = qword & (1 <<I40E_RX_DESC_STATUS_L2TAG1P_SHIFT ) ? +PKT_RX_VLAN_PKT :0 ;/* Check if RSS_HASH */ - flags |= (uint16_t )((((qword >>I40E_RX_DESC_STATUS_FLTSTAT_SHIFT ) & + flags |= (((qword >>I40E_RX_DESC_STATUS_FLTSTAT_SHIFT ) &I40E_RX_DESC_FLTSTAT_RSS_HASH ) == -I40E_RX_DESC_FLTSTAT_RSS_HASH ) ?PKT_RX_RSS_HASH :0 ); +I40E_RX_DESC_FLTSTAT_RSS_HASH ) ?PKT_RX_RSS_HASH :0 ;return flags; } -static inline uint16_t +static inline uint64_t i40e_rxd_error_to_pkt_flags (uint64_t qword) { -uint16_t flags =0 ; +uint64_t flags =0 ;uint64_t error_bits = (qword >>I40E_RXD_QW1_ERROR_SHIFT );#define I40E_RX_ERR_BITS 0x3f
@@ -143,12 +143,12 @@ i40e_rxd_error_to_pkt_flags(uint64_t qword) }/* Translate pkt types to pkt flags */ -static inline uint16_t +static inline uint64_t i40e_rxd_ptype_to_pkt_flags (uint64_t qword) {uint8_t ptype = (uint8_t )((qword &I40E_RXD_QW1_PTYPE_MASK ) >>I40E_RXD_QW1_PTYPE_SHIFT ); -static const uint16_t ip_ptype_map[I40E_MAX_PKT_TYPE ] = { +static const uint64_t ip_ptype_map[I40E_MAX_PKT_TYPE ] = {0 ,/* PTYPE 0 */ 0 ,/* PTYPE 1 */ 0 ,/* PTYPE 2 */
@@ -567,7 +567,7 @@ i40e_rx_scan_hw_ring(struct i40e_rx_queue *rxq)uint32_t rx_status;int32_t s[I40E_LOOK_AHEAD ], nb_dd;int32_t i, j, nb_rx =0 ; -uint16_t pkt_flags; +uint64_t pkt_flags; rxdp = &rxq->rx_ring[rxq->rx_tail]; rxep = &rxq->sw_ring[rxq->rx_tail];
@@ -789,7 +789,7 @@ i40e_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)uint16_t rx_packet_len;uint16_t rx_id, nb_hold;uint64_t dma_addr; -uint16_t pkt_flags; +uint64_t pkt_flags; nb_rx =0 ; nb_hold =0 ;
@@ -896,10 +896,11 @@ i40e_recv_scattered_pkts(void *rx_queue,struct rte_mbuf *last_seg = rxq->pkt_last_seg;struct rte_mbuf *nmb, *rxm;uint16_t rx_id = rxq->rx_tail; -uint16_t nb_rx =0 , nb_hold =0 , rx_packet_len, pkt_flags; +uint16_t nb_rx =0 , nb_hold =0 , rx_packet_len;uint32_t rx_status;uint64_t qword1;uint64_t dma_addr; +uint64_t pkt_flags;while (nb_rx < nb_pkts) { rxdp = &rx_ring[rx_id];
@@ -1046,7 +1047,7 @@ i40e_recv_scattered_pkts(void *rx_queue,/* Check if the context descriptor is needed for TX offloading */ static inline uint16_t -i40e_calc_context_desc (uint16_t flags) +i40e_calc_context_desc (uint64_t flags) {uint16_t mask =0 ;
@@ -1075,7 +1076,7 @@ i40e_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)uint32_t td_offset;uint32_t tx_flags;uint32_t td_tag; -uint16_t ol_flags; +uint64_t ol_flags;uint8_t l2_len;uint8_t l3_len;uint16_t nb_used;
lib/librte_pmd_ixgbe/ixgbe_rxtx.c +29 -32
167 lines not on this page · raw message
lib/librte_pmd_ixgbe/ixgbe_rxtx.h +1 -1
@@ -177,7 +177,7 @@ union ixgbe_vlan_macip {*/ struct ixgbe_advctx_info { -uint16_t flags;/**< ol_flags for context build. */ +uint64_t flags;/**< ol_flags for context build. */ uint32_t cmp_mask;/**< compare mask for vlan_macip_lens */ union ixgbe_vlan_macip vlan_macip_lens;/**< vlan, mac ip length. */ };
Signature · 1 line
1.9.3
PATCH v2 4/13 mbuf: introduce a flag to indicate a control mbuf
In reply to a message not in the archiveSince the flags field is now 64-bits, we can allow one bit to be used to indicate a control i.e. non-packet mbuf. Dedicate the high bit (bit 63) for this purpose and add in a utility macro to test if a given mbuf has the bit set or not.
Updated in V2:
* Fix typo in comment in rte_mbuf.h
--- lib/librte_mbuf/rte_mbuf.c | 2 ++ lib/librte_mbuf/rte_mbuf.h | 19 +++++++++++++++++++ 2 files changed, 21 insertions(+)
lib/librte_mbuf/rte_mbuf.c +2 -0
@@ -69,7 +69,9 @@ rte_ctrlmbuf_init(struct rte_mempool *mp, void *_m, __attribute__((unused)) unsigned i) { +struct rte_mbuf *m = _m;rte_pktmbuf_init (mp, opaque_arg, _m, i); + m->ol_flags |=CTRL_MBUF_FLAG ; }/*
lib/librte_mbuf/rte_mbuf.h +19 -0
@@ -91,6 +91,7 @@ extern "C" {#define PKT_TX_IPV4_CSUM 0x1000/**< Alias of PKT_TX_IP_CKSUM. */ #define PKT_TX_IPV4 PKT_RX_IPV4_HDR/**< IPv4 with no IP checksum offload. */ #define PKT_TX_IPV6 PKT_RX_IPV6_HDR/**< IPv6 packet */ +/* * Bit 14~13 used for L4 packet type with checksum enabled. * 00: Reserved
@@ -106,6 +107,9 @@ extern "C" {/* Bit 15 */ #define PKT_TX_IEEE1588_TMST 0x8000/**< TX IEEE1588 packet to timestamp. */ +/* Use final bit of flags to indicate a control mbuf */ +#define CTRL_MBUF_FLAG (1ULL << 63) +/** * Bit Mask to indicate what bits required for building TX context */
@@ -471,6 +475,21 @@ void rte_ctrlmbuf_init(struct rte_mempool *mp, void *opaque_arg,*/ #define rte_ctrlmbuf_len (m) rte_pktmbuf_data_len(m) +/** +* Tests if an mbuf is a control mbuf +* +* @param m +* The mbuf to be tested +* @return +* - True (1) if the mbuf is a control mbuf +* - False(0) otherwise +*/ +static inline int +rte_is_ctrlmbuf (struct rte_mbuf *m) +{ +return (!!(m->ol_flags &CTRL_MBUF_FLAG )); +} +/* Operations on pkt mbuf */ /**
Signature · 1 line
1.9.3
PATCH v2 5/13 mbuf: minor changes for readability
In reply to a message not in the archive* Ensure comments line up correctly
* Simplify the #ifdefs around the refcnt fields to make them clearer
Updates in V2:
* None
--- lib/librte_mbuf/rte_mbuf.h | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-)
lib/librte_mbuf/rte_mbuf.h +16 -17
@@ -126,7 +126,6 @@ struct rte_mbuf {uint16_t buf_len;/**< Length of segment buffer. */ uint16_t data_off; -#ifdef RTE_MBUF_REFCNT /** * 16-bit Reference counter. * It should only be accessed using the following functions:
@@ -136,21 +135,21 @@ struct rte_mbuf {* config option. */ union { -rte_atomic16_t refcnt_atomic;/**< Atomically accessed refcnt */ -uint16_t refcnt;/**< Non-atomically accessed refcnt */ - }; -#else -uint16_t refcnt_reserved;/**< Do not use this field */ +#ifdef RTE_MBUF_REFCNT +rte_atomic16_t refcnt_atomic;/**< Atomically accessed refcnt */ +uint16_t refcnt;/**< Non-atomically accessed refcnt */ #endif -uint8_t nb_segs;/**< Number of segments. */ -uint8_t port;/**< Input port. */ +uint16_t refcnt_reserved;/**< Do not use this field */ + }; +uint8_t nb_segs;/**< Number of segments. */ +uint8_t port;/**< Input port. */ -uint64_t ol_flags;/**< Offload features. */ +uint64_t ol_flags;/**< Offload features. */ /* remaining bytes are set on RX when pulling packet from descriptor */ -uint16_t reserved2;/**< Unused field. Required for padding */ -uint16_t data_len;/**< Amount of data in segment buffer. */ -uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ +uint16_t reserved2;/**< Unused field. Required for padding */ +uint16_t data_len;/**< Amount of data in segment buffer. */ +uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ union {uint16_t l2_l3_len;/**< combined l2/l3 lengths as single var */ struct {
@@ -158,15 +157,15 @@ struct rte_mbuf { uint16_t l2_len:7; /**< L2 (MAC) Header Length. */ }; }; -uint16_t vlan_tci;/**< VLAN Tag Control Identifier (CPU order). */ +uint16_t vlan_tci;/**< VLAN Tag Control Identifier (CPU order) */ union { -uint32_t rss;/**< RSS hash result if RSS enabled */ +uint32_t rss;/**< RSS hash result if RSS enabled */ struct {uint16_t hash;uint16_t id; - } fdir;/**< Filter identifier if FDIR enabled */ -uint32_t sched;/**< Hierarchical scheduler */ - } hash;/**< hash information */ + } fdir;/**< Filter identifier if FDIR enabled */ +uint32_t sched;/**< Hierarchical scheduler */ + } hash;/**< hash information */ /* fields only used in slow path or on TX */ struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */
Signature · 1 line
1.9.3
PATCH v2 6/13 mbuf: use macros only to access the mbuf metadata
In reply to a message not in the archiveRemoved the explicit zero-sized metadata definition at the end of the mbuf data structure. Updated the metadata macros to take account of this change so that all existing code which uses those macros still works.
Updates in V2:
* None
--- lib/librte_mbuf/rte_mbuf.h | 22 ++++++++-------------- 1 file changed, 8 insertions(+), 14 deletions(-)
lib/librte_mbuf/rte_mbuf.h +8 -14
@@ -171,31 +171,25 @@ struct rte_mbuf {struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */ struct rte_mbuf *next;/**< Next segment of scattered packet. */ -union { -uint8_t metadata[0 ]; -uint16_t metadata16[0 ]; -uint32_t metadata32[0 ]; -uint64_t metadata64[0 ]; - } __rte_cache_aligned; } __rte_cache_aligned;#define RTE_MBUF_METADATA_UINT8 (mbuf, offset) \ - (mbuf->metadata[offset]) + (((uint8_t *)&(mbuf)[1])[offset])#define RTE_MBUF_METADATA_UINT16 (mbuf, offset) \ - (mbuf->metadata16[offset/sizeof(uint16_t)]) + (((uint16_t *)&(mbuf)[1])[offset/sizeof(uint16_t)])#define RTE_MBUF_METADATA_UINT32 (mbuf, offset) \ - (mbuf->metadata32[offset/sizeof(uint32_t)]) + (((uint32_t *)&(mbuf)[1])[offset/sizeof(uint32_t)])#define RTE_MBUF_METADATA_UINT64 (mbuf, offset) \ - (mbuf->metadata64[offset/sizeof(uint64_t)]) + (((uint64_t *)&(mbuf)[1])[offset/sizeof(uint64_t)])#define RTE_MBUF_METADATA_UINT8_PTR (mbuf, offset) \ - (&mbuf->metadata[offset]) + (&RTE_MBUF_METADATA_UINT8(mbuf, offset))#define RTE_MBUF_METADATA_UINT16_PTR (mbuf, offset) \ - (&mbuf->metadata16[offset/sizeof(uint16_t)]) + (&RTE_MBUF_METADATA_UINT16(mbuf, offset))#define RTE_MBUF_METADATA_UINT32_PTR (mbuf, offset) \ - (&mbuf->metadata32[offset/sizeof(uint32_t)]) + (&RTE_MBUF_METADATA_UINT32(mbuf, offset))#define RTE_MBUF_METADATA_UINT64_PTR (mbuf, offset) \ - (&mbuf->metadata64[offset/sizeof(uint64_t)]) + (&RTE_MBUF_METADATA_UINT64(mbuf, offset))/** * Given the buf_addr returns the pointer to corresponding mbuf.
Signature · 1 line
1.9.3
PATCH v2 7/13 mbuf: move metadata macros to rte_port library
In reply to a message not in the archiveThe metadata macros are only used by libs and apps using the rte_port packet framework library, so move them to a header file there.
--- lib/librte_mbuf/rte_mbuf.h | 18 ------------------ lib/librte_port/rte_port.h | 22 ++++++++++++++++++++++ 2 files changed, 22 insertions(+), 18 deletions(-)
lib/librte_mbuf/rte_mbuf.h +0 -18
@@ -173,24 +173,6 @@ struct rte_mbuf { } __rte_cache_aligned; -#define RTE_MBUF_METADATA_UINT8 (mbuf, offset) \ - (((uint8_t *)&(mbuf)[1])[offset]) -#define RTE_MBUF_METADATA_UINT16 (mbuf, offset) \ - (((uint16_t *)&(mbuf)[1])[offset/sizeof(uint16_t)]) -#define RTE_MBUF_METADATA_UINT32 (mbuf, offset) \ - (((uint32_t *)&(mbuf)[1])[offset/sizeof(uint32_t)]) -#define RTE_MBUF_METADATA_UINT64 (mbuf, offset) \ - (((uint64_t *)&(mbuf)[1])[offset/sizeof(uint64_t)]) - -#define RTE_MBUF_METADATA_UINT8_PTR (mbuf, offset) \ - (&RTE_MBUF_METADATA_UINT8(mbuf, offset)) -#define RTE_MBUF_METADATA_UINT16_PTR (mbuf, offset) \ - (&RTE_MBUF_METADATA_UINT16(mbuf, offset)) -#define RTE_MBUF_METADATA_UINT32_PTR (mbuf, offset) \ - (&RTE_MBUF_METADATA_UINT32(mbuf, offset)) -#define RTE_MBUF_METADATA_UINT64_PTR (mbuf, offset) \ - (&RTE_MBUF_METADATA_UINT64(mbuf, offset)) -/** * Given the buf_addr returns the pointer to corresponding mbuf. */
lib/librte_port/rte_port.h +22 -0
@@ -50,6 +50,28 @@ extern "C" {#include <stdint.h> #include <rte_mbuf.h> +/** +* Macros to allow accessing metadata stored in the mbuf headroom +* just beyond the end of the mbuf data structure returned by a port +*/ +#define RTE_MBUF_METADATA_UINT8 (mbuf, offset) \ + (((uint8_t *)&(mbuf)[1])[offset]) +#define RTE_MBUF_METADATA_UINT16 (mbuf, offset) \ + (((uint16_t *)&(mbuf)[1])[offset/sizeof(uint16_t)]) +#define RTE_MBUF_METADATA_UINT32 (mbuf, offset) \ + (((uint32_t *)&(mbuf)[1])[offset/sizeof(uint32_t)]) +#define RTE_MBUF_METADATA_UINT64 (mbuf, offset) \ + (((uint64_t *)&(mbuf)[1])[offset/sizeof(uint64_t)]) + +#define RTE_MBUF_METADATA_UINT8_PTR (mbuf, offset) \ + (&RTE_MBUF_METADATA_UINT8(mbuf, offset)) +#define RTE_MBUF_METADATA_UINT16_PTR (mbuf, offset) \ + (&RTE_MBUF_METADATA_UINT16(mbuf, offset)) +#define RTE_MBUF_METADATA_UINT32_PTR (mbuf, offset) \ + (&RTE_MBUF_METADATA_UINT32(mbuf, offset)) +#define RTE_MBUF_METADATA_UINT64_PTR (mbuf, offset) \ + (&RTE_MBUF_METADATA_UINT64(mbuf, offset)) +/* * Port IN *
Signature · 1 line
1.9.3
PATCH v2 8/13 mbuf: add named points inside the mbuf structure
In reply to a message not in the archiveAdd markers or "labels" at given points inside the mbuf which can be used instead of individual fields to identify the start of logical sections inside the mbuf.
The use of typedefs and dummy fields was chosen over using unions because of a couple reasons:
* unions cause an extra level of indentation (more likely two levels as a union containing a struct for multiple fields would be needed). This makes the lines longer than they need to be and increases the need for wrapping. [This was the main reason]
* with markers, you can apply multiple markers at the same point if wanted.
Updates in V2:
* None
--- lib/librte_mbuf/rte_mbuf.h | 9 +++++++++ 1 file changed, 9 insertions(+)
lib/librte_mbuf/rte_mbuf.h +9 -0
@@ -115,14 +115,22 @@ extern "C" {*/ #define PKT_TX_OFFLOAD_MASK (PKT_TX_VLAN_PKT | PKT_TX_IP_CKSUM | PKT_TX_L4_MASK) +/* define a set of marker types that can be used to refer to set points in the +* mbuf */ +typedef void *MARKER [0 ];/**< generic marker for a point in a structure */ +typedef uint64_t MARKER64 [0 ];/**< marker that allows us to overwrite 8 bytes +* with a single assignment */ /** * The generic rte_mbuf, containing a packet mbuf. */ struct rte_mbuf { +MARKER cacheline0; +void *buf_addr;/**< Virtual address of segment buffer. */ phys_addr_t buf_physaddr;/**< Physical address of segment buffer. */ /* next 8 bytes are initialised on RX descriptor rearm */ +MARKER64 rearm_data;uint16_t buf_len;/**< Length of segment buffer. */ uint16_t data_off;
@@ -147,6 +155,7 @@ struct rte_mbuf {uint64_t ol_flags;/**< Offload features. */ /* remaining bytes are set on RX when pulling packet from descriptor */ +MARKER rx_descriptor_fields1;uint16_t reserved2;/**< Unused field. Required for padding */ uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segments. */
Signature · 1 line
1.9.3
PATCH v2 9/13 ixgbe: rework vector pmd following mbuf changes
In reply to a message not in the archiveThe vector PMD expects fields to be in a specific order so that it can do vector operations on multiple fields at a time. Following mbuf rework, adjust driver to take account of the new layout and re-enable it in the config.
Updates in V2:
* Remove extra line at end of file to eliminate git warning
--- config/common_linuxapp | 2 +- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 2 +- lib/librte_pmd_ixgbe/ixgbe_rxtx.h | 4 +- lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c | 135 ++++++++++++---------------------- 4 files changed, 53 insertions(+), 90 deletions(-)
config/common_linuxapp +1 -1
@@ -191,7 +191,7 @@ CONFIG_RTE_LIBRTE_IXGBE_DEBUG_DRIVER=n CONFIG_RTE_LIBRTE_IXGBE_PF_DISABLE_STRIP_CRC=n CONFIG_RTE_LIBRTE_IXGBE_RX_ALLOW_BULK_ALLOC=y CONFIG_RTE_LIBRTE_IXGBE_ALLOW_UNSUPPORTED_SFP=n -CONFIG_RTE_IXGBE_INC_VECTOR=n +CONFIG_RTE_IXGBE_INC_VECTOR=y CONFIG_RTE_IXGBE_RX_OLFLAGS_ENABLE=y #
lib/librte_pmd_ixgbe/ixgbe_rxtx.c +1 -1
@@ -2166,7 +2166,7 @@ ixgbe_dev_rx_queue_setup(struct rte_eth_dev *dev,if (!ixgbe_rx_vec_condition_check (dev)) {PMD_INIT_LOG (INFO ,"Vector rx enabled, please make " "sure RX burst size no less than 32.\n" ); -ixgbe_rxq_vec_setup (rxq, socket_id); +ixgbe_rxq_vec_setup (rxq); dev->rx_pkt_burst = ixgbe_recv_pkts_vec; }#endif
lib/librte_pmd_ixgbe/ixgbe_rxtx.h +2 -2
@@ -115,6 +115,7 @@ struct igb_rx_queue {struct igb_rx_entry *sw_ring;/**< address of RX software ring. */ struct rte_mbuf *pkt_first_seg;/**< First segment of current packet. */ struct rte_mbuf *pkt_last_seg;/**< Last segment of current packet. */ +uint64_t mbuf_initializer;/**< value to init mbufs */ uint16_t nb_rx_desc;/**< number of RX descriptors. */ uint16_t rx_tail;/**< current value of RDT register. */ uint16_t nb_rx_hold;/**< number of held free RX desc. */
@@ -126,7 +127,6 @@ struct igb_rx_queue {#ifdef RTE_IXGBE_INC_VECTOR uint16_t rxrearm_nb;/**< the idx we start the re-arming from */ uint16_t rxrearm_start;/**< number of remaining to be re-armed */ -__m128i misc_info;/**< cache XMM combine port_id/crc/nb_segs */ #endif uint16_t rx_free_thresh;/**< max free RX desc to hold. */ uint16_t queue_id;/**< RX queue index. */
@@ -259,7 +259,7 @@ struct ixgbe_txq_ops {uint16_t ixgbe_recv_pkts_vec (void *rx_queue,struct rte_mbuf **rx_pkts,uint16_t nb_pkts);uint16_t ixgbe_xmit_pkts_vec (void *tx_queue,struct rte_mbuf **tx_pkts,uint16_t nb_pkts);int ixgbe_txq_vec_setup (struct igb_tx_queue *txq,unsigned int socket_id); -int ixgbe_rxq_vec_setup (struct igb_rx_queue *rxq,unsigned int socket_id); +int ixgbe_rxq_vec_setup (struct igb_rx_queue *rxq);int ixgbe_rx_vec_condition_check (struct rte_eth_dev *dev);#endif
lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +49 -86
242 lines not on this page · raw message
Signature · 1 line
1.9.3
PATCH v2 10/13 mbuf: split mbuf across two cache lines.
In reply to a message not in the archiveThis change splits the mbuf in two to move the pool and next pointers to the second cache line. This frees up 16 bytes in first cache line.
The reason for this change is that we believe that there is no possible way that we can ever fit all the fields we need to fit into a 64-byte mbuf, and so we need to start looking at a 128-byte mbuf instead. Examples of new fields that need to fit in, include -
* 32-bits more for filter information for support for the new filters in the i40e driver (and possibly other future drivers)
* an additional 2-4 bytes for storing info on a second vlan tag to allow drivers to support double Vlan/QinQ
* 4-bytes for storing a sequence number to enable out of order packet processing and subsequent packet reordering
as well as potentially a number of other fields or splitting out fields that are superimposed over each other right now, e.g. for the qos scheduler. We also want to allow space for use by other non-Intel NIC drivers that may be open-sourced to dpdk.org in the future too, where they support fields and offloads that currently supported hardware doesn't.
If we accept the fact of a 2-cache-line mbuf, then the issue becomes how to rework things so that we spread our fields over the two cache lines while causing the lowest slow-down possible. The general approach that we are looking to take is to focus the first cache line on fields that are updated on RX , so that receive only deals with one cache line. The second cache line can be used for application data and information that will only be used on the TX leg. This would allow us to work on the first cache line in RX as now, and have the second cache line being prefetched in the background so that it is available when necessary. Hardware prefetches should help us out here. We also may move rarely used, or slow-path RX fields e.g. such as those for chained mbufs with jumbo frames, to the second cache line, depending upon the performance impact and bytes savings achieved.
Updated in V2:
* Expanded commit description to include contents of a previous mail describing some of the logic behind expanding mbuf to two cache lines
* Update kni mbuf structure
--- app/test/test_mbuf.c | 2 +- lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h | 6 +++--- lib/librte_mbuf/rte_mbuf.h | 3 ++- 3 files changed, 6 insertions(+), 5 deletions(-)
app/test/test_mbuf.c +1 -1
@@ -782,7 +782,7 @@ test_failing_mbuf_sanity_check(void)static int test_mbuf (void ) { -RTE_BUILD_BUG_ON (sizeof (struct rte_mbuf ) !=64 ); +RTE_BUILD_BUG_ON (sizeof (struct rte_mbuf ) !=CACHE_LINE_SIZE *2 );/* create pktmbuf pool if it does not exist */ if (pktmbuf_pool ==NULL ) {
lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h +3 -3
@@ -108,7 +108,7 @@ struct rte_kni_fifo {* Padding is necessary to assure the offsets of these fields */ struct rte_kni_mbuf { -void *buf_addr; +void *buf_addr __attribute__((__aligned__ (64 )));char pad0[10 ];uint16_t data_off;/**< Start address of data in segment buffer. */ char pad1[4 ];
@@ -117,9 +117,9 @@ struct rte_kni_mbuf {uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segment data_len. */ char pad3[8 ]; -void *pool; + void *pool __attribute__((__aligned__(64)));void *next; -} __attribute__((__aligned__(64))); +};/* * Struct used to create a KNI device. Passed to the kernel in IOCTL call
lib/librte_mbuf/rte_mbuf.h +2 -1
@@ -176,7 +176,8 @@ struct rte_mbuf {uint32_t sched;/**< Hierarchical scheduler */ } hash;/**< hash information */ -/* fields only used in slow path or on TX */ +/* second cache line - fields only used in slow path or on TX */ + MARKER cacheline1 __rte_cache_aligned;struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */ struct rte_mbuf *next;/**< Next segment of scattered packet. */
Signature · 1 line
1.9.3
PATCH v2 11/13 mbuf: move l2_len and l3_len to second cache line
In reply to a message not in the archiveThe l2_len and l3_len fields are used for TX offloads and so should be put on the second cache line, along with the other fields only used on TX.
Updates in V2:
* The l2 and l3 lengths can be accessed as a single uint16_t for performance, as well as individually.
--- lib/librte_mbuf/rte_mbuf.h | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-)
lib/librte_mbuf/rte_mbuf.h +9 -7
@@ -159,13 +159,7 @@ struct rte_mbuf {uint16_t reserved2;/**< Unused field. Required for padding */ uint16_t data_len;/**< Amount of data in segment buffer. */ uint32_t pkt_len;/**< Total pkt len: sum of all segments. */ -union { -uint16_t l2_l3_len;/**< combined l2/l3 lengths as single var */ -struct { -uint16_t l3_len:9 ;/**< L3 (IP) Header Length. */ -uint16_t l2_len:7 ;/**< L2 (MAC) Header Length. */ - }; - }; +uint16_t reserved;uint16_t vlan_tci;/**< VLAN Tag Control Identifier (CPU order) */ union {uint32_t rss;/**< RSS hash result if RSS enabled */
@@ -181,6 +175,14 @@ struct rte_mbuf {struct rte_mempool *pool;/**< Pool from which mbuf was allocated. */ struct rte_mbuf *next;/**< Next segment of scattered packet. */ +/* fields to support TX offloads */ +union { +uint16_t l2_l3_len;/**< combined l2/l3 lengths as single var */ +struct { +uint16_t l3_len:9 ;/**< L3 (IP) Header Length. */ +uint16_t l2_len:7 ;/**< L2 (MAC) Header Length. */ + }; + }; } __rte_cache_aligned;/**
Signature · 1 line
1.9.3
PATCH v2 12/13 ixgbe: Fix perf regression due to moved pool ptr
In reply to a message not in the archiveAdjust the fast-path code to fix the regression caused by the pool pointer moving to the second cache line. This change adjusts the prefetching and also the way in which the mbufs are freed back to the mempool.
Note: slow-path e.g. path supporting jumbo frames, is still slower, but is dealt with by a later commit
Updates in V2:
* fixup checkpatch issue
--- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 8 ++- lib/librte_pmd_ixgbe/ixgbe_rxtx.h | 14 +----- lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c | 92 +++++++++++++---------------------- 3 files changed, 38 insertions(+), 76 deletions(-)
lib/librte_pmd_ixgbe/ixgbe_rxtx.c +3 -5
@@ -142,10 +142,6 @@ ixgbe_tx_free_bufs(struct igb_tx_queue *txq)*/ txep = &(txq->sw_ring[txq->tx_next_dd - (txq->tx_rs_thresh -1 )]); -/* prefetch the mbufs that are about to be freed */ -for (i =0 ; i < txq->tx_rs_thresh; ++i) -rte_prefetch0 ((txep + i)->mbuf); -/* free buffers one at a time */ if ((txq->txq_flags & (uint32_t )ETH_TXQ_FLAGS_NOREFCOUNT ) !=0 ) {for (i =0 ; i < txq->tx_rs_thresh; ++i, ++txep) {
@@ -186,6 +182,7 @@ tx4(volatile union ixgbe_adv_tx_desc *txdp, struct rte_mbuf **pkts) ((uint32_t )DCMD_DTYP_FLAGS | pkt_len); txdp->read.olinfo_status = (pkt_len <<IXGBE_ADVTXD_PAYLEN_SHIFT ); +rte_prefetch0 (&(*pkts)->pool); } }
@@ -205,6 +202,7 @@ tx1(volatile union ixgbe_adv_tx_desc *txdp, struct rte_mbuf **pkts) ((uint32_t )DCMD_DTYP_FLAGS | pkt_len); txdp->read.olinfo_status = (pkt_len <<IXGBE_ADVTXD_PAYLEN_SHIFT ); +rte_prefetch0 (&(*pkts)->pool); }/*
@@ -1875,7 +1873,7 @@ ixgbe_dev_tx_queue_setup(struct rte_eth_dev *dev,PMD_INIT_LOG (INFO ,"Using simple tx code path\n" );#ifdef RTE_IXGBE_INC_VECTOR if (txq->tx_rs_thresh <=RTE_IXGBE_TX_MAX_FREE_BUF_SZ && -ixgbe_txq_vec_setup (txq, socket_id) ==0 ) { +ixgbe_txq_vec_setup (txq) ==0 ) {PMD_INIT_LOG (INFO ,"Vector tx enabled.\n" ); dev->tx_pkt_burst = ixgbe_xmit_pkts_vec; }
lib/librte_pmd_ixgbe/ixgbe_rxtx.h +1 -13
@@ -96,14 +96,6 @@ struct igb_tx_entry_v { };/** -* continuous entry sequence, gather by the same mempool -*/ -struct igb_tx_entry_seq { -const struct rte_mempool * pool; -uint32_t same_pool; -}; - -/** * Structure associated with each RX queue. */ struct igb_rx_queue {
@@ -190,10 +182,6 @@ struct igb_tx_queue {volatile union ixgbe_adv_tx_desc *tx_ring;uint64_t tx_ring_phys_addr;/**< TX ring DMA address. */ struct igb_tx_entry *sw_ring;/**< virtual address of SW ring. */ -#ifdef RTE_IXGBE_INC_VECTOR -/** continuous tx entry sequence within the same mempool */ -struct igb_tx_entry_seq *sw_ring_seq; -#endif volatile uint32_t *tdt_reg_addr;/**< Address of TDT register. */ uint16_t nb_tx_desc;/**< number of TX descriptors. */ uint16_t tx_tail;/**< current value of TDT reg. */
@@ -258,7 +246,7 @@ struct ixgbe_txq_ops {#ifdef RTE_IXGBE_INC_VECTOR uint16_t ixgbe_recv_pkts_vec (void *rx_queue,struct rte_mbuf **rx_pkts,uint16_t nb_pkts);uint16_t ixgbe_xmit_pkts_vec (void *tx_queue,struct rte_mbuf **tx_pkts,uint16_t nb_pkts); -int ixgbe_txq_vec_setup (struct igb_tx_queue *txq,unsigned int socket_id); +int ixgbe_txq_vec_setup (struct igb_tx_queue *txq);int ixgbe_rxq_vec_setup (struct igb_rx_queue *rxq);int ixgbe_rx_vec_condition_check (struct rte_eth_dev *dev);#endif
lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +34 -58
187 lines not on this page · raw message
Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): 1410441347-22840-13-git-send-email-bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
PATCH v3 12/13 ixgbe: Fix perf regression due to moved pool ptr
Adjust the fast-path code to fix the regression caused by the pool pointer moving to the second cache line. This change adjusts the prefetching and also the way in which the mbufs are freed back to the mempool.
Note: slow-path e.g. path supporting jumbo frames, is still slower, but is dealt with by a later commit
Updates in V2:
* fixup checkpatch issue
Updates in V3:
* The variable definitions for freeing mbufs now need to be included whether or not reference counting is enabled.
--- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 8 ++- lib/librte_pmd_ixgbe/ixgbe_rxtx.h | 14 +----- lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c | 94 +++++++++++++---------------------- 3 files changed, 38 insertions(+), 78 deletions(-)
lib/librte_pmd_ixgbe/ixgbe_rxtx.c +3 -5
@@ -142,10 +142,6 @@ ixgbe_tx_free_bufs(struct igb_tx_queue *txq)*/ txep = &(txq->sw_ring[txq->tx_next_dd - (txq->tx_rs_thresh -1 )]); -/* prefetch the mbufs that are about to be freed */ -for (i =0 ; i < txq->tx_rs_thresh; ++i) -rte_prefetch0 ((txep + i)->mbuf); -/* free buffers one at a time */ if ((txq->txq_flags & (uint32_t )ETH_TXQ_FLAGS_NOREFCOUNT ) !=0 ) {for (i =0 ; i < txq->tx_rs_thresh; ++i, ++txep) {
@@ -186,6 +182,7 @@ tx4(volatile union ixgbe_adv_tx_desc *txdp, struct rte_mbuf **pkts) ((uint32_t )DCMD_DTYP_FLAGS | pkt_len); txdp->read.olinfo_status = (pkt_len <<IXGBE_ADVTXD_PAYLEN_SHIFT ); +rte_prefetch0 (&(*pkts)->pool); } }
@@ -205,6 +202,7 @@ tx1(volatile union ixgbe_adv_tx_desc *txdp, struct rte_mbuf **pkts) ((uint32_t )DCMD_DTYP_FLAGS | pkt_len); txdp->read.olinfo_status = (pkt_len <<IXGBE_ADVTXD_PAYLEN_SHIFT ); +rte_prefetch0 (&(*pkts)->pool); }/*
@@ -1875,7 +1873,7 @@ ixgbe_dev_tx_queue_setup(struct rte_eth_dev *dev,PMD_INIT_LOG (INFO ,"Using simple tx code path\n" );#ifdef RTE_IXGBE_INC_VECTOR if (txq->tx_rs_thresh <=RTE_IXGBE_TX_MAX_FREE_BUF_SZ && -ixgbe_txq_vec_setup (txq, socket_id) ==0 ) { +ixgbe_txq_vec_setup (txq) ==0 ) {PMD_INIT_LOG (INFO ,"Vector tx enabled.\n" ); dev->tx_pkt_burst = ixgbe_xmit_pkts_vec; }
lib/librte_pmd_ixgbe/ixgbe_rxtx.h +1 -13
@@ -96,14 +96,6 @@ struct igb_tx_entry_v { };/** -* continuous entry sequence, gather by the same mempool -*/ -struct igb_tx_entry_seq { -const struct rte_mempool * pool; -uint32_t same_pool; -}; - -/** * Structure associated with each RX queue. */ struct igb_rx_queue {
@@ -190,10 +182,6 @@ struct igb_tx_queue {volatile union ixgbe_adv_tx_desc *tx_ring;uint64_t tx_ring_phys_addr;/**< TX ring DMA address. */ struct igb_tx_entry *sw_ring;/**< virtual address of SW ring. */ -#ifdef RTE_IXGBE_INC_VECTOR -/** continuous tx entry sequence within the same mempool */ -struct igb_tx_entry_seq *sw_ring_seq; -#endif volatile uint32_t *tdt_reg_addr;/**< Address of TDT register. */ uint16_t nb_tx_desc;/**< number of TX descriptors. */ uint16_t tx_tail;/**< current value of TDT reg. */
@@ -258,7 +246,7 @@ struct ixgbe_txq_ops {#ifdef RTE_IXGBE_INC_VECTOR uint16_t ixgbe_recv_pkts_vec (void *rx_queue,struct rte_mbuf **rx_pkts,uint16_t nb_pkts);uint16_t ixgbe_xmit_pkts_vec (void *tx_queue,struct rte_mbuf **tx_pkts,uint16_t nb_pkts); -int ixgbe_txq_vec_setup (struct igb_tx_queue *txq,unsigned int socket_id); +int ixgbe_txq_vec_setup (struct igb_tx_queue *txq);int ixgbe_rxq_vec_setup (struct igb_rx_queue *rxq);int ixgbe_rx_vec_condition_check (struct rte_eth_dev *dev);#endif
lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +34 -60
192 lines not on this page · raw message
Signature · 1 line
1.9.3
PATCH v2 13/13 ixgbe: Improve slow-path perf: vector scattered RX
In reply to a message not in the archiveProvide a wrapper routine to enable receive of scattered packets with a vector driver. This improves the performance of the slow-path RX.
Updates in V2:
* Fixed up some checkpatch errors and warnings
--- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 16 ++++ lib/librte_pmd_ixgbe/ixgbe_rxtx.h | 8 +- lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c | 171 ++++++++++++++++++++++++++++++++-- 3 files changed, 186 insertions(+), 9 deletions(-)
lib/librte_pmd_ixgbe/ixgbe_rxtx.c +16 -0
@@ -3476,12 +3476,20 @@ ixgbe_dev_rx_init(struct rte_eth_dev *dev)if ((dev->data->dev_conf.rxmode.max_rx_pkt_len +2 *IXGBE_VLAN_TAG_SIZE ) > buf_size){ dev->data->scattered_rx =1 ; +#ifdef RTE_IXGBE_INC_VECTOR + dev->rx_pkt_burst = ixgbe_recv_scattered_pkts_vec; +#else dev->rx_pkt_burst = ixgbe_recv_scattered_pkts; +#endif } }if (dev->data->dev_conf.rxmode.enable_scatter) { +#ifdef RTE_IXGBE_INC_VECTOR + dev->rx_pkt_burst = ixgbe_recv_scattered_pkts_vec; +#else dev->rx_pkt_burst = ixgbe_recv_scattered_pkts; +#endif dev->data->scattered_rx =1 ; }
@@ -3969,12 +3977,20 @@ ixgbevf_dev_rx_init(struct rte_eth_dev *dev)if ((dev->data->dev_conf.rxmode.max_rx_pkt_len +2 *IXGBE_VLAN_TAG_SIZE ) > buf_size) { dev->data->scattered_rx =1 ; +#ifdef RTE_IXGBE_INC_VECTOR + dev->rx_pkt_burst = ixgbe_recv_scattered_pkts_vec; +#else dev->rx_pkt_burst = ixgbe_recv_scattered_pkts; +#endif } }if (dev->data->dev_conf.rxmode.enable_scatter) { +#ifdef RTE_IXGBE_INC_VECTOR + dev->rx_pkt_burst = ixgbe_recv_scattered_pkts_vec; +#else dev->rx_pkt_burst = ixgbe_recv_scattered_pkts; +#endif dev->data->scattered_rx =1 ; }
lib/librte_pmd_ixgbe/ixgbe_rxtx.h +6 -2
@@ -244,8 +244,12 @@ struct ixgbe_txq_ops { IXGBE_ADVTXD_DCMD_EOP)#ifdef RTE_IXGBE_INC_VECTOR -uint16_t ixgbe_recv_pkts_vec (void *rx_queue,struct rte_mbuf **rx_pkts,uint16_t nb_pkts); -uint16_t ixgbe_xmit_pkts_vec (void *tx_queue,struct rte_mbuf **tx_pkts,uint16_t nb_pkts); +uint16_t ixgbe_recv_pkts_vec (void *rx_queue,struct rte_mbuf **rx_pkts, +uint16_t nb_pkts); +uint16_t ixgbe_recv_scattered_pkts_vec (void *rx_queue, +struct rte_mbuf **rx_pkts,uint16_t nb_pkts); +uint16_t ixgbe_xmit_pkts_vec (void *tx_queue,struct rte_mbuf **tx_pkts, +uint16_t nb_pkts);int ixgbe_txq_vec_setup (struct igb_tx_queue *txq);int ixgbe_rxq_vec_setup (struct igb_rx_queue *rxq);int ixgbe_rx_vec_condition_check (struct rte_eth_dev *dev);
lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +164 -7
221 lines not on this page · raw message
Signature · 1 line
1.9.3
2014-09-11 14:15, Bruce Richardson:
Quoted 38 lines
This patch set continues on from the changes in part 1, and depends upon that patch set.
This patch set reorders the fields in the mbuf structure and splits the structure across two cache lines, given lots of new space for new fields to be added. This set uses some of that space by expanding the
ol_flagsfield. A part 3 patchset is planned to introduce some other new fields into the new mbuf structure.With the splitting of the mbuf across multiple cache lines, performance degradations are seen inside the drivers, both fast-path and slow path. For the fast-path, this patchset reworks the way in which the pool pointer is used to free packets post-TX, which removes the perf regression. For the slow path, an alternative approach is taken - a new scattered packets RX function is introduced into the vector PMD. Using this function, throughput for the slow path RX-TX using testpmd is increased by up to 20% over the original baseline.
Changes in V2:
* General minor updates follow comments on V1 set
* Updated a number of patches to include KNI mbuf changes where needed
* Deferred the patch to add the packet type field to a future patch set
* After removing meta-data element from mbuf structure also added in patch to move the macros torte_port/rte_port.hBruce Richardson (11):
mbuf: reorder fields by time of use
mbuf: expandol_flagsfield to 64-bits
mbuf: introduce a flag to indicate a control mbuf
mbuf: minor changes for readability
mbuf: use macros only to access the mbuf metadata
mbuf: add named points inside the mbuf structure
ixgbe: rework vector pmd following mbuf changes
mbuf: split mbuf across two cache lines.
mbuf: movel2_lenandl3_lento second cache line
ixgbe: Fix perf regression due to moved pool ptr
ixgbe: Improve slow-path perf: vector scattered RX
Quoted 2 lines
Olivier Matz (1):
mbuf: replace data pointer by an offset
Applied for version 1.8.0.
Note that ixgbe patches are not reviewed but I make an exception here, because Intel is responsible of ixgbe performances and we have some time to fix it (if needed) before the release.
Signature · 1 line
Thomas