examples/ioat: create sample app on ioat driver usage
PATCH 1/1 examples/ioat: create sample app on ioat driver usage
From: Pawel Modrak <[email protected]>
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 691 +++++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1010 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 13 + examples/meson.build | 1 + 8 files changed, 1777 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
doc/guides/sample_app_ug/index.rst +1 -0
@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtual
doc/guides/sample_app_ug/intro.rst +4 -0
@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.
doc/guides/sample_app_ug/ioat.rst new file +691 -0
692 lines not on this page · raw message
examples/Makefile +3 -0
@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify DIRS-y += flow_filtering DIRS-y += helloworld +ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y) +DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat +endif DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline ifeq ($(CONFIG_RTE_LIBRTE_LPM),y) DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
examples/ioat/Makefile new file +54 -0
54 lines not on this page · raw message
examples/ioat/ioatfwd.c new file +1010 -0
1,010 lines not on this page · raw message
examples/ioat/meson.build new file +13 -0
@@ -0,0 +1,13 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +deps += ['pmd_ioat'] + +sources = files( + 'ioatfwd.c' +)
examples/meson.build +1 -0
@@ -16,6 +16,7 @@ all_examples = [ 'eventdev_pipeline', 'exception_path', 'fips_validation', 'flow_classify', 'flow_filtering', 'helloworld', + 'ioat', 'ip_fragmentation', 'ip_pipeline', 'ip_reassembly', 'ipsec-secgw', 'ipv4_multicast', 'kni',
Signature · 1 line
2.22.0.windows.1
examples/meson.build
Quoted 1,863 lines · Marcin Baran
From: Pawel Modrak <[email protected]>
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
Signed-off-by Pawel Modrak <[email protected]>Signed-off-by Marcin Baran <[email protected]>--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 691 +++++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1010 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 13 + examples/meson.build | 1 + 8 files changed, 1777 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.builddoc/guides/sample_app_ug/index.rst +1
-0@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtualdoc/guides/sample_app_ug/intro.rst +4
-0@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.doc/guides/sample_app_ug/ioat.rst new file +691
-0@@ -0,0 +1,691 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Intel Corporation. + +Sample Application of packet copying using Intel\|reg| QuickData Technology +============================================================================ + +Overview +-------- + +This sample is intended as a demonstration of the basic components of a DPDK +forwarding application and example of how to use IOAT driver API to make +packets copies. + +Also while forwarding, the MAC addresses are affected as follows: + +* The source MAC address is replaced by the TX port MAC address + +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID + +This application can be used to compare performance of using software packet +copy with copy done using a DMA device for different sizes of packets. +The example will print out statistics each second. The stats shows +received/send packets and packets dropped or failed to copy. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``ioat`` sub-directory. + + +Running the Application +----------------------- + +In order to run the hardware copy application, the copying device +needs to be bound to user-space IO driver. + +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData Technology* +guide for information on using the driver. + +The application requires a number of command line options: + +.. code-block:: console + + ./build/ioatfwd [EAL options] -- -p MASK [-C CT] [--[no-]mac-updating] + +where, + +* p MASK: A hexadecimal bitmask of the ports to configure + +* c CT: Performed packet copy type: software (sw) or hardware using + DMA (rawdev) + +* s RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for + software copy mode + +* --[no-]mac-updating: Whether MAC address of packets should be changed + or not + +The application can be launched in 2 different configurations: + +* Performing software packet copying + +* Performing hardware packet copying + +Each port needs 2 lcores: one of them receives incoming traffic and makes +a copy of each packet. The second lcore then updates MAC address and sends +the copy. For each configuration an additional lcore is needed since +master lcore in use which is responsible for configuration, statistics +printing and safe deinitialization of all ports and devices. + +The application can use a maximum of 8 ports. + +To run the application in a Linux environment with 3 lcores (one of them +is master lcore), 1 port (port 0), software copying and MAC updating issue +the command: + +.. code-block:: console + + $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw + +To run the application in a Linux environment with 5 lcores (one of them +is master lcore), 2 ports (ports 0 and 1), hardware copying and no MAC +updating issue the command: + +.. code-block:: console + + $ ./build/ioatfwd -l 0-4 -n 1 -- -p 0x3 --no-mac-updating -c rawdev + +Refer to the *DPDK Getting Started Guide* for general information on +running applications and the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +The following sections provide an explanation of the main components of the +code. + +All DPDK library functions used in the sample code are prefixed with +``rte_`` and are explained in detail in the *DPDK API Documentation*. + + +The Main Function +~~~~~~~~~~~~~~~~~ + +The ``main()`` function performs the initialization and calls the execution +threads for each lcore. + +The first task is to initialize the Environment Abstraction Layer (EAL). +The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()`` +function. The value returned is the number of parsed arguments: + +.. code-block:: c + + /* init EAL */ + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n"); + + +The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers) +used by the application: + +.. code-block:: c + + nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd + + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE), + MIN_POOL_SIZE); + + /* Create the mbuf pool */ + ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs, + MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE, + rte_socket_id()); + if (ioat_pktmbuf_pool == NULL) + rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n"); + +Mbufs are the packet buffer structure used by DPDK. They are explained in +detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*. + +The ``main()`` function also initializes the ports: + +.. code-block:: c + + /* Initialise each port */ + RTE_ETH_FOREACH_DEV(portid) { + port_init(portid, ioat_pktmbuf_pool); + } + +Each port is configured using ``port_init()``: + +.. code-block:: c + + static inline void + port_init(uint16_t portid, struct rte_mempool *mbuf_pool) + { + struct rte_eth_rxconf rxq_conf; + struct rte_eth_txconf txq_conf; + struct rte_eth_conf local_port_conf = port_conf; + struct rte_eth_dev_info dev_info; + int ret; + + /* Skip ports that are not enabled */ + if ((ioat_enabled_port_mask & (1 << portid)) == 0) { + printf("Skipping disabled port %u\n", portid); + return; + } + + /* Init port */ + printf("Initializing port %u... ", portid); + fflush(stdout); + rte_eth_dev_info_get(portid, &dev_info); + if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE) + local_port_conf.txmode.offloads |= + DEV_TX_OFFLOAD_MBUF_FAST_FREE; + ret = rte_eth_dev_configure(portid, 1, 1, &local_port_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Cannot configure device: err=%d, port=%u\n", + ret, portid); + + ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd, + &nb_txd); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot adjust number of descriptors: err=%d, port=%u\n", + ret, portid); + + rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]); + + /* Init one RX queue */ + fflush(stdout); + rxq_conf = dev_info.default_rxconf; + rxq_conf.offloads = local_port_conf.rxmode.offloads; + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, + rte_eth_dev_socket_id(portid), + &rxq_conf, + mbuf_pool); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup:err=%d, port=%u\n", + ret, portid); + + /* Init one TX queue on each port */ + fflush(stdout); + txq_conf = dev_info.default_txconf; + txq_conf.offloads = local_port_conf.txmode.offloads; + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, + rte_eth_dev_socket_id(portid), + &txq_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_tx_queue_setup:err=%d, port=%u\n", + ret, portid); + + /* Initialize TX buffers */ + tx_buffer[portid] = rte_zmalloc_socket("tx_buffer", + RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0, + rte_eth_dev_socket_id(portid)); + if (tx_buffer[portid] == NULL) + rte_exit(EXIT_FAILURE, "Cannot allocate buffer for tx " + "on port %u\n", portid); + + rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST); + + ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid], + rte_eth_tx_buffer_count_callback, + &port_statistics[portid].tx_dropped); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot set error callback for tx buffer on port %u\n", + portid); + + /* Start device */ + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n", + ret, portid); + + rte_eth_promiscuous_enable(portid); + + printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n", + portid, + ioat_ports_eth_addr[portid].addr_bytes[0], + ioat_ports_eth_addr[portid].addr_bytes[1], + ioat_ports_eth_addr[portid].addr_bytes[2], + ioat_ports_eth_addr[portid].addr_bytes[3], + ioat_ports_eth_addr[portid].addr_bytes[4], + ioat_ports_eth_addr[portid].addr_bytes[5]); + } + +The Ethernet ports are configured with local settings using the +``rte_eth_dev_configure()`` function and the ``port_conf`` struct: + +.. code-block:: c + + static struct rte_eth_conf port_conf = { + .rxmode = { + .max_rx_pkt_len = RTE_ETHER_MAX_LEN, + }, + }; + +For this example the ports are set up with 1 RX and 1 TX queue using the +``rte_eth_rx_queue_setup()`` and ``rte_eth_tx_queue_setup()`` functions. + +The Ethernet port is then started: + +.. code-block:: c + + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n", + ret, portid); + + +Finally the RX port is set in promiscuous mode: + +.. code-block:: c + + rte_eth_promiscuous_enable(portid); + + +After that each port application assigns resources needed. + +.. code-block:: c + + check_link_status(ioat_enabled_port_mask); + + if (!cfg.nb_ports) { + rte_exit(EXIT_FAILURE, + "All available ports are disabled. Please set portmask.\n"); + } + + /* Check if there is enough lcores for all ports. */ + cfg.nb_lcores = rte_lcore_count() - 1; + if (cfg.nb_lcores < 1) + rte_exit(EXIT_FAILURE, + "There should be at least one slave lcore.\n"); + + ret = 0; + + if (copy_mode == COPY_MODE_IOAT_NUM) { + assign_rawdevs(); + } else /* copy_mode == COPY_MODE_SW_NUM */ { + assign_rings(); + } + +A link status is checked of each port enabled by port mask +using ``check_link_status()`` function. + +.. code-block:: c + + /* Check the link status of all ports in up to 9s, and print them finally */ + static void + check_link_status(uint32_t port_mask) + { + + uint16_t portid; + struct rte_eth_link link; + + cfg.nb_ports = 0; + + printf("\nChecking link status\n"); + fflush(stdout); + RTE_ETH_FOREACH_DEV(portid) { + if (force_quit) + return; + if ((port_mask & (1 << portid)) == 0) + continue; + + store_port_nb(portid); + + memset(&link, 0, sizeof(link)); + rte_eth_link_get(portid, &link); + + /* Print link status */ + if (link.link_status) { + printf( + "Port %d Link Up. Speed %u Mbps - %s\n", + portid, link.link_speed, + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + ("full-duplex") : ("half-duplex\n")); + } + else + printf("Port %d Link Down\n", portid); + } + } + +Depending on mode set (whether copy should be done by software or by hardware) +special structures are assigned to each port. If software copy was chosen, +application have to assign ring structures for packet exchanging between lcores +assigned to ports. + +.. code-block:: c + + static void + assign_rings(void) + { + uint32_t i; + + for (i = 0; i < cfg.nb_ports; i++) { + char ring_name[20]; + + snprintf(ring_name, 20, "rx_to_tx_ring_%u", i); + /* Create ring for inter core communication */ + cfg.ports[i].rx_to_tx_ring = rte_ring_create( + ring_name, ring_size, + rte_socket_id(), RING_F_SP_ENQ); + + if (cfg.ports[i].rx_to_tx_ring == NULL) + rte_exit(EXIT_FAILURE, "%s\n", + rte_strerror(rte_errno)); + } + } + + +When using hardware copy each port is assigned an IOAT device +(``assign_rawdevs()``) using IOAT Rawdev Driver API functions: + +.. code-block:: c + + static void + assign_rawdevs(void) + { + uint16_t nb_rawdev = 0; + uint32_t i; + + for (i = 0; i < cfg.nb_ports; i++) { + struct rte_rawdev_info rdev_info = {0}; + rte_rawdev_info_get(0, &rdev_info); + + if (strcmp(rdev_info.driver_name, "rawdev_ioat") == 0) { + configure_rawdev_queue(i); + cfg.ports[i].dev_id = i; + ++nb_rawdev; + } + } + + RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev); + + if (nb_rawdev < cfg.nb_ports) + rte_exit(EXIT_FAILURE, "Not enough IOAT rawdevs (%u) for ports (%u).\n", + nb_rawdev, cfg.nb_ports); + } + + +The initialization of hardware device is done by ``rte_rawdev_configure()`` +function and ``rte_rawdev_info`` struct. After configuration the device is +started using ``rte_rawdev_start()`` function. Each of the above operations +is done in ``configure_rawdev_queue()``. + +.. code-block:: c + + static void + configure_rawdev_queue(uint32_t dev_id) + { + struct rte_rawdev_info info = { .dev_private = &dev_config }; + + /* Configure hardware copy device */ + dev_config.ring_size = ring_size; + + if (rte_rawdev_configure(dev_id, &info) != 0) { + rte_exit(EXIT_FAILURE, + "Error with rte_rawdev_configure()\n"); + } + rte_rawdev_info_get(dev_id, &info); + if (dev_config.ring_size != ring_size) { + rte_exit(EXIT_FAILURE, + "Error, ring size is not %d (%d)\n", + ring_size, (int)dev_config.ring_size); + } + if (rte_rawdev_start(dev_id) != 0) { + rte_exit(EXIT_FAILURE, + "Error with rte_rawdev_start()\n"); + } + } + +If initialization is successful memory for hardware device +statistics is allocated. + +Finally ``main()`` functions starts all processing lcores and starts +printing stats in a loop on master lcore. The application can be +interrupted and closed using ``Ctrl-C``. The master lcore waits for +all slave processes to finish, deallocates resources and exits. + +The processing lcores launching function are described below. + +The Lcores Launching Functions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As described above ``main()`` function invokes ``run_transmission()`` +function in order to start processing for each lcore: + +.. code-block:: c + + static void run_transmission(void) + { + uint32_t lcore_id = rte_lcore_id(); + + RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n", + __func__, rte_lcore_id()); + + if (cfg.nb_lcores == 1) { + lcore_id = rte_get_next_lcore(lcore_id, true, true); + rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop, NULL, lcore_id); + } else if (cfg.nb_lcores > 1) { + lcore_id = rte_get_next_lcore(lcore_id, true, true); + rte_eal_remote_launch((lcore_function_t *)rx_main_loop, NULL, lcore_id); + + lcore_id = rte_get_next_lcore(lcore_id, true, true); + rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL, lcore_id); + } + } + +The function launches rx/tx processing functions on configured lcores +for each port using ``rte_eal_remote_launch()``. The configured ports, +their number and number of assigned lcores are stored in user-defined +``rxtx_transmission_config`` struct that is initialized before launching +tasks: + +.. code-block:: c + + struct rxtx_transmission_config { + struct rxtx_port_config ports[RTE_MAX_ETHPORTS]; + uint16_t nb_ports; + uint16_t nb_lcores; + }; + +The Lcores Processing Functions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For receiving packets on each port an ``ioat_rx_port()`` function is used. +Depending on mode the user chose, it will enqueue packets to IOAT rawdev +and then invoke copy process (hardware copy), or perform software copy +of each packet using ``pktmbuf_sw_copy()`` function and enqueue them to +rte_ring: + +.. code-block:: c + + /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ + static void + ioat_rx_port(struct rxtx_port_config *rx_config) + { + uint32_t nb_rx, nb_enq, i; + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; + + nb_rx = rte_eth_rx_burst(rx_config->rx_portId, 0, + pkts_burst, MAX_PKT_BURST); + + if (nb_rx == 0) + return; + + port_statistics[rx_config->rx_portId].rx += nb_rx; + + if (copy_mode == COPY_MODE_IOAT_NUM) { + /* Perform packet hardware copy */ + nb_enq = ioat_enqueue_packets(rx_config, + pkts_burst, nb_rx); + + if (nb_enq > 0) + rte_ioat_do_copies(rx_config->dev_id); + } else { + /* Perform packet software copy, free source packets */ + int ret; + struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST]; + + ret = rte_pktmbuf_alloc_bulk(ioat_pktmbuf_pool, + pkts_burst_copy, nb_rx); + + if (unlikely(ret < 0)) + rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n"); + + for (i = 0; i < nb_rx; i++) { + pktmbuf_sw_copy(pkts_burst[i], pkts_burst_copy[i]); + rte_pktmbuf_free(pkts_burst[i]); + } + + nb_enq = rte_ring_enqueue_burst(rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx, NULL); + + /* Free any not enqueued packets. */ + for (i = nb_enq; i < nb_rx; i++) + rte_pktmbuf_free(pkts_burst_copy[i]); + } + + port_statistics[rx_config->rx_portId].copy_dropped + += (nb_rx - nb_enq); + } + +The packets are received in burst mode using ``rte_eth_rx_burst()`` +function. When using hardware copy mode the packets are enqueued in +copying device's buffer using ``ioat_enqueue_packets()`` which calls +``rte_ioat_enqueue_copy()``. When all received packets are in the +buffer the copies are invoked by calling ``rte_ioat_do_copies()``. +Function ``rte_ioat_enqueue_copy()`` operates on physical address of +the packet. Structure ``rte_mbuf`` contains only physical address to +start of the data buffer (``buf_iova``). Thus the address is shifted +by ``addr_offset`` value in order to get pointer to ``rearm_data`` +member of ``rte_mbuf``. That way the packet is copied all at once +(with data and metadata). + +.. code-block:: c + + static uint32_t + ioat_enqueue_packets(struct rxtx_port_config *rx_config, + struct rte_mbuf **pkts, uint32_t nb_rx) + { + int ret; + uint32_t i; + struct rte_mbuf *pkts_copy[MAX_PKT_BURST]; + + const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr, + &pkts[0]->rearm_data); + + ret = rte_pktmbuf_alloc_bulk(ioat_pktmbuf_pool, pkts_copy, nb_rx); + + if (unlikely(ret < 0)) + rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n"); + + for (i = 0; i < nb_rx; i++) { + /* Perform data copy */ + ret = rte_ioat_enqueue_copy(rx_config->dev_id, + pkts[i]->buf_iova + - addr_offset, + pkts_copy[i]->buf_iova + - addr_offset, + rte_pktmbuf_data_len(pkts[i]) + + addr_offset, + (uintptr_t)pkts[i], + (uintptr_t)pkts_copy[i], + 0 /* nofence */); + + if (ret != 1) + break; + } + + ret = i; + /* Free any not enqueued packets. */ + for (; i < nb_rx; i++) { + rte_pktmbuf_free(pkts[i]); + rte_pktmbuf_free(pkts_copy[i]); + } + + return ret; + } + + +All done copies are processed by ``ioat_tx_port()`` function. When using +hardware copy mode the function invokes ``rte_ioat_completed_copies()`` +to gather copied packets. If software copy mode is used the function +dequeues copied packets from rte_ring. Then each packet MAC address +is changed if it was enabled. After that copies are sent in burst mode +using `` rte_eth_tx_burst()``. + + +.. code-block:: c + + /* Transmit packets from IOAT rawdev/rte_ring for one port. */ + static void + ioat_tx_port(struct rxtx_port_config *tx_config) + { + uint32_t i, nb_dq; + struct rte_mbuf *mbufs_src[MAX_PKT_BURST]; + struct rte_mbuf *mbufs_dst[MAX_PKT_BURST]; + + if (copy_mode == COPY_MODE_IOAT_NUM) { + /* Deque the mbufs from IOAT device. */ + nb_dq = rte_ioat_completed_copies(tx_config->dev_id, + MAX_PKT_BURST, (void *)mbufs_src, (void *)mbufs_dst); + } else { + /* Deque the mbufs from rx_to_tx_ring. */ + nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring, + (void *)mbufs_dst, MAX_PKT_BURST, NULL); + } + + if (nb_dq == 0) + return; + + /* Free source packets */ + if (copy_mode == COPY_MODE_IOAT_NUM) { + for (i = 0; i < nb_dq; i++) + rte_pktmbuf_free(mbufs_src[i]); + } + + /* Update macs if enabled */ + if (mac_updating) { + for (i = 0; i < nb_dq; i++) + update_mac_addrs(mbufs_dst[i], + tx_config->tx_portId); + } + + const uint16_t nb_tx = rte_eth_tx_burst(tx_config->tx_portId, + 0, (void *)mbufs_dst, nb_dq); + + port_statistics[tx_config->tx_portId].tx += nb_tx; + + /* Free any unsent packets. */ + if (unlikely(nb_tx < nb_dq)) { + for (i = nb_tx; i < nb_dq; i++) + rte_pktmbuf_free(mbufs_dst[i]); + } + } + +The Packet Copying Functions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In order to perform packet copy there is a user-defined function +``pktmbuf_sw_copy()`` used. It copies a whole packet by copying +metadata from source packet to new mbuf, and then copying a data +chunk of source packet. Both memory copies are done using +``rte_memcpy()``: + +.. code-block:: c + + static inline void + pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) + { + /* Copy packet metadata */ + rte_memcpy(&dst->rearm_data, + &src->rearm_data, + offsetof(struct rte_mbuf, cacheline1) + - offsetof(struct rte_mbuf, rearm_data)); + + /* Copy packet data */ + rte_memcpy(rte_pktmbuf_mtod(dst, char *), + rte_pktmbuf_mtod(src, char *), src->data_len); + } + +The metadata in this example is copied from ``rearm_data`` member of +``rte_mbuf`` struct up to ``cacheline1``. + +In order to understand why software packet copying is done as shown +above please refer to the "Mbuf Library" section of the +*DPDK Programmer's Guide*.examples/Makefile +3
-0@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify DIRS-y += flow_filtering DIRS-y += helloworld +ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y) +DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat +endif DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline ifeq ($(CONFIG_RTE_LIBRTE_LPM),y) DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassemblyexamples/ioat/Makefile new file +54
-0@@ -0,0 +1,54 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +# binary name +APP = ioatfwd + +# all source are stored in SRCS-y +SRCS-y := ioatfwd.c + +# Build using pkg-config variables if possible +ifeq ($(shell pkg-config --exists libdpdk && echo 0),0) + +all: shared +.PHONY: shared static +shared: build/$(APP)-shared + ln -sf $(APP)-shared build/$(APP) +static: build/$(APP)-static + ln -sf $(APP)-static build/$(APP) + +PC_FILE := $(shell pkg-config --path libdpdk) +CFLAGS += -O3 $(shell pkg-config --cflags libdpdk) +LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk) +LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk) + +build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build + $(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED) + +build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build + $(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC) + +build: + @mkdir -p $@ + +.PHONY: clean +clean: + rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared + test -d build && rmdir -p build || true + +else # Build using legacy build system +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, detect a build directory, by looking for a path with a .config +RTE_TARGET ?= $(notdir $(abspath $(dir $(firstword $(wildcard $(RTE_SDK)/*/.config))))) + +include $(RTE_SDK)/mk/rte.vars.mk + + +CFLAGS += -O3 +CFLAGS += $(WERROR_FLAGS) + +include $(RTE_SDK)/mk/rte.extapp.mk +endifexamples/ioat/ioatfwd.c new file +1010
-0@@ -0,0 +1,1010 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2019 Intel Corporation +*/ + +#include <stdint.h> +#include <getopt.h> +#include <signal.h> +#include <stdbool.h> +#include <unistd.h> + +#include <rte_malloc.h> +#include <rte_ethdev.h> +#include <rte_rawdev.h> +#include <rte_ioat_rawdev.h> + +/* size of ring used for software copying between rx and tx. */ +#define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1 +#define MAX_PKT_BURST 32 +#define MEMPOOL_CACHE_SIZE 512 +#define MIN_POOL_SIZE 65536U +#define CMD_LINE_OPT_MAC_UPDATING "mac-updating" +#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating" +#define CMD_LINE_OPT_PORTMASK "portmask" +#define CMD_LINE_OPT_NB_QUEUE "nb-queue" +#define CMD_LINE_OPT_COPY_TYPE "copy-type" +#define CMD_LINE_OPT_RING_SIZE "ring-size" + +/* configurable number of RX/TX ring descriptors */ +#define RX_DEFAULT_RINGSIZE 1024 +#define TX_DEFAULT_RINGSIZE 1024 + +/* max number of RX queues per port */ +#define MAX_RX_QUEUES_COUNT 8 + +struct rxtx_port_config { +/* common config */ +uint16_t rxtx_port; +uint16_t nb_queues; +/* for software copy mode */ +struct rte_ring *rx_to_tx_ring; +/* for IOAT rawdev copy mode */ +uint16_t ioat_ids[MAX_RX_QUEUES_COUNT ]; +}; + +struct rxtx_transmission_config { +struct rxtx_port_config ports[RTE_MAX_ETHPORTS ]; +uint16_t nb_ports; +uint16_t nb_lcores; +}; + +/* per-port statistics struct */ +struct ioat_port_statistics { +uint64_t rx[RTE_MAX_ETHPORTS ]; +uint64_t tx[RTE_MAX_ETHPORTS ]; +uint64_t tx_dropped[RTE_MAX_ETHPORTS ]; +uint64_t copy_dropped[RTE_MAX_ETHPORTS ]; +}; +struct ioat_port_statistics port_statistics; + +struct total_statistics { +uint64_t total_packets_dropped; +uint64_t total_packets_tx; +uint64_t total_packets_rx; +uint64_t total_successful_enqueues; +uint64_t total_failed_enqueues; +}; + +typedef enum copy_mode_t { +#define COPY_MODE_SW "sw" + COPY_MODE_SW_NUM, +#define COPY_MODE_IOAT "rawdev" + COPY_MODE_IOAT_NUM, +COPY_MODE_INVALID_NUM , +COPY_MODE_SIZE_NUM =COPY_MODE_INVALID_NUM +}copy_mode_t ; + +/* mask of enabled ports */ +static uint32_t ioat_enabled_port_mask; + +/* number of RX queues per port */ +static uint16_t nb_queues =1 ; + +/* MAC updating enabled by default. */ +static int mac_updating =1 ; + +/* hardare copy mode enabled by default. */ +static copy_mode_t copy_mode =COPY_MODE_IOAT_NUM ; + +/* size of IOAT rawdev ring for hardware copy mode or +* rte_ring for software copy mode +*/ +static unsigned short ring_size =2048 ; + +/* global transmission config */ +struct rxtx_transmission_config cfg; + +/* configurable number of RX/TX ring descriptors */ +static uint16_t nb_rxd =RX_DEFAULT_RINGSIZE ; +static uint16_t nb_txd =TX_DEFAULT_RINGSIZE ; + +static volatile bool force_quit; + +/* ethernet addresses of ports */ +static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS ]; + +static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ]; +struct rte_mempool *ioat_pktmbuf_pool; + +/* Print out statistics for one port. */ +static void +print_port_stats (uint16_t port_id) +{ +printf ("\nStatistics for port %u ------------------------------" +"\nPackets sent: %34" PRIu64 +"\nPackets received: %30" PRIu64 +"\nPackets dropped on tx: %25" PRIu64 +"\nPackets dropped on copy: %23" PRIu64, + port_id, + port_statistics.tx[port_id], + port_statistics.rx[port_id], + port_statistics.tx_dropped[port_id], + port_statistics.copy_dropped[port_id]); +} + +/* Print out statistics for one IOAT rawdev device. */ +static void +print_rawdev_stats (uint32_t dev_id,uint64_t *xstats, +uint16_t nb_xstats,struct rte_rawdev_xstats_name *names_xstats) +{ +uint16_t i; + +printf ("\nIOAT channel %u" , dev_id); +for (i =0 ; i < nb_xstats; i++) +if (strstr (names_xstats[i].name,"enqueues" )) +printf ("\n\t %s: %*" PRIu64, + names_xstats[i].name, + (int )(37 -strlen (names_xstats[i].name)), + xstats[i]); +} + +static void +print_total_stats (struct total_statistics *ts) +{ +printf ("\nAggregate statistics ===============================" +"\nTotal packets sent: %28" PRIu64 +"\nTotal packets received: %24" PRIu64 +"\nTotal packets dropped: %25" PRIu64, + ts->total_packets_tx, + ts->total_packets_rx, + ts->total_packets_dropped); + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +printf ("\nTotal IOAT successful enqueues: %16" PRIu64 +"\nTotal IOAT failed enqueues: %20" PRIu64, + ts->total_successful_enqueues, + ts->total_failed_enqueues); + } + +printf ("\n====================================================\n" ); +} + +/* Print out statistics on packets dropped. */ +static void +print_stats (char *prgname) +{ +struct total_statistics ts; +uint32_t i, port_id, dev_id; +struct rte_rawdev_xstats_name *names_xstats; +uint64_t *xstats; +unsigned int *ids_xstats; +unsigned int nb_xstats, id_fail_enq, id_succ_enq; +char status_string[120 ];/* to print at the top of the output */ +int status_strlen; + + +const char clr[] = {27 ,'[' ,'2' ,'J' ,'\0' }; +const char topLeft[] = {27 ,'[' ,'1' ,';' ,'1' ,'H' ,'\0' }; + + status_strlen =snprintf (status_string,sizeof (status_string), +"%s, " , prgname); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Worker Threads = %d, " , +rte_lcore_count () >2 ?2 :1 ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Copy Mode = %s,\n" , copy_mode ==COPY_MODE_SW_NUM ? +COPY_MODE_SW :COPY_MODE_IOAT ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Updating MAC = %s, " , mac_updating ? +"enabled" :"disabled" ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Rx Queues = %d, " , nb_queues); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Ring Size = %d\n" , ring_size); + +/* Allocate memory for xstats names and values */ + nb_xstats =rte_rawdev_xstats_names_get ( + cfg.ports[0 ].ioat_ids[0 ],NULL ,0 ); + + names_xstats =malloc (sizeof (*names_xstats) * nb_xstats); +if (names_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat names memory\n" ); + } +rte_rawdev_xstats_names_get (cfg.ports[0 ].ioat_ids[0 ], + names_xstats, nb_xstats); + + ids_xstats =malloc (sizeof (*ids_xstats) * nb_xstats); +if (ids_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat ids_xstats memory\n" ); + } + +for (i =0 ; i < nb_xstats; i++) + ids_xstats[i] = i; + + xstats =malloc (sizeof (*xstats) * nb_xstats); +if (xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat memory\n" ); + } + +/* Get failed/successful enqueues stats index */ + id_fail_enq = id_succ_enq = nb_xstats; +for (i =0 ; i < nb_xstats; i++) { +if (!strcmp (names_xstats[i].name,"failed_enqueues" )) + id_fail_enq = i; +else if (!strcmp (names_xstats[i].name,"successful_enqueues" )) + id_succ_enq = i; +if (id_fail_enq < nb_xstats && id_succ_enq < nb_xstats) +break ; + } +if (id_fail_enq == nb_xstats || id_succ_enq == nb_xstats) { +rte_exit (EXIT_FAILURE , +"Error getting failed/successful enqueues stats index\n" ); + } + +while (!force_quit) { +/* Sleep for 1 second each round - init sleep allows reading +* messages from app startup. +*/ +sleep (1 ); + +/* Clear screen and move to top left */ +printf ("%s%s" , clr, topLeft); + +memset (&ts,0 ,sizeof (struct total_statistics )); + +printf ("%s" , status_string); + +for (i =0 ; i < cfg.nb_ports; i++) { + port_id = cfg.ports[i].rxtx_port; +print_port_stats (port_id); + + ts.total_packets_dropped += + port_statistics.tx_dropped[port_id] + + port_statistics.copy_dropped[port_id]; + ts.total_packets_tx += port_statistics.tx[port_id]; + ts.total_packets_rx += port_statistics.rx[port_id]; + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +uint32_t j; + +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { + dev_id = cfg.ports[i].ioat_ids[j]; +rte_rawdev_xstats_get (dev_id, + ids_xstats, xstats, nb_xstats); + +print_rawdev_stats (dev_id, xstats, + nb_xstats, names_xstats); + + ts.total_successful_enqueues += + xstats[id_succ_enq]; + ts.total_failed_enqueues += + xstats[id_fail_enq]; + } + } + } +printf ("\n" ); + +print_total_stats (&ts); + } + +free (names_xstats); +free (xstats); +free (ids_xstats); +} + +static void +update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) +{ +struct rte_ether_hdr *eth; +void *tmp; + + eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *); + +/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but +* it's acceptable cause it gets overwritten by rte_ether_addr_copy +*/ + tmp = ð->d_addr.addr_bytes[0 ]; + *((uint64_t *)tmp) =0x000000000002 + ((uint64_t )dest_portid <<40 ); + +/* src addr */ +rte_ether_addr_copy (&ioat_ports_eth_addr[dest_portid], ð->s_addr); +} + +static inline void +pktmbuf_sw_copy (struct rte_mbuf *src,struct rte_mbuf *dst) +{ +/* Copy packet metadata */ +rte_memcpy (&dst->rearm_data, + &src->rearm_data, + offsetof(struct rte_mbuf , cacheline1) + - offsetof(struct rte_mbuf , rearm_data)); + +/* Copy packet data */ + rte_memcpy(rte_pktmbuf_mtod(dst, char *), + rte_pktmbuf_mtod(src, char *), src->data_len); +} + +static uint32_t +ioat_enqueue_packets (struct rte_mbuf **pkts, +uint32_t nb_rx,uint16_t dev_id) +{ +int ret; +uint32_t i; +struct rte_mbuf *pkts_copy[MAX_PKT_BURST ]; + +const uint64_t addr_offset =RTE_PTR_DIFF (pkts[0 ]->buf_addr, + &pkts[0 ]->rearm_data); + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE ,"Unable to allocate memory.\n" ); + +for (i =0 ; i < nb_rx; i++) { +/* Perform data copy */ + ret =rte_ioat_enqueue_copy (dev_id, + pkts[i]->buf_iova + - addr_offset, + pkts_copy[i]->buf_iova + - addr_offset, +rte_pktmbuf_data_len (pkts[i]) + + addr_offset, + (uintptr_t )pkts[i], + (uintptr_t )pkts_copy[i], +0 /* nofence */ ); + +if (ret !=1 ) +break ; + } + + ret = i; +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i); +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], + nb_rx - i); + + +return ret; +} + +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ +static void +ioat_rx_port (struct rxtx_port_config *rx_config) +{ +uint32_t nb_rx, nb_enq, i, j; +struct rte_mbuf *pkts_burst[MAX_PKT_BURST ]; + +for (i =0 ; i < rx_config->nb_queues; i++) { + + nb_rx =rte_eth_rx_burst (rx_config->rxtx_port, i, + pkts_burst,MAX_PKT_BURST ); + +if (nb_rx ==0 ) +continue ; + + port_statistics.rx[rx_config->rxtx_port] += nb_rx; + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Perform packet hardware copy */ + nb_enq =ioat_enqueue_packets (pkts_burst, + nb_rx, rx_config->ioat_ids[i]); +if (nb_enq >0 ) +rte_ioat_do_copies (rx_config->ioat_ids[i]); + }else { +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } + + port_statistics.copy_dropped[rx_config->rxtx_port] += + (nb_rx - nb_enq); + } +} + +/* Transmit packets from IOAT rawdev/rte_ring for one port. */ +static void +ioat_tx_port (struct rxtx_port_config *tx_config) +{ +uint32_t i, j, nb_dq =0 ; +struct rte_mbuf *mbufs_src[MAX_PKT_BURST ]; +struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ]; + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Deque the mbufs from IOAT device. */ +for (i =0 ; i < tx_config->nb_queues; i++) { + nb_dq =rte_ioat_completed_copies ( + tx_config->ioat_ids[i],MAX_PKT_BURST , + (void *)mbufs_src, (void *)mbufs_dst); + +if (nb_dq ==0 ) +break ; + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)mbufs_src, nb_dq); + +/* Update macs if enabled */ +if (mac_updating) { +for (j =0 ; j < nb_dq; j++) +update_mac_addrs (mbufs_dst[j], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst ( + tx_config->rxtx_port,0 , + (void *)mbufs_dst, nb_dq); + + port_statistics.tx[tx_config->rxtx_port] += nb_tx; + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } + }else { +/* Deque the mbufs from rx_to_tx_ring. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); + +if (nb_dq ==0 ) +return ; + +/* Update macs if enabled */ +if (mac_updating) { +for (i =0 ; i < nb_dq; i++) +update_mac_addrs (mbufs_dst[i], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, +0 , (void *)mbufs_dst, nb_dq); + + port_statistics.tx[tx_config->rxtx_port] += nb_tx; + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } +} + +/* Main rx processing loop for IOAT rawdev. */ +static void +rx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx loop for copy on lcore %u\n" , +rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) +ioat_rx_port (&cfg.ports[i]); +} + +/* Main tx processing loop for hardware copy. */ +static void +tx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main tx loop for copy on lcore %u\n" , +rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) +ioat_tx_port (&cfg.ports[i]); +} + +/* Main rx and tx loop if only one slave lcore available */ +static void +rxtx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx and tx loop for copy on" +" lcore %u\n" ,rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) { +ioat_rx_port (&cfg.ports[i]); +ioat_tx_port (&cfg.ports[i]); + } +} + +static void start_forwarding_cores (void ) +{ +uint32_t lcore_id =rte_lcore_id (); + +RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , + __func__,rte_lcore_id ()); + +if (cfg.nb_lcores ==1 ) { + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); + }else if (cfg.nb_lcores >1 ) { + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rx_main_loop, +NULL , lcore_id); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)tx_main_loop,NULL , + lcore_id); + } +} + +/* Display usage */ +static void +ioat_usage (const char *prgname) +{ +printf ("%s [EAL options] -- -p PORTMASK [-q NQ]\n" +" -p --portmask: hexadecimal bitmask of ports to configure\n" +" -q NQ: number of RX queues per port (default is 1)\n" +" --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default)\n" +" When enabled:\n" +" - The source MAC address is replaced by the TX port MAC address\n" +" - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n" +" -c --copy-type CT: type of copy: sw|rawdev\n" +" -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n" , + prgname); +} + +static int +ioat_parse_portmask (const char *portmask) +{ +char *end =NULL ; +unsigned long pm; + +/* Parse hexadecimal string */ + pm =strtoul (portmask, &end,16 ); +if ((portmask[0 ] =='\0' ) || (end ==NULL ) || (*end !='\0' )) +return -1 ; + +return pm; +} + +static copy_mode_t +ioat_parse_copy_mode (const char *copy_mode) +{ +if (strcmp (copy_mode,COPY_MODE_SW ) ==0 ) +return COPY_MODE_SW_NUM ; +else if (strcmp (copy_mode,COPY_MODE_IOAT ) ==0 ) +return COPY_MODE_IOAT_NUM ; + +return COPY_MODE_INVALID_NUM ; +} + +/* Parse the argument given in the command line of the application */ +static int +ioat_parse_args (int argc,char **argv,unsigned int nb_ports) +{ +static const char short_options[] = +"p:" /* portmask */ +"q:" /* number of RX queues per port */ +"c:" /* copy type (sw|rawdev) */ +"s:" /* ring size */ + ; + +static const struct option lgopts[] = { + {CMD_LINE_OPT_MAC_UPDATING , no_argument, &mac_updating,1 }, + {CMD_LINE_OPT_NO_MAC_UPDATING , no_argument, &mac_updating,0 }, + {CMD_LINE_OPT_PORTMASK , required_argument,NULL ,'p' }, + {CMD_LINE_OPT_NB_QUEUE , required_argument,NULL ,'q' }, + {CMD_LINE_OPT_COPY_TYPE , required_argument,NULL ,'c' }, + {CMD_LINE_OPT_RING_SIZE , required_argument,NULL ,'s' }, + {NULL ,0 ,0 ,0 } + }; + +const unsigned int default_port_mask = (1 << nb_ports) -1 ; +int opt, ret; +char **argvopt; +int option_index; +char *prgname = argv[0 ]; + + ioat_enabled_port_mask = default_port_mask; + argvopt = argv; + +while ((opt =getopt_long (argc, argvopt, short_options, + lgopts, &option_index)) !=EOF ) { + +switch (opt) { +/* portmask */ +case 'p' : + ioat_enabled_port_mask =ioat_parse_portmask (optarg); +if (ioat_enabled_port_mask & ~default_port_mask || + ioat_enabled_port_mask <=0 ) { +printf ("Invalid portmask, %s, suggest 0x%x\n" , + optarg, default_port_mask); +ioat_usage (prgname); +return -1 ; + } +break ; + +case 'q' : + nb_queues =atoi (optarg); +if (nb_queues ==0 || nb_queues >MAX_RX_QUEUES_COUNT ) { +printf ("Invalid RX queues number %s. Max %u\n" , + optarg,MAX_RX_QUEUES_COUNT ); +ioat_usage (prgname); +return -1 ; + } +break ; + +case 'c' : + copy_mode =ioat_parse_copy_mode (optarg); +if (copy_mode ==COPY_MODE_INVALID_NUM ) { +printf ("Invalid copy type. Use: sw, rawdev\n" ); +ioat_usage (prgname); +return -1 ; + } +break ; + +case 's' : + ring_size =atoi (optarg); +if (ring_size ==0 ) { +printf ("Invalid ring size, %s.\n" , optarg); +ioat_usage (prgname); +return -1 ; + } +break ; + +/* long options */ +case 0 : +break ; + +default : +ioat_usage (prgname); +return -1 ; + } + } + +printf ("MAC updating %s\n" , mac_updating ?"enabled" :"disabled" ); +if (optind >=0 ) + argv[optind-1 ] = prgname; + + ret = optind-1 ; + optind =1 ;/* reset getopt lib */ +return ret; +} + +/* check link status, return true if at least one port is up */ +static int +check_link_status (uint32_t port_mask) +{ +uint16_t portid; +struct rte_eth_link link; +int retval =0 ; + +printf ("\nChecking link status\n" ); +RTE_ETH_FOREACH_DEV (portid) { +if ((port_mask & (1 << portid)) ==0 ) +continue ; + +memset (&link,0 ,sizeof (link)); +rte_eth_link_get (portid, &link); + +/* Print link status */ +if (link.link_status) { +printf ( +"Port %d Link Up. Speed %u Mbps - %s\n" , + portid, link.link_speed, + (link.link_duplex ==ETH_LINK_FULL_DUPLEX ) ? + ("full-duplex" ) : ("half-duplex\n" )); + retval =1 ; + }else +printf ("Port %d Link Down\n" , portid); + } +return retval; +} + +static void +configure_rawdev_queue (uint32_t dev_id) +{ +struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size }; +struct rte_rawdev_info info = { .dev_private = &dev_config }; + +if (rte_rawdev_configure (dev_id, &info) !=0 ) { +rte_exit (EXIT_FAILURE , +"Error with rte_rawdev_configure()\n" ); + } +if (rte_rawdev_start (dev_id) !=0 ) { +rte_exit (EXIT_FAILURE , +"Error with rte_rawdev_start()\n" ); + } +} + +static void +assign_rawdevs (void ) +{ +uint16_t nb_rawdev =0 , rdev_id =0 ; +uint32_t i, j; + +for (i =0 ; i < cfg.nb_ports; i++) { +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { +struct rte_rawdev_info rdev_info = {0 }; + +do { +if (rdev_id ==rte_rawdev_count ()) + goto end; +rte_rawdev_info_get (rdev_id++, &rdev_info); + }while (strcmp (rdev_info.driver_name, +IOAT_PMD_RAWDEV_NAME_STR ) !=0 ); + + cfg.ports[i].ioat_ids[j] = rdev_id -1 ; +configure_rawdev_queue (cfg.ports[i].ioat_ids[j]); + ++nb_rawdev; + } + } +end: +if (nb_rawdev < cfg.nb_ports * cfg.ports[0 ].nb_queues) +rte_exit (EXIT_FAILURE , +"Not enough IOAT rawdevs (%u) for all queues (%u).\n" , + nb_rawdev, cfg.nb_ports * cfg.ports[0 ].nb_queues); +RTE_LOG (INFO ,IOAT ,"Number of used rawdevs: %u.\n" , nb_rawdev); +} + +static void +assign_rings (void ) +{ +uint32_t i; + +for (i =0 ; i < cfg.nb_ports; i++) { +char ring_name[RTE_RING_NAMESIZE ]; + +snprintf (ring_name,sizeof (ring_name),"rx_to_tx_ring_%u" , i); +/* Create ring for inter core communication */ + cfg.ports[i].rx_to_tx_ring =rte_ring_create ( + ring_name, ring_size, +rte_socket_id (),RING_F_SP_ENQ |RING_F_SC_DEQ ); + +if (cfg.ports[i].rx_to_tx_ring ==NULL ) +rte_exit (EXIT_FAILURE ,"Ring create failed: %s\n" , +rte_strerror (rte_errno)); + } +} + +/* +* Initializes a given port using global settings and with the RX buffers +* coming from the mbuf_pool passed as a parameter. +*/ +static inline void +port_init (uint16_t portid,struct rte_mempool *mbuf_pool,uint16_t nb_queues) +{ +/* configuring port to use RSS for multiple RX queues */ +static const struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode =ETH_MQ_RX_RSS , + .max_rx_pkt_len =RTE_ETHER_MAX_LEN + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key =NULL , + .rss_hf =ETH_RSS_PROTO_MASK , + } + } + }; + +struct rte_eth_rxconf rxq_conf; +struct rte_eth_txconf txq_conf; +struct rte_eth_conf local_port_conf = port_conf; +struct rte_eth_dev_info dev_info; +int ret, i; + +/* Skip ports that are not enabled */ +if ((ioat_enabled_port_mask & (1 << portid)) ==0 ) { +printf ("Skipping disabled port %u\n" , portid); +return ; + } + +/* Init port */ +printf ("Initializing port %u... " , portid); +fflush (stdout); +rte_eth_dev_info_get (portid, &dev_info); + local_port_conf.rx_adv_conf.rss_conf.rss_hf &= + dev_info.flow_type_rss_offloads; +if (dev_info.tx_offload_capa &DEV_TX_OFFLOAD_MBUF_FAST_FREE ) + local_port_conf.txmode.offloads |= +DEV_TX_OFFLOAD_MBUF_FAST_FREE ; + ret =rte_eth_dev_configure (portid, nb_queues,1 , &local_port_conf); +if (ret <0 ) +rte_exit (EXIT_FAILURE ,"Cannot configure device:" +" err=%d, port=%u\n" , ret, portid); + + ret =rte_eth_dev_adjust_nb_rx_tx_desc (portid, &nb_rxd, + &nb_txd); +if (ret <0 ) +rte_exit (EXIT_FAILURE , +"Cannot adjust number of descriptors: err=%d, port=%u\n" , + ret, portid); + +rte_eth_macaddr_get (portid, &ioat_ports_eth_addr[portid]); + +/* Init RX queues */ + rxq_conf = dev_info.default_rxconf; + rxq_conf.offloads = local_port_conf.rxmode.offloads; +for (i =0 ; i < nb_queues; i++) { + ret =rte_eth_rx_queue_setup (portid, i, nb_rxd, +rte_eth_dev_socket_id (portid), &rxq_conf, + mbuf_pool); +if (ret <0 ) +rte_exit (EXIT_FAILURE , +"rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n" , + ret, portid, i); + } + +/* Init one TX queue on each port */ + txq_conf = dev_info.default_txconf; + txq_conf.offloads = local_port_conf.txmode.offloads; + ret =rte_eth_tx_queue_setup (portid,0 , nb_txd, +rte_eth_dev_socket_id (portid), + &txq_conf); +if (ret <0 ) +rte_exit (EXIT_FAILURE , +"rte_eth_tx_queue_setup:err=%d,port=%u\n" , + ret, portid); + +/* Initialize TX buffers */ + tx_buffer[portid] =rte_zmalloc_socket ("tx_buffer" , +RTE_ETH_TX_BUFFER_SIZE (MAX_PKT_BURST ),0 , +rte_eth_dev_socket_id (portid)); +if (tx_buffer[portid] ==NULL ) +rte_exit (EXIT_FAILURE , +"Cannot allocate buffer for tx on port %u\n" , + portid); + +rte_eth_tx_buffer_init (tx_buffer[portid],MAX_PKT_BURST ); + + ret =rte_eth_tx_buffer_set_err_callback (tx_buffer[portid], + rte_eth_tx_buffer_count_callback, + &port_statistics.tx_dropped[portid]); +if (ret <0 ) +rte_exit (EXIT_FAILURE , +"Cannot set error callback for tx buffer on port %u\n" , + portid); + +/* Start device */ + ret =rte_eth_dev_start (portid); +if (ret <0 ) +rte_exit (EXIT_FAILURE , +"rte_eth_dev_start:err=%d, port=%u\n" , + ret, portid); + +rte_eth_promiscuous_enable (portid); + +printf ("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n" , + portid, + ioat_ports_eth_addr[portid].addr_bytes[0 ], + ioat_ports_eth_addr[portid].addr_bytes[1 ], + ioat_ports_eth_addr[portid].addr_bytes[2 ], + ioat_ports_eth_addr[portid].addr_bytes[3 ], + ioat_ports_eth_addr[portid].addr_bytes[4 ], + ioat_ports_eth_addr[portid].addr_bytes[5 ]); + + cfg.ports[cfg.nb_ports].rxtx_port = portid; + cfg.ports[cfg.nb_ports++].nb_queues = nb_queues; +} + +static void +signal_handler (int signum) +{ +if (signum ==SIGINT || signum ==SIGTERM ) { +printf ("\n\nSignal %d received, preparing to exit...\n" , + signum); + force_quit = true; + } +} + +int +main (int argc,char **argv) +{ +int ret; +uint16_t nb_ports, portid; +uint32_t i; +unsigned int nb_mbufs; + +/* Init EAL */ + ret =rte_eal_init (argc, argv); +if (ret <0 ) +rte_exit (EXIT_FAILURE ,"Invalid EAL arguments\n" ); + argc -= ret; + argv += ret; + + force_quit = false; +signal (SIGINT , signal_handler); +signal (SIGTERM , signal_handler); + + nb_ports =rte_eth_dev_count_avail (); +if (nb_ports ==0 ) +rte_exit (EXIT_FAILURE ,"No Ethernet ports - bye\n" ); + +/* Parse application arguments (after the EAL ones) */ + ret =ioat_parse_args (argc, argv, nb_ports); +if (ret <0 ) +rte_exit (EXIT_FAILURE ,"Invalid IOAT arguments\n" ); + + nb_mbufs =RTE_MAX (nb_ports * (nb_queues * (nb_rxd + nb_txd + +4 *MAX_PKT_BURST ) +rte_lcore_count () *MEMPOOL_CACHE_SIZE ), +MIN_POOL_SIZE ); + +/* Create the mbuf pool */ + ioat_pktmbuf_pool =rte_pktmbuf_pool_create ("mbuf_pool" , nb_mbufs, +MEMPOOL_CACHE_SIZE ,0 ,RTE_MBUF_DEFAULT_BUF_SIZE , +rte_socket_id ()); +if (ioat_pktmbuf_pool ==NULL ) +rte_exit (EXIT_FAILURE ,"Cannot init mbuf pool\n" ); + +/* Initialise each port */ + cfg.nb_ports =0 ; +RTE_ETH_FOREACH_DEV (portid) +port_init (portid, ioat_pktmbuf_pool, nb_queues); + +/* Initialize port xstats */ +memset (&port_statistics,0 ,sizeof (port_statistics)); + +while (!check_link_status (ioat_enabled_port_mask) && !force_quit) +sleep (1 ); + +/* Check if there is enough lcores for all ports. */ + cfg.nb_lcores =rte_lcore_count () -1 ; +if (cfg.nb_lcores <1 ) +rte_exit (EXIT_FAILURE , +"There should be at least one slave lcore.\n" ); + +if (copy_mode ==COPY_MODE_IOAT_NUM ) +assign_rawdevs (); +else /* copy_mode == COPY_MODE_SW_NUM */ +assign_rings (); + +start_forwarding_cores (); +/* master core prints stats while other cores forward */ +print_stats (argv[0 ]); + +/* force_quit is true when we get here */ +rte_eal_mp_wait_lcore (); + +uint32_t j; + +for (i =0 ; i < cfg.nb_ports; i++) { +printf ("Closing port %d\n" , cfg.ports[i].rxtx_port); +rte_eth_dev_stop (cfg.ports[i].rxtx_port); +rte_eth_dev_close (cfg.ports[i].rxtx_port); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { +printf ("Stopping rawdev %d\n" , + cfg.ports[i].ioat_ids[j]); +rte_rawdev_stop (cfg.ports[i].ioat_ids[j]); + } + }else /* copy_mode == COPY_MODE_SW_NUM */ +rte_ring_free (cfg.ports[i].rx_to_tx_ring); + } + +printf ("Bye...\n" ); +return 0 ; +}examples/ioat/meson.build new file +13
-0@@ -0,0 +1,13 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +deps += ['pmd_ioat'] + +sources = files( + 'ioatfwd.c' +)examples/meson.build +0
-0
You'll need to add the pmd_ioat dependency to this example it seems:
examples/meson.build:89:4: ERROR: Problem encountered: Missing dependency "pmd_ioat" for example "ioat"
Quoted 8 lines
examples/meson.build +1
-0@@ -16,6 +16,7 @@ all_examples = [ 'eventdev_pipeline', 'exception_path', 'fips_validation', 'flow_classify', 'flow_filtering', 'helloworld', + 'ioat', 'ip_fragmentation', 'ip_pipeline', 'ip_reassembly', 'ipsec-secgw', 'ipv4_multicast', 'kni',
Quoted 13 lines · Aaron Conole
Quoted 11 lines · Marcin Baran
From: Pawel Modrak <[email protected]>
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
Signed-off-by Pawel Modrak <[email protected]>Signed-off-by Marcin Baran <[email protected]>---
<snip>
Quoted 26 lines
examples/meson.build
Quoted 23 lines
examples/ioat/meson.build new file +13
-0@@ -0,0 +1,13 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +deps += ['pmd_ioat'] + +sources = files( + 'ioatfwd.c' +)examples/meson.build +0
-0You'll need to add the
pmd_ioatdependency to this example it seems:
Quoted 2 lines
examples/meson.build:89:4: ERROR: Problem encountered: Missing dependency "
pmd_ioat" for example "ioat"
Actually, its added. What I think is missing is a check to disable the app if the dependency is not available.
/Bruce
Quoted 11 lines · Marcin Baran
From: Pawel Modrak <[email protected]>
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
Signed-off-by Pawel Modrak <[email protected]>Signed-off-by Marcin Baran <[email protected]>---
Thanks, Pawel and Marcin. Some comments on doc and code inline below.
Quoted 47 lines
doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 691 +++++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1010 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 13 + examples/meson.build | 1 + 8 files changed, 1777 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.builddoc/guides/sample_app_ug/index.rst +1
-0@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtualdoc/guides/sample_app_ug/intro.rst +4
-0@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.doc/guides/sample_app_ug/ioat.rst new file +0
-0
doc/guides/sample_app_ug/ioat.rst line 5
Quoted 5 lines
doc/guides/sample_app_ug/ioat.rst new file +5
-0@@ -0,0 +1,691 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Intel Corporation. + +Sample Application of packet copying using Intel\|reg| QuickData Technology +============================================================================
You need a space before the |reg| bit otherwise the reg doesn't get the symbol replaced. It should be "Intel\ |reg|".
Quoted 31 lines
doc/guides/sample_app_ug/ioat.rst new file +31
-0+ +Overview +-------- + +This sample is intended as a demonstration of the basic components of a DPDK +forwarding application and example of how to use IOAT driver API to make +packets copies. + +Also while forwarding, the MAC addresses are affected as follows: + +* The source MAC address is replaced by the TX port MAC address + +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID + +This application can be used to compare performance of using software packet +copy with copy done using a DMA device for different sizes of packets. +The example will print out statistics each second. The stats shows +received/send packets and packets dropped or failed to copy. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``ioat`` sub-directory. + + +Running the Application +----------------------- + +In order to run the hardware copy application, the copying device
doc/guides/sample_app_ug/ioat.rst line 46
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+needs to be bound to user-space IO driver. + +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData Technology* +guide for information on using the driver. + +The application requires a number of command line options: + +.. code-block:: console + + ./build/ioatfwd [EAL options] -- -p MASK [-C CT] [--[no-]mac-updating]
I think the app uses lower case "c" rather than upper case, as called out below. Since the "CT" value can only be one of two possibilities, I think you should explicitly include them, e.g. "[-c <sw|rawdev>]". "rawdev" is also a rather long name for this parameter, why not just call them sw and hw?
doc/guides/sample_app_ug/ioat.rst line 57
Quoted 11 lines
doc/guides/sample_app_ug/ioat.rst new file +11
-0+ +where, + +* p MASK: A hexadecimal bitmask of the ports to configure + +* c CT: Performed packet copy type: software (sw) or hardware using + DMA (rawdev) + +* s RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for + software copy mode +
This parameter is missing from the summary above.
doc/guides/sample_app_ug/ioat.rst line 65
Quoted 8 lines
doc/guides/sample_app_ug/ioat.rst new file +8
-0+* --[no-]mac-updating: Whether MAC address of packets should be changed + or not + +The application can be launched in 2 different configurations: + +* Performing software packet copying + +* Performing hardware packet copying
Two thoughts here:
a) is this not obvious from the parameter list
b) is not more that two configurations, given that you can have:
* sw copy with mac updating
* sw copy without mac updating
* etc.
not including the possibly port-mask, ring size and single-core vs two core configurations.
doc/guides/sample_app_ug/ioat.rst line 72
Quoted 7 lines
doc/guides/sample_app_ug/ioat.rst new file +7
-0+ +Each port needs 2 lcores: one of them receives incoming traffic and makes +a copy of each packet. The second lcore then updates MAC address and sends +the copy. For each configuration an additional lcore is needed since +master lcore in use which is responsible for configuration, statistics +printing and safe deinitialization of all ports and devices. +
I believe the app also supports running with 1 or 2 cores total, right?
doc/guides/sample_app_ug/ioat.rst line 73
Quoted 1 line
doc/guides/sample_app_ug/ioat.rst new file +1
-0+The application can use a maximum of 8 ports.
Why this limitation?
Quoted 20 lines
doc/guides/sample_app_ug/ioat.rst new file +20
-0+ +To run the application in a Linux environment with 3 lcores (one of them +is master lcore), 1 port (port 0), software copying and MAC updating issue +the command: + +.. code-block:: console + + $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw + +To run the application in a Linux environment with 5 lcores (one of them +is master lcore), 2 ports (ports 0 and 1), hardware copying and no MAC +updating issue the command: + +.. code-block:: console + + $ ./build/ioatfwd -l 0-4 -n 1 -- -p 0x3 --no-mac-updating -c rawdev + +Refer to the *DPDK Getting Started Guide* for general information on +running applications and the Environment Abstraction Layer (EAL) options. +
<snip>
Quoted 14 lines
examples/ioat/ioatfwd.c new file +8
-0@@ -0,0 +1,1010 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2019 Intel Corporation +*/ + +#include <stdint.h> +#include <getopt.h> +#include <signal.h> +#include <stdbool.h>
examples/ioat/ioatfwd.c line 18
Quoted 10 lines
examples/ioat/ioatfwd.c new file +10
-0+#include <unistd.h> + +#include <rte_malloc.h> +#include <rte_ethdev.h> +#include <rte_rawdev.h> +#include <rte_ioat_rawdev.h> + +/* size of ring used for software copying between rx and tx. */ +#define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1 +#define MAX_PKT_BURST 32
Seems a low max, assume this is actually the default burst size?
Quoted 133 lines
examples/ioat/ioatfwd.c new file +133
-0+#define MEMPOOL_CACHE_SIZE 512 +#define MIN_POOL_SIZE 65536U +#define CMD_LINE_OPT_MAC_UPDATING "mac-updating" +#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating" +#define CMD_LINE_OPT_PORTMASK "portmask" +#define CMD_LINE_OPT_NB_QUEUE "nb-queue" +#define CMD_LINE_OPT_COPY_TYPE "copy-type" +#define CMD_LINE_OPT_RING_SIZE "ring-size" + +/* configurable number of RX/TX ring descriptors */ +#define RX_DEFAULT_RINGSIZE 1024 +#define TX_DEFAULT_RINGSIZE 1024 + +/* max number of RX queues per port */ +#define MAX_RX_QUEUES_COUNT 8 + +struct rxtx_port_config { +/* common config */ +uint16_t rxtx_port; +uint16_t nb_queues; +/* for software copy mode */ +struct rte_ring *rx_to_tx_ring; +/* for IOAT rawdev copy mode */ +uint16_t ioat_ids[MAX_RX_QUEUES_COUNT ]; +}; + +struct rxtx_transmission_config { +struct rxtx_port_config ports[RTE_MAX_ETHPORTS ]; +uint16_t nb_ports; +uint16_t nb_lcores; +}; + +/* per-port statistics struct */ +struct ioat_port_statistics { +uint64_t rx[RTE_MAX_ETHPORTS ]; +uint64_t tx[RTE_MAX_ETHPORTS ]; +uint64_t tx_dropped[RTE_MAX_ETHPORTS ]; +uint64_t copy_dropped[RTE_MAX_ETHPORTS ]; +}; +struct ioat_port_statistics port_statistics; + +struct total_statistics { +uint64_t total_packets_dropped; +uint64_t total_packets_tx; +uint64_t total_packets_rx; +uint64_t total_successful_enqueues; +uint64_t total_failed_enqueues; +}; + +typedef enum copy_mode_t { +#define COPY_MODE_SW "sw" + COPY_MODE_SW_NUM, +#define COPY_MODE_IOAT "rawdev" + COPY_MODE_IOAT_NUM, +COPY_MODE_INVALID_NUM , +COPY_MODE_SIZE_NUM =COPY_MODE_INVALID_NUM +}copy_mode_t ; + +/* mask of enabled ports */ +static uint32_t ioat_enabled_port_mask; + +/* number of RX queues per port */ +static uint16_t nb_queues =1 ; + +/* MAC updating enabled by default. */ +static int mac_updating =1 ; + +/* hardare copy mode enabled by default. */ +static copy_mode_t copy_mode =COPY_MODE_IOAT_NUM ; + +/* size of IOAT rawdev ring for hardware copy mode or +* rte_ring for software copy mode +*/ +static unsigned short ring_size =2048 ; + +/* global transmission config */ +struct rxtx_transmission_config cfg; + +/* configurable number of RX/TX ring descriptors */ +static uint16_t nb_rxd =RX_DEFAULT_RINGSIZE ; +static uint16_t nb_txd =TX_DEFAULT_RINGSIZE ; + +static volatile bool force_quit; + +/* ethernet addresses of ports */ +static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS ]; + +static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ]; +struct rte_mempool *ioat_pktmbuf_pool; + +/* Print out statistics for one port. */ +static void +print_port_stats (uint16_t port_id) +{ +printf ("\nStatistics for port %u ------------------------------" +"\nPackets sent: %34" PRIu64 +"\nPackets received: %30" PRIu64 +"\nPackets dropped on tx: %25" PRIu64 +"\nPackets dropped on copy: %23" PRIu64, + port_id, + port_statistics.tx[port_id], + port_statistics.rx[port_id], + port_statistics.tx_dropped[port_id], + port_statistics.copy_dropped[port_id]); +} + +/* Print out statistics for one IOAT rawdev device. */ +static void +print_rawdev_stats (uint32_t dev_id,uint64_t *xstats, +uint16_t nb_xstats,struct rte_rawdev_xstats_name *names_xstats) +{ +uint16_t i; + +printf ("\nIOAT channel %u" , dev_id); +for (i =0 ; i < nb_xstats; i++) +if (strstr (names_xstats[i].name,"enqueues" )) +printf ("\n\t %s: %*" PRIu64, + names_xstats[i].name, + (int )(37 -strlen (names_xstats[i].name)), + xstats[i]); +} + +static void +print_total_stats (struct total_statistics *ts) +{ +printf ("\nAggregate statistics ===============================" +"\nTotal packets sent: %28" PRIu64 +"\nTotal packets received: %24" PRIu64 +"\nTotal packets dropped: %25" PRIu64, + ts->total_packets_tx, + ts->total_packets_rx, + ts->total_packets_dropped); +
examples/ioat/ioatfwd.c line 161
Quoted 10 lines
examples/ioat/ioatfwd.c new file +10
-0+if (copy_mode ==COPY_MODE_IOAT_NUM ) { +printf ("\nTotal IOAT successful enqueues: %16" PRIu64 +"\nTotal IOAT failed enqueues: %20" PRIu64, + ts->total_successful_enqueues, + ts->total_failed_enqueues); + } + +printf ("\n====================================================\n" ); +} +
For these stats, it would be nice to have deltas i.e. pps, rather than (or as well as) the raw packet count numbers. Since your main stats loop below has a "sleep(1)" at the start, just computing the deltas should give a good enough PPS value.
Quoted 130 lines
examples/ioat/ioatfwd.c new file +130
-0+/* Print out statistics on packets dropped. */ +static void +print_stats (char *prgname) +{ +struct total_statistics ts; +uint32_t i, port_id, dev_id; +struct rte_rawdev_xstats_name *names_xstats; +uint64_t *xstats; +unsigned int *ids_xstats; +unsigned int nb_xstats, id_fail_enq, id_succ_enq; +char status_string[120 ];/* to print at the top of the output */ +int status_strlen; + + +const char clr[] = {27 ,'[' ,'2' ,'J' ,'\0' }; +const char topLeft[] = {27 ,'[' ,'1' ,';' ,'1' ,'H' ,'\0' }; + + status_strlen =snprintf (status_string,sizeof (status_string), +"%s, " , prgname); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Worker Threads = %d, " , +rte_lcore_count () >2 ?2 :1 ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Copy Mode = %s,\n" , copy_mode ==COPY_MODE_SW_NUM ? +COPY_MODE_SW :COPY_MODE_IOAT ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Updating MAC = %s, " , mac_updating ? +"enabled" :"disabled" ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Rx Queues = %d, " , nb_queues); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Ring Size = %d\n" , ring_size); + +/* Allocate memory for xstats names and values */ + nb_xstats =rte_rawdev_xstats_names_get ( + cfg.ports[0 ].ioat_ids[0 ],NULL ,0 ); + + names_xstats =malloc (sizeof (*names_xstats) * nb_xstats); +if (names_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat names memory\n" ); + } +rte_rawdev_xstats_names_get (cfg.ports[0 ].ioat_ids[0 ], + names_xstats, nb_xstats); + + ids_xstats =malloc (sizeof (*ids_xstats) * nb_xstats); +if (ids_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat ids_xstats memory\n" ); + } + +for (i =0 ; i < nb_xstats; i++) + ids_xstats[i] = i; + + xstats =malloc (sizeof (*xstats) * nb_xstats); +if (xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat memory\n" ); + } + +/* Get failed/successful enqueues stats index */ + id_fail_enq = id_succ_enq = nb_xstats; +for (i =0 ; i < nb_xstats; i++) { +if (!strcmp (names_xstats[i].name,"failed_enqueues" )) + id_fail_enq = i; +else if (!strcmp (names_xstats[i].name,"successful_enqueues" )) + id_succ_enq = i; +if (id_fail_enq < nb_xstats && id_succ_enq < nb_xstats) +break ; + } +if (id_fail_enq == nb_xstats || id_succ_enq == nb_xstats) { +rte_exit (EXIT_FAILURE , +"Error getting failed/successful enqueues stats index\n" ); + } + +while (!force_quit) { +/* Sleep for 1 second each round - init sleep allows reading +* messages from app startup. +*/ +sleep (1 ); + +/* Clear screen and move to top left */ +printf ("%s%s" , clr, topLeft); + +memset (&ts,0 ,sizeof (struct total_statistics )); + +printf ("%s" , status_string); + +for (i =0 ; i < cfg.nb_ports; i++) { + port_id = cfg.ports[i].rxtx_port; +print_port_stats (port_id); + + ts.total_packets_dropped += + port_statistics.tx_dropped[port_id] + + port_statistics.copy_dropped[port_id]; + ts.total_packets_tx += port_statistics.tx[port_id]; + ts.total_packets_rx += port_statistics.rx[port_id]; + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +uint32_t j; + +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { + dev_id = cfg.ports[i].ioat_ids[j]; +rte_rawdev_xstats_get (dev_id, + ids_xstats, xstats, nb_xstats); + +print_rawdev_stats (dev_id, xstats, + nb_xstats, names_xstats); + + ts.total_successful_enqueues += + xstats[id_succ_enq]; + ts.total_failed_enqueues += + xstats[id_fail_enq]; + } + } + } +printf ("\n" ); + +print_total_stats (&ts); + } + +free (names_xstats); +free (xstats); +free (ids_xstats); +}
<snip>
Quoted 10 lines · Marcin Baran
From: Pawel Modrak <[email protected]>
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
Signed-off-by Pawel Modrak <[email protected]>Signed-off-by Marcin Baran <[email protected]>---
Thanks, Pawel and Marcin. Some comments on doc and code inline below.
Quoted 50 lines
doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 691 +++++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1010 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 13 + examples/meson.build | 1 + 8 files changed, 1777 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode100644 examples/ioat/Makefile create mode 100644
examples/ioat/ioatfwd.ccreate mode 100644 examples/ioat/meson.builddoc/guides/sample_app_ug/index.rst +1
-0@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtualdoc/guides/sample_app_ug/intro.rst +5
-0@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev +driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.doc/guides/sample_app_ug/ioat.rst new file +0
-0
doc/guides/sample_app_ug/ioat.rst line 7
Quoted 7 lines
doc/guides/sample_app_ug/ioat.rst new file +7
-0@@ -0,0 +1,691 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Intel Corporation. + +Sample Application of packet copying using Intel\|reg| QuickData +Technology +===================================================================== +=======
You need a space before the |reg| bit otherwise the reg doesn't get the symbol replaced. It should be "Intel\ |reg|". [Marcin] Will fix this in v2
Quoted 32 lines
doc/guides/sample_app_ug/ioat.rst new file +32
-0+ +Overview +-------- + +This sample is intended as a demonstration of the basic components of +a DPDK forwarding application and example of how to use IOAT driver +API to make packets copies. + +Also while forwarding, the MAC addresses are affected as follows: + +* The source MAC address is replaced by the TX port MAC address + +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID + +This application can be used to compare performance of using software +packet copy with copy done using a DMA device for different sizes of packets. +The example will print out statistics each second. The stats shows +received/send packets and packets dropped or failed to copy. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``ioat`` sub-directory. + + +Running the Application +----------------------- + +In order to run the hardware copy application, the copying device +needs to be bound to user-space IO driver.
doc/guides/sample_app_ug/ioat.rst
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData +Technology* guide for information on using the driver. + +The application requires a number of command line options: + +.. code-block:: console + + ./build/ioatfwd [EAL options] -- -p MASK [-C CT] + [--[no-]mac-updating]
I think the app uses lower case "c" rather than upper case, as called out below. Since the "CT" value can only be one of two possibilities, I think you should explicitly include them, e.g. "[-c <sw|rawdev>]". "rawdev" is also a rather long name for this parameter, why not just call them sw and hw? [Marcin] Yes, it should be lower case. It is my mistake, the doc was not updated before sending patch. Proper guide is already prepared for v2. As for the "rawdev" parameter name, I will correct it.
doc/guides/sample_app_ug/ioat.rst line 57
Quoted 11 lines
doc/guides/sample_app_ug/ioat.rst new file +11
-0+ +where, + +* p MASK: A hexadecimal bitmask of the ports to configure + +* c CT: Performed packet copy type: software (sw) or hardware using + DMA (rawdev) + +* s RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for + software copy mode +
This parameter is missing from the summary above. [Marcin] Didn't update doc along with code. It is fixed for v2.
doc/guides/sample_app_ug/ioat.rst line 65
Quoted 8 lines
doc/guides/sample_app_ug/ioat.rst new file +8
-0+* --[no-]mac-updating: Whether MAC address of packets should be changed + or not + +The application can be launched in 2 different configurations: + +* Performing software packet copying + +* Performing hardware packet copying
Two thoughts here:
a) is this not obvious from the parameter list
b) is not more that two configurations, given that you can have:
* sw copy with mac updating
* sw copy without mac updating
* etc.
not including the possibly port-mask, ring size and single-core vs two core configurations. [Marcin] Good point. I will rephrase that.
doc/guides/sample_app_ug/ioat.rst line 75
Quoted 7 lines
doc/guides/sample_app_ug/ioat.rst new file +7
-0+ +Each port needs 2 lcores: one of them receives incoming traffic and +makes a copy of each packet. The second lcore then updates MAC +address and sends the copy. For each configuration an additional +lcore is needed since master lcore in use which is responsible for +configuration, statistics printing and safe deinitialization of all ports and devices. +
I believe the app also supports running with 1 or 2 cores total, right? [Marcin]Yes, that's right. Didn't update doc along with code. It is fixed for v2.
doc/guides/sample_app_ug/ioat.rst line 73
Quoted 1 line
doc/guides/sample_app_ug/ioat.rst new file +1
-0+The application can use a maximum of 8 ports.
Why this limitation?
[Marcin] Seemed reasonable since on testing machine one IOAT device had 8 channels.
Quoted 21 lines
doc/guides/sample_app_ug/ioat.rst new file +21
-0+ +To run the application in a Linux environment with 3 lcores (one of +them is master lcore), 1 port (port 0), software copying and MAC +updating issue the command: + +.. code-block:: console + + $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw + +To run the application in a Linux environment with 5 lcores (one of +them is master lcore), 2 ports (ports 0 and 1), hardware copying and +no MAC updating issue the command: + +.. code-block:: console + + $ ./build/ioatfwd -l 0-4 -n 1 -- -p 0x3 --no-mac-updating -c + rawdev + +Refer to the *DPDK Getting Started Guide* for general information on +running applications and the Environment Abstraction Layer (EAL) options. +
<snip>
Quoted 11 lines
examples/ioat/ioatfwd.c new renamed from examples/ioat/ioatfwd.c +0
-0file mode 100644 index 000000000..8463d82f3
examples/ioat/ioatfwd.c new file +6
-0@@ -0,0 +1,1010 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2019 Intel Corporation */ + +#include <stdint.h> +#include <getopt.h> +#include <signal.h>
examples/ioat/ioatfwd.c line 16
Quoted 10 lines
examples/ioat/ioatfwd.c new file +10
-0+#include <stdbool.h> +#include <unistd.h> + +#include <rte_malloc.h> +#include <rte_ethdev.h> +#include <rte_rawdev.h> +#include <rte_ioat_rawdev.h> + +/* size of ring used for software copying between rx and tx. */ +#define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1 #define MAX_PKT_BURST 32
Seems a low max, assume this is actually the default burst size?
Quoted 122 lines
examples/ioat/ioatfwd.c new file +122
-0+#define MEMPOOL_CACHE_SIZE 512 +#define MIN_POOL_SIZE 65536U +#define CMD_LINE_OPT_MAC_UPDATING "mac-updating" +#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating" +#define CMD_LINE_OPT_PORTMASK "portmask" +#define CMD_LINE_OPT_NB_QUEUE "nb-queue" +#define CMD_LINE_OPT_COPY_TYPE "copy-type" +#define CMD_LINE_OPT_RING_SIZE "ring-size" + +/* configurable number of RX/TX ring descriptors */ #define +RX_DEFAULT_RINGSIZE 1024 #define TX_DEFAULT_RINGSIZE 1024 + +/* max number of RX queues per port */ #define MAX_RX_QUEUES_COUNT 8 + +struct rxtx_port_config { +/* common config */ +uint16_t rxtx_port; +uint16_t nb_queues; +/* for software copy mode */ +struct rte_ring *rx_to_tx_ring; +/* for IOAT rawdev copy mode */ +uint16_t ioat_ids[MAX_RX_QUEUES_COUNT ]; }; + +struct rxtx_transmission_config { +struct rxtx_port_config ports[RTE_MAX_ETHPORTS ]; +uint16_t nb_ports; +uint16_t nb_lcores; +}; + +/* per-port statistics struct */ +struct ioat_port_statistics { +uint64_t rx[RTE_MAX_ETHPORTS ]; +uint64_t tx[RTE_MAX_ETHPORTS ]; +uint64_t tx_dropped[RTE_MAX_ETHPORTS ]; +uint64_t copy_dropped[RTE_MAX_ETHPORTS ]; };struct +ioat_port_statistics port_statistics; + +struct total_statistics { +uint64_t total_packets_dropped; +uint64_t total_packets_tx; +uint64_t total_packets_rx; +uint64_t total_successful_enqueues; +uint64_t total_failed_enqueues; +}; + +typedef enum copy_mode_t { +#define COPY_MODE_SW "sw" + COPY_MODE_SW_NUM, +#define COPY_MODE_IOAT "rawdev" + COPY_MODE_IOAT_NUM, +COPY_MODE_INVALID_NUM , +COPY_MODE_SIZE_NUM =COPY_MODE_INVALID_NUM }copy_mode_t ; + +/* mask of enabled ports */ +static uint32_t ioat_enabled_port_mask; + +/* number of RX queues per port */ +static uint16_t nb_queues =1 ; + +/* MAC updating enabled by default. */ static int mac_updating =1 ; + +/* hardare copy mode enabled by default. */ static copy_mode_t +copy_mode =COPY_MODE_IOAT_NUM ; + +/* size of IOAT rawdev ring for hardware copy mode or +* rte_ring for software copy mode +*/ +static unsigned short ring_size =2048 ; + +/* global transmission config */ +struct rxtx_transmission_config cfg; + +/* configurable number of RX/TX ring descriptors */ static uint16_t +nb_rxd =RX_DEFAULT_RINGSIZE ;static uint16_t nb_txd = +TX_DEFAULT_RINGSIZE ; + +static volatile bool force_quit; + +/* ethernet addresses of ports */ +static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS ]; + +static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ]; +struct rte_mempool *ioat_pktmbuf_pool; + +/* Print out statistics for one port. */ static void +print_port_stats (uint16_t port_id) { +printf ("\nStatistics for port %u ------------------------------" +"\nPackets sent: %34" PRIu64 +"\nPackets received: %30" PRIu64 +"\nPackets dropped on tx: %25" PRIu64 +"\nPackets dropped on copy: %23" PRIu64, + port_id, + port_statistics.tx[port_id], + port_statistics.rx[port_id], + port_statistics.tx_dropped[port_id], + port_statistics.copy_dropped[port_id]); +} + +/* Print out statistics for one IOAT rawdev device. */ static void +print_rawdev_stats (uint32_t dev_id,uint64_t *xstats, +uint16_t nb_xstats,struct rte_rawdev_xstats_name *names_xstats) { +uint16_t i; + +printf ("\nIOAT channel %u" , dev_id); +for (i =0 ; i < nb_xstats; i++) +if (strstr (names_xstats[i].name,"enqueues" )) +printf ("\n\t %s: %*" PRIu64, + names_xstats[i].name, + (int )(37 -strlen (names_xstats[i].name)), + xstats[i]); +} + +static void +print_total_stats (struct total_statistics *ts) { +printf ("\nAggregate statistics ===============================" +"\nTotal packets sent: %28" PRIu64 +"\nTotal packets received: %24" PRIu64 +"\nTotal packets dropped: %25" PRIu64, + ts->total_packets_tx, + ts->total_packets_rx, + ts->total_packets_dropped); +
examples/ioat/ioatfwd.c line 161
Quoted 10 lines
examples/ioat/ioatfwd.c new file +10
-0+if (copy_mode ==COPY_MODE_IOAT_NUM ) { +printf ("\nTotal IOAT successful enqueues: %16" PRIu64 +"\nTotal IOAT failed enqueues: %20" PRIu64, + ts->total_successful_enqueues, + ts->total_failed_enqueues); + } + +printf ("\n====================================================\n" ); +} +
For these stats, it would be nice to have deltas i.e. pps, rather than (or as well as) the raw packet count numbers. Since your main stats loop below has a "sleep(1)" at the start, just computing the deltas should give a good enough PPS value. [Marcin] Ok, will be prepared for v2 as PPS value.
Quoted 128 lines
examples/ioat/ioatfwd.c new file +128
-0+/* Print out statistics on packets dropped. */ static void +print_stats (char *prgname) { +struct total_statistics ts; +uint32_t i, port_id, dev_id; +struct rte_rawdev_xstats_name *names_xstats; +uint64_t *xstats; +unsigned int *ids_xstats; +unsigned int nb_xstats, id_fail_enq, id_succ_enq; +char status_string[120 ];/* to print at the top of the output */ +int status_strlen; + + +const char clr[] = {27 ,'[' ,'2' ,'J' ,'\0' }; +const char topLeft[] = {27 ,'[' ,'1' ,';' ,'1' ,'H' ,'\0' }; + + status_strlen =snprintf (status_string,sizeof (status_string), +"%s, " , prgname); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Worker Threads = %d, " , +rte_lcore_count () >2 ?2 :1 ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Copy Mode = %s,\n" , copy_mode ==COPY_MODE_SW_NUM ? +COPY_MODE_SW :COPY_MODE_IOAT ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Updating MAC = %s, " , mac_updating ? +"enabled" :"disabled" ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Rx Queues = %d, " , nb_queues); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Ring Size = %d\n" , ring_size); + +/* Allocate memory for xstats names and values */ + nb_xstats =rte_rawdev_xstats_names_get ( + cfg.ports[0 ].ioat_ids[0 ],NULL ,0 ); + + names_xstats =malloc (sizeof (*names_xstats) * nb_xstats); +if (names_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat names memory\n" ); + } +rte_rawdev_xstats_names_get (cfg.ports[0 ].ioat_ids[0 ], + names_xstats, nb_xstats); + + ids_xstats =malloc (sizeof (*ids_xstats) * nb_xstats); +if (ids_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat ids_xstats memory\n" ); + } + +for (i =0 ; i < nb_xstats; i++) + ids_xstats[i] = i; + + xstats =malloc (sizeof (*xstats) * nb_xstats); +if (xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat memory\n" ); + } + +/* Get failed/successful enqueues stats index */ + id_fail_enq = id_succ_enq = nb_xstats; +for (i =0 ; i < nb_xstats; i++) { +if (!strcmp (names_xstats[i].name,"failed_enqueues" )) + id_fail_enq = i; +else if (!strcmp (names_xstats[i].name,"successful_enqueues" )) + id_succ_enq = i; +if (id_fail_enq < nb_xstats && id_succ_enq < nb_xstats) +break ; + } +if (id_fail_enq == nb_xstats || id_succ_enq == nb_xstats) { +rte_exit (EXIT_FAILURE , +"Error getting failed/successful enqueues stats index\n" ); + } + +while (!force_quit) { +/* Sleep for 1 second each round - init sleep allows reading +* messages from app startup. +*/ +sleep (1 ); + +/* Clear screen and move to top left */ +printf ("%s%s" , clr, topLeft); + +memset (&ts,0 ,sizeof (struct total_statistics )); + +printf ("%s" , status_string); + +for (i =0 ; i < cfg.nb_ports; i++) { + port_id = cfg.ports[i].rxtx_port; +print_port_stats (port_id); + + ts.total_packets_dropped += + port_statistics.tx_dropped[port_id] + + port_statistics.copy_dropped[port_id]; + ts.total_packets_tx += port_statistics.tx[port_id]; + ts.total_packets_rx += port_statistics.rx[port_id]; + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +uint32_t j; + +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { + dev_id = cfg.ports[i].ioat_ids[j]; +rte_rawdev_xstats_get (dev_id, + ids_xstats, xstats, nb_xstats); + +print_rawdev_stats (dev_id, xstats, + nb_xstats, names_xstats); + + ts.total_successful_enqueues += + xstats[id_succ_enq]; + ts.total_failed_enqueues += + xstats[id_fail_enq]; + } + } + } +printf ("\n" ); + +print_total_stats (&ts); + } + +free (names_xstats); +free (xstats); +free (ids_xstats); +}
<snip>
v2 cover examples/ioat: sample app on ioat driver usage
In reply to Marcin BaranA new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
v2:
- change printing total stats to be deltas
- update documentation
- fix 1 thread/sw copy/multiple Rx queues packet dropping
- divide patch into several presenting functional change
Marcin Baran (4):
examples/ioat: add software copy support examples/ioat: add rawdev copy mode support examples/ioat: add stats printing for each port doc/guides/: provide IOAT sample app guide
Pawel Modrak (2):
examples/ioat: create sample app on ioat driver usage examples/ioat: add two threads configuration
doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1025 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 13 + examples/meson.build | 1 + 8 files changed, 1865 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
Signature · 1 line
2.22.0.windows.1
PATCH v2 1/6 examples/ioat: create sample app on ioat driver usage
From: Pawel Modrak <[email protected]>
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The change covers ports initialization, closing connection and argument parsing.
--- examples/Makefile | 3 + examples/ioat/Makefile | 54 +++++ examples/ioat/ioatfwd.c | 439 ++++++++++++++++++++++++++++++++++++++ examples/ioat/meson.build | 13 ++ examples/meson.build | 1 + 5 files changed, 510 insertions(+) create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
examples/Makefile +3 -0
@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify DIRS-y += flow_filtering DIRS-y += helloworld +ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y) +DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat +endif DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline ifeq ($(CONFIG_RTE_LIBRTE_LPM),y) DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
examples/ioat/Makefile new file +54 -0
54 lines not on this page · raw message
examples/ioat/ioatfwd.c new file +439 -0
439 lines not on this page · raw message
examples/ioat/meson.build new file +13 -0
@@ -0,0 +1,13 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +deps += ['pmd_ioat'] + +sources = files( + 'ioatfwd.c' +)
examples/meson.build +1 -0
@@ -16,6 +16,7 @@ all_examples = [ 'eventdev_pipeline', 'exception_path', 'fips_validation', 'flow_classify', 'flow_filtering', 'helloworld', + 'ioat', 'ip_fragmentation', 'ip_pipeline', 'ip_reassembly', 'ipsec-secgw', 'ipv4_multicast', 'kni',
Signature · 1 line
2.22.0.windows.1
PATCH v2 2/6 examples/ioat: add software copy support
In reply to Marcin BaranAdded support for copying packets
using software copy mode and MAC address
changing. The copies are processed using
one lcore.
--- examples/ioat/ioatfwd.c | 211 +++++++++++++++++++++++++++++++++++----- 1 file changed, 188 insertions(+), 23 deletions(-)
examples/ioat/ioatfwd.c +188 -23
@@ -13,7 +13,7 @@#include <rte_rawdev.h> #include <rte_ioat_rawdev.h> -/* size of ring used for software copying between rx and tx. */ +/* size of ring used for software copying between rx and tx. */ #define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1#define MAX_PKT_BURST 32#define MEMPOOL_CACHE_SIZE 512
@@ -89,6 +89,142 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ];struct rte_mempool *ioat_pktmbuf_pool; +static void +update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) +{ +struct rte_ether_hdr *eth; +void *tmp; + + eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *); + +/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but +* it's acceptable cause it gets overwritten by rte_ether_addr_copy +*/ + tmp = ð->d_addr.addr_bytes[0 ]; + *((uint64_t *)tmp) =0x000000000002 + ((uint64_t )dest_portid <<40 ); + +/* src addr */ +rte_ether_addr_copy (&ioat_ports_eth_addr[dest_portid], ð->s_addr); +} + +static inline void +pktmbuf_sw_copy (struct rte_mbuf *src,struct rte_mbuf *dst) +{ +/* Copy packet metadata */ +rte_memcpy (&dst->rearm_data, + &src->rearm_data, + offsetof(struct rte_mbuf , cacheline1) + - offsetof(struct rte_mbuf , rearm_data)); + +/* Copy packet data */ + rte_memcpy(rte_pktmbuf_mtod(dst, char *), + rte_pktmbuf_mtod(src, char *), src->data_len); +} + +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ +static void +ioat_rx_port (struct rxtx_port_config *rx_config) +{ +uint32_t nb_rx, nb_enq, i, j; +struct rte_mbuf *pkts_burst[MAX_PKT_BURST ]; + +for (i =0 ; i < rx_config->nb_queues; i++) { + + nb_rx =rte_eth_rx_burst (rx_config->rxtx_port, i, + pkts_burst,MAX_PKT_BURST ); + +if (nb_rx ==0 ) +continue ; + +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } +} + +/* Transmit packets from IOAT rawdev/rte_ring for one port. */ +static void +ioat_tx_port (struct rxtx_port_config *tx_config) +{ +uint32_t i, nb_dq =0 ; +struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ]; + +/* Deque the mbufs from rx_to_tx_ring. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); + +if (nb_dq ==0 ) +return ; + +/* Update macs if enabled */ +if (mac_updating) { +for (i =0 ; i < nb_dq; i++) +update_mac_addrs (mbufs_dst[i], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, +0 , (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); +} + +/* Main rx and tx loop if only one slave lcore available */ +static void +rxtx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx and tx loop for copy on" +" lcore %u\n" ,rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) { +ioat_rx_port (&cfg.ports[i]); +ioat_tx_port (&cfg.ports[i]); + } +} + +static void start_forwarding_cores (void ) +{ +uint32_t lcore_id =rte_lcore_id (); + +RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , + __func__,rte_lcore_id ()); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); +} +/* Display usage */ static void ioat_usage (const char *prgname)
@@ -102,7 +238,7 @@ ioat_usage(const char *prgname)" - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n" " -c --copy-type CT: type of copy: sw|rawdev\n" " -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n" , - prgname); + prgname); } static int
@@ -161,16 +297,16 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports) argvopt = argv;while ((opt =getopt_long (argc, argvopt, short_options, - lgopts, &option_index)) !=EOF ) { + lgopts, &option_index)) !=EOF ) {switch (opt) { -/* portmask */ +/* portmask */ case 'p' : ioat_enabled_port_mask =ioat_parse_portmask (optarg);if (ioat_enabled_port_mask & ~default_port_mask || - ioat_enabled_port_mask <=0 ) { + ioat_enabled_port_mask <=0 ) {printf ("Invalid portmask, %s, suggest 0x%x\n" , - optarg, default_port_mask); + optarg, default_port_mask);ioat_usage (prgname);return -1 ; }
@@ -204,7 +340,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports) }break ; -/* long options */ +/* long options */ case 0 :break ;
@@ -216,9 +352,9 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)printf ("MAC updating %s\n" , mac_updating ?"enabled" :"disabled" );if (optind >=0 ) - argv[optind-1 ] = prgname; + argv[optind -1 ] = prgname; - ret = optind-1 ; + ret = optind -1 ; optind =1 ;/* reset getopt lib */ return ret; }
@@ -253,6 +389,26 @@ check_link_status(uint32_t port_mask)return retval; } +static void +assign_rings (void ) +{ +uint32_t i; + +for (i =0 ; i < cfg.nb_ports; i++) { +char ring_name[RTE_RING_NAMESIZE ]; + +snprintf (ring_name,sizeof (ring_name),"rx_to_tx_ring_%u" , i); +/* Create ring for inter core communication */ + cfg.ports[i].rx_to_tx_ring =rte_ring_create ( + ring_name, ring_size, +rte_socket_id (),RING_F_SP_ENQ |RING_F_SC_DEQ ); + +if (cfg.ports[i].rx_to_tx_ring ==NULL ) +rte_exit (EXIT_FAILURE ,"Ring create failed: %s\n" , +rte_strerror (rte_errno)); + } +} +/* * Initializes a given port using global settings and with the RX buffers * coming from the mbuf_pool passed as a parameter.
@@ -263,7 +419,7 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)/* configuring port to use RSS for multiple RX queues */ static const struct rte_eth_conf port_conf = { .rxmode = { - .mq_mode =ETH_MQ_RX_RSS , + .mq_mode =ETH_MQ_RX_RSS , .max_rx_pkt_len =RTE_ETHER_MAX_LEN }, .rx_adv_conf = {
@@ -294,14 +450,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues) dev_info.flow_type_rss_offloads;if (dev_info.tx_offload_capa &DEV_TX_OFFLOAD_MBUF_FAST_FREE ) local_port_conf.txmode.offloads |= -DEV_TX_OFFLOAD_MBUF_FAST_FREE ; +DEV_TX_OFFLOAD_MBUF_FAST_FREE ; ret =rte_eth_dev_configure (portid, nb_queues,1 , &local_port_conf);if (ret <0 )rte_exit (EXIT_FAILURE ,"Cannot configure device:" " err=%d, port=%u\n" , ret, portid); ret =rte_eth_dev_adjust_nb_rx_tx_desc (portid, &nb_rxd, - &nb_txd); + &nb_txd);if (ret <0 ) rte_exit(EXIT_FAILURE, "Cannot adjust number of descriptors: err=%d, port=%u\n",
@@ -326,8 +482,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues) txq_conf = dev_info.default_txconf; txq_conf.offloads = local_port_conf.txmode.offloads; ret =rte_eth_tx_queue_setup (portid,0 , nb_txd, -rte_eth_dev_socket_id (portid), - &txq_conf); +rte_eth_dev_socket_id (portid), + &txq_conf);if (ret <0 ) rte_exit(EXIT_FAILURE, "rte_eth_tx_queue_setup:err=%d,port=%u\n",
@@ -335,8 +491,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)/* Initialize TX buffers */ tx_buffer[portid] =rte_zmalloc_socket ("tx_buffer" , -RTE_ETH_TX_BUFFER_SIZE (MAX_PKT_BURST ),0 , -rte_eth_dev_socket_id (portid)); +RTE_ETH_TX_BUFFER_SIZE (MAX_PKT_BURST ),0 , +rte_eth_dev_socket_id (portid));if (tx_buffer[portid] ==NULL ) rte_exit(EXIT_FAILURE, "Cannot allocate buffer for tx on port %u\n",
@@ -354,13 +510,13 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)rte_eth_promiscuous_enable (portid);printf ("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n" , - portid, - ioat_ports_eth_addr[portid].addr_bytes[0 ], - ioat_ports_eth_addr[portid].addr_bytes[1 ], - ioat_ports_eth_addr[portid].addr_bytes[2 ], - ioat_ports_eth_addr[portid].addr_bytes[3 ], - ioat_ports_eth_addr[portid].addr_bytes[4 ], - ioat_ports_eth_addr[portid].addr_bytes[5 ]); + portid, + ioat_ports_eth_addr[portid].addr_bytes[0 ], + ioat_ports_eth_addr[portid].addr_bytes[1 ], + ioat_ports_eth_addr[portid].addr_bytes[2 ], + ioat_ports_eth_addr[portid].addr_bytes[3 ], + ioat_ports_eth_addr[portid].addr_bytes[4 ], + ioat_ports_eth_addr[portid].addr_bytes[5 ]); cfg.ports[cfg.nb_ports].rxtx_port = portid; cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
@@ -428,10 +584,19 @@ main(int argc, char **argv)if (cfg.nb_lcores <1 )rte_exit (EXIT_FAILURE ,"There should be at least one slave lcore.\n" ); + +assign_rings (); + +start_forwarding_cores (); + +/* force_quit is true when we get here */ +rte_eal_mp_wait_lcore (); +for (i =0 ; i < cfg.nb_ports; i++) {printf ("Closing port %d\n" , cfg.ports[i].rxtx_port);rte_eth_dev_stop (cfg.ports[i].rxtx_port);rte_eth_dev_close (cfg.ports[i].rxtx_port); +rte_ring_free (cfg.ports[i].rx_to_tx_ring); }printf ("Bye...\n" );
Signature · 1 line
2.22.0.windows.1
PATCH v2 3/6 examples/ioat: add rawdev copy mode support
In reply to Marcin BaranAdded support for copying packets using
rawdev device. Each port's Rx queue is
assigned DMA channel for copy.
--- examples/ioat/ioatfwd.c | 236 ++++++++++++++++++++++++++++++++-------- 1 file changed, 189 insertions(+), 47 deletions(-)
examples/ioat/ioatfwd.c +189 -47
@@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) rte_pktmbuf_mtod(src, char *), src->data_len); } +static uint32_t +ioat_enqueue_packets (struct rte_mbuf **pkts, +uint32_t nb_rx,uint16_t dev_id) +{ +int ret; +uint32_t i; +struct rte_mbuf *pkts_copy[MAX_PKT_BURST ]; + +const uint64_t addr_offset =RTE_PTR_DIFF (pkts[0 ]->buf_addr, + &pkts[0 ]->rearm_data); + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE ,"Unable to allocate memory.\n" ); + +for (i =0 ; i < nb_rx; i++) { +/* Perform data copy */ + ret =rte_ioat_enqueue_copy (dev_id, + pkts[i]->buf_iova + - addr_offset, + pkts_copy[i]->buf_iova + - addr_offset, +rte_pktmbuf_data_len (pkts[i]) + + addr_offset, + (uintptr_t )pkts[i], + (uintptr_t )pkts_copy[i], +0 /* nofence */ ); + +if (ret !=1 ) +break ; + } + + ret = i; +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i); +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], + nb_rx - i); + + +return ret; +} +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ static void ioat_rx_port (struct rxtx_port_config *rx_config)
@@ -136,32 +180,40 @@ ioat_rx_port(struct rxtx_port_config *rx_config)if (nb_rx ==0 )continue ; -/* Perform packet software copy, free source packets */ -int ret; -struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; - - ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, - (void *)pkts_burst_copy, nb_rx); - -if (unlikely (ret <0 )) -rte_exit (EXIT_FAILURE , -"Unable to allocate memory.\n" ); - -for (j =0 ; j < nb_rx; j++) -pktmbuf_sw_copy (pkts_burst[j], - pkts_burst_copy[j]); - -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)pkts_burst, nb_rx); - - nb_enq =rte_ring_enqueue_burst ( - rx_config->rx_to_tx_ring, - (void *)pkts_burst_copy, nb_rx,NULL ); - -/* Free any not enqueued packets. */ -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)&pkts_burst_copy[nb_enq], - nb_rx - nb_enq); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Perform packet hardware copy */ + nb_enq =ioat_enqueue_packets (pkts_burst, + nb_rx, rx_config->ioat_ids[i]); +if (nb_enq >0 ) +rte_ioat_do_copies (rx_config->ioat_ids[i]); + }else { +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } } }
@@ -169,31 +221,64 @@ ioat_rx_port(struct rxtx_port_config *rx_config)static void ioat_tx_port (struct rxtx_port_config *tx_config) { -uint32_t i, nb_dq =0 ; +uint32_t i, j, nb_dq =0 ; +struct rte_mbuf *mbufs_src[MAX_PKT_BURST ];struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ]; -/* Deque the mbufs from rx_to_tx_ring. */ - nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, - (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Deque the mbufs from IOAT device. */ +for (i =0 ; i < tx_config->nb_queues; i++) { + nb_dq =rte_ioat_completed_copies ( + tx_config->ioat_ids[i],MAX_PKT_BURST , + (void *)mbufs_src, (void *)mbufs_dst); -if (nb_dq ==0 ) -return ; +if (nb_dq ==0 ) +break ; -/* Update macs if enabled */ -if (mac_updating) { -for (i =0 ; i < nb_dq; i++) -update_mac_addrs (mbufs_dst[i], - tx_config->rxtx_port); - } +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)mbufs_src, nb_dq); -const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, -0 , (void *)mbufs_dst, nb_dq); +/* Update macs if enabled */ +if (mac_updating) { +for (j =0 ; j < nb_dq; j++) +update_mac_addrs (mbufs_dst[j], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst ( + tx_config->rxtx_port,0 , + (void *)mbufs_dst, nb_dq); -/* Free any unsent packets. */ -if (unlikely (nb_tx < nb_dq)) -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)&mbufs_dst[nb_tx], - nb_dq - nb_tx); +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } + }else { +/* Deque the mbufs from rx_to_tx_ring. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); + +if (nb_dq ==0 ) +return ; + +/* Update macs if enabled */ +if (mac_updating) { +for (i =0 ; i < nb_dq; i++) +update_mac_addrs (mbufs_dst[i], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, +0 , (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } }/* Main rx and tx loop if only one slave lcore available */
@@ -389,6 +474,52 @@ check_link_status(uint32_t port_mask)return retval; } +static void +configure_rawdev_queue (uint32_t dev_id) +{ +struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size }; +struct rte_rawdev_info info = { .dev_private = &dev_config }; + +if (rte_rawdev_configure (dev_id, &info) !=0 ) { +rte_exit (EXIT_FAILURE , +"Error with rte_rawdev_configure()\n" ); + } +if (rte_rawdev_start (dev_id) !=0 ) { +rte_exit (EXIT_FAILURE , +"Error with rte_rawdev_start()\n" ); + } +} + +static void +assign_rawdevs (void ) +{ +uint16_t nb_rawdev =0 , rdev_id =0 ; +uint32_t i, j; + +for (i =0 ; i < cfg.nb_ports; i++) { +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { +struct rte_rawdev_info rdev_info = {0 }; + +do { +if (rdev_id ==rte_rawdev_count ()) + goto end; +rte_rawdev_info_get (rdev_id++, &rdev_info); + }while (strcmp (rdev_info.driver_name, +IOAT_PMD_RAWDEV_NAME_STR ) !=0 ); + + cfg.ports[i].ioat_ids[j] = rdev_id -1 ; +configure_rawdev_queue (cfg.ports[i].ioat_ids[j]); + ++nb_rawdev; + } + } +end: +if (nb_rawdev < cfg.nb_ports * cfg.ports[0 ].nb_queues) +rte_exit (EXIT_FAILURE , +"Not enough IOAT rawdevs (%u) for all queues (%u).\n" , + nb_rawdev, cfg.nb_ports * cfg.ports[0 ].nb_queues); +RTE_LOG (INFO ,IOAT ,"Number of used rawdevs: %u.\n" , nb_rawdev); +} +static void assign_rings (void ) {
@@ -585,18 +716,29 @@ main(int argc, char **argv)rte_exit (EXIT_FAILURE ,"There should be at least one slave lcore.\n" ); -assign_rings (); +if (copy_mode ==COPY_MODE_IOAT_NUM ) +assign_rawdevs (); +else /* copy_mode == COPY_MODE_SW_NUM */ +assign_rings ();start_forwarding_cores ();/* force_quit is true when we get here */ rte_eal_mp_wait_lcore (); +uint32_t j;for (i =0 ; i < cfg.nb_ports; i++) {printf ("Closing port %d\n" , cfg.ports[i].rxtx_port);rte_eth_dev_stop (cfg.ports[i].rxtx_port);rte_eth_dev_close (cfg.ports[i].rxtx_port); -rte_ring_free (cfg.ports[i].rx_to_tx_ring); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { +printf ("Stopping rawdev %d\n" , + cfg.ports[i].ioat_ids[j]); +rte_rawdev_stop (cfg.ports[i].ioat_ids[j]); + } + }else /* copy_mode == COPY_MODE_SW_NUM */ +rte_ring_free (cfg.ports[i].rx_to_tx_ring); }printf ("Bye...\n" );
Signature · 1 line
2.22.0.windows.1
PATCH v2 4/6 examples/ioat: add two threads configuration
In reply to Marcin BaranFrom: Pawel Modrak <[email protected]>
Added possibility to use two lcores: first for
packet receiving and copying, second for packets
sending.
--- examples/ioat/ioatfwd.c | 92 ++++++++++++++++++++++++++++++----------- 1 file changed, 67 insertions(+), 25 deletions(-)
examples/ioat/ioatfwd.c +67 -25
@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], nb_rx - i); -return ret; }
@@ -226,8 +225,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ];if (copy_mode ==COPY_MODE_IOAT_NUM ) { -/* Deque the mbufs from IOAT device. */ for (i =0 ; i < tx_config->nb_queues; i++) { +/* Deque the mbufs from IOAT device. */ nb_dq =rte_ioat_completed_copies ( tx_config->ioat_ids[i],MAX_PKT_BURST , (void *)mbufs_src, (void *)mbufs_dst);
@@ -256,31 +255,64 @@ ioat_tx_port(struct rxtx_port_config *tx_config) nb_dq - nb_tx); } }else { -/* Deque the mbufs from rx_to_tx_ring. */ - nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, - (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); - -if (nb_dq ==0 ) -return ; - -/* Update macs if enabled */ -if (mac_updating) { -for (i =0 ; i < nb_dq; i++) -update_mac_addrs (mbufs_dst[i], - tx_config->rxtx_port); - } +for (i =0 ; i < tx_config->nb_queues; i++) { +/* Deque the mbufs from IOAT device. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); -const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, -0 , (void *)mbufs_dst, nb_dq); +if (nb_dq ==0 ) +return ; -/* Free any unsent packets. */ -if (unlikely (nb_tx < nb_dq)) -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)&mbufs_dst[nb_tx], - nb_dq - nb_tx); +/* Update macs if enabled */ +if (mac_updating) { +for (j =0 ; j < nb_dq; j++) +update_mac_addrs (mbufs_dst[j], + tx_config->rxtx_port); + } + +const uint16_t nb_tx = +rte_eth_tx_burst (tx_config->rxtx_port,0 , + (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } } } +/* Main rx processing loop for IOAT rawdev. */ +static void +rx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx loop for copy on lcore %u\n" , +rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) +ioat_rx_port (&cfg.ports[i]); +} + +/* Main tx processing loop for hardware copy. */ +static void +tx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main tx loop for copy on lcore %u\n" , +rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) +ioat_tx_port (&cfg.ports[i]); +} +/* Main rx and tx loop if only one slave lcore available */ static void rxtx_main_loop (void )
@@ -305,9 +337,19 @@ static void start_forwarding_cores(void)RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , __func__,rte_lcore_id ()); - lcore_id =rte_get_next_lcore (lcore_id, true, true); -rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, -NULL , lcore_id); +if (cfg.nb_lcores ==1 ) { + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); + }else if (cfg.nb_lcores >1 ) { + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rx_main_loop, +NULL , lcore_id); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)tx_main_loop,NULL , + lcore_id); + } }/* Display usage */
Signature · 1 line
2.22.0.windows.1
PATCH v2 5/6 examples/ioat: add stats printing for each port
In reply to Marcin BaranAdded printing stats of ports each second. The stats printing is done using master core. The information provided informs about packets received, dropped and send as well as statistics of rawdev devices.
--- examples/ioat/ioatfwd.c | 245 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 241 insertions(+), 4 deletions(-)
examples/ioat/ioatfwd.c +241 -4
@@ -48,10 +48,27 @@ struct rxtx_transmission_config {uint16_t nb_lcores; }; +/* per-port statistics struct */ +struct ioat_port_statistics { +uint64_t rx[RTE_MAX_ETHPORTS ]; +uint64_t tx[RTE_MAX_ETHPORTS ]; +uint64_t tx_dropped[RTE_MAX_ETHPORTS ]; +uint64_t copy_dropped[RTE_MAX_ETHPORTS ]; +}; +struct ioat_port_statistics port_statistics; + +struct total_statistics { +uint64_t total_packets_dropped; +uint64_t total_packets_tx; +uint64_t total_packets_rx; +uint64_t total_successful_enqueues; +uint64_t total_failed_enqueues; +}; + typedef enum copy_mode_t { #define COPY_MODE_SW "sw" COPY_MODE_SW_NUM, -#define COPY_MODE_IOAT "rawdev" +#define COPY_MODE_IOAT "hw" COPY_MODE_IOAT_NUM, COPY_MODE_INVALID_NUM, COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
@@ -89,6 +106,204 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ];struct rte_mempool *ioat_pktmbuf_pool; +/* Print out statistics for one port. */ +static void +print_port_stats (uint16_t port_id) +{ +printf ("\nStatistics for port %u ------------------------------" +"\nPackets sent: %34" PRIu64 +"\nPackets received: %30" PRIu64 +"\nPackets dropped on tx: %25" PRIu64 +"\nPackets dropped on copy: %23" PRIu64, + port_id, + port_statistics.tx[port_id], + port_statistics.rx[port_id], + port_statistics.tx_dropped[port_id], + port_statistics.copy_dropped[port_id]); +} + +/* Print out statistics for one IOAT rawdev device. */ +static void +print_rawdev_stats (uint32_t dev_id,uint64_t *xstats, +unsigned int *ids_xstats,uint16_t nb_xstats, +struct rte_rawdev_xstats_name *names_xstats) +{ +uint16_t i; + +printf ("\nIOAT channel %u" , dev_id); +for (i =0 ; i < nb_xstats; i++) +printf ("\n\t %s: %*" PRIu64, + names_xstats[ids_xstats[i]].name, + (int )(37 -strlen (names_xstats[ids_xstats[i]].name)), + xstats[i]); +} + +static void +print_total_stats (struct total_statistics *ts) +{ +printf ("\nAggregate statistics ===============================" +"\nTotal packets Tx: %24" PRIu64" [pps]" +"\nTotal packets Rx: %24" PRIu64" [pps]" +"\nTotal packets dropped: %19" PRIu64" [pps]" , + ts->total_packets_tx, + ts->total_packets_rx, + ts->total_packets_dropped); + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +printf ("\nTotal IOAT successful enqueues: %8" PRIu64" [enq/s]" +"\nTotal IOAT failed enqueues: %12" PRIu64" [enq/s]" , + ts->total_successful_enqueues, + ts->total_failed_enqueues); + } + +printf ("\n====================================================\n" ); +} + +/* Print out statistics on packets dropped. */ +static void +print_stats (char *prgname) +{ +struct total_statistics ts, delta_ts; +uint32_t i, port_id, dev_id; +struct rte_rawdev_xstats_name *names_xstats; +uint64_t *xstats; +unsigned int *ids_xstats, nb_xstats; +char status_string[120 ];/* to print at the top of the output */ +int status_strlen; + + +const char clr[] = {27 ,'[' ,'2' ,'J' ,'\0' }; +const char topLeft[] = {27 ,'[' ,'1' ,';' ,'1' ,'H' ,'\0' }; + + status_strlen =snprintf (status_string,sizeof (status_string), +"%s, " , prgname); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Worker Threads = %d, " , +rte_lcore_count () >2 ?2 :1 ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Copy Mode = %s,\n" , copy_mode ==COPY_MODE_SW_NUM ? +COPY_MODE_SW :COPY_MODE_IOAT ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Updating MAC = %s, " , mac_updating ? +"enabled" :"disabled" ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Rx Queues = %d, " , nb_queues); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Ring Size = %d\n" , ring_size); + +/* Allocate memory for xstats names and values */ + nb_xstats =rte_rawdev_xstats_names_get ( + cfg.ports[0 ].ioat_ids[0 ],NULL ,0 ); + + names_xstats =malloc (sizeof (*names_xstats) * nb_xstats); +if (names_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat names memory\n" ); + } +rte_rawdev_xstats_names_get (cfg.ports[0 ].ioat_ids[0 ], + names_xstats, nb_xstats); + + ids_xstats =malloc (sizeof (*ids_xstats) *2 ); +if (ids_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat ids_xstats memory\n" ); + } + + xstats =malloc (sizeof (*xstats) *2 ); +if (xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat memory\n" ); + } + +/* Get failed/successful enqueues stats index */ + ids_xstats[0 ] = ids_xstats[1 ] = nb_xstats; +for (i =0 ; i < nb_xstats; i++) { +if (!strcmp (names_xstats[i].name,"failed_enqueues" )) + ids_xstats[0 ] = i; +else if (!strcmp (names_xstats[i].name,"successful_enqueues" )) + ids_xstats[1 ] = i; +if (ids_xstats[0 ] < nb_xstats && ids_xstats[1 ] < nb_xstats) +break ; + } +if (ids_xstats[0 ] == nb_xstats || ids_xstats[1 ] == nb_xstats) { +rte_exit (EXIT_FAILURE , +"Error getting failed/successful enqueues stats index\n" ); + } + +memset (&ts,0 ,sizeof (struct total_statistics )); + +while (!force_quit) { +/* Sleep for 1 second each round - init sleep allows reading +* messages from app startup. +*/ +sleep (1 ); + +/* Clear screen and move to top left */ +printf ("%s%s" , clr, topLeft); + +memset (&delta_ts,0 ,sizeof (struct total_statistics )); + +printf ("%s" , status_string); + +for (i =0 ; i < cfg.nb_ports; i++) { + port_id = cfg.ports[i].rxtx_port; +print_port_stats (port_id); + + delta_ts.total_packets_dropped += + port_statistics.tx_dropped[port_id] + + port_statistics.copy_dropped[port_id]; + delta_ts.total_packets_tx += + port_statistics.tx[port_id]; + delta_ts.total_packets_rx += + port_statistics.rx[port_id]; + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +uint32_t j; + +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { + dev_id = cfg.ports[i].ioat_ids[j]; +rte_rawdev_xstats_get (dev_id, + ids_xstats, xstats,2 ); + +print_rawdev_stats (dev_id, xstats, + ids_xstats,2 , names_xstats); + + delta_ts.total_failed_enqueues += + xstats[ids_xstats[0 ]]; + delta_ts.total_successful_enqueues += + xstats[ids_xstats[1 ]]; + } + } + } + + delta_ts.total_packets_tx -= ts.total_packets_tx; + delta_ts.total_packets_rx -= ts.total_packets_rx; + delta_ts.total_packets_dropped -= ts.total_packets_dropped; + delta_ts.total_failed_enqueues -= ts.total_failed_enqueues; + delta_ts.total_successful_enqueues -= + ts.total_successful_enqueues; + +printf ("\n" ); +print_total_stats (&delta_ts); + + ts.total_packets_tx += delta_ts.total_packets_tx; + ts.total_packets_rx += delta_ts.total_packets_rx; + ts.total_packets_dropped += delta_ts.total_packets_dropped; + ts.total_failed_enqueues += delta_ts.total_failed_enqueues; + ts.total_successful_enqueues += + delta_ts.total_successful_enqueues; + } + +free (names_xstats); +free (xstats); +free (ids_xstats); +} +static void update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) {
@@ -179,6 +394,8 @@ ioat_rx_port(struct rxtx_port_config *rx_config)if (nb_rx ==0 )continue ; + port_statistics.rx[rx_config->rxtx_port] += nb_rx; +if (copy_mode ==COPY_MODE_IOAT_NUM ) {/* Perform packet hardware copy */ nb_enq = ioat_enqueue_packets(pkts_burst,
@@ -213,6 +430,9 @@ ioat_rx_port(struct rxtx_port_config *rx_config) (void *)&pkts_burst_copy[nb_enq], nb_rx - nb_enq); } + + port_statistics.copy_dropped[rx_config->rxtx_port] += + (nb_rx - nb_enq); } }
@@ -248,6 +468,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config) tx_config->rxtx_port,0 , (void *)mbufs_dst, nb_dq); + port_statistics.tx[tx_config->rxtx_port] += nb_tx; +/* Free any unsent packets. */ if (unlikely (nb_tx < nb_dq)) rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -274,6 +496,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)rte_eth_tx_burst (tx_config->rxtx_port,0 , (void *)mbufs_dst, nb_dq); + port_statistics.tx[tx_config->rxtx_port] += nb_tx; +/* Free any unsent packets. */ if (unlikely (nb_tx < nb_dq)) rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -363,7 +587,7 @@ ioat_usage(const char *prgname)" When enabled:\n" " - The source MAC address is replaced by the TX port MAC address\n" " - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n" -" -c --copy-type CT: type of copy: sw|rawdev\n" +" -c --copy-type CT: type of copy: sw|hw\n" " -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n" , prgname); }
@@ -400,7 +624,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)static const char short_options[] ="p:" /* portmask */ "q:" /* number of RX queues per port */ -"c:" /* copy type (sw|rawdev) */ +"c:" /* copy type (sw|hw) */ "s:" /* ring size */ ;
@@ -452,7 +676,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)case 'c' : copy_mode =ioat_parse_copy_mode (optarg);if (copy_mode ==COPY_MODE_INVALID_NUM ) { -printf ("Invalid copy type. Use: sw, rawdev\n" ); +printf ("Invalid copy type. Use: sw, hw\n" );ioat_usage (prgname);return -1 ; }
@@ -673,6 +897,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)rte_eth_tx_buffer_init (tx_buffer[portid],MAX_PKT_BURST ); + ret =rte_eth_tx_buffer_set_err_callback (tx_buffer[portid], + rte_eth_tx_buffer_count_callback, + &port_statistics.tx_dropped[portid]); +if (ret <0 ) +rte_exit (EXIT_FAILURE , +"Cannot set error callback for tx buffer on port %u\n" , + portid); +/* Start device */ ret =rte_eth_dev_start (portid);if (ret <0 )
@@ -749,6 +981,9 @@ main(int argc, char **argv)RTE_ETH_FOREACH_DEV (portid)port_init (portid, ioat_pktmbuf_pool, nb_queues); +/* Initialize port xstats */ +memset (&port_statistics,0 ,sizeof (port_statistics)); +while (!check_link_status (ioat_enabled_port_mask) && !force_quit)sleep (1 );
@@ -764,6 +999,8 @@ main(int argc, char **argv)assign_rings ();start_forwarding_cores (); +/* master core prints stats while other cores forward */ +print_stats (argv[0 ]);/* force_quit is true when we get here */ rte_eal_mp_wait_lcore ();
Signature · 1 line
2.22.0.windows.1
PATCH v2 6/6 doc/guides/: provide IOAT sample app guide
In reply to Marcin BaranAdded guide for IOAT sample app usage and
code description.
--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++++++++++ 3 files changed, 769 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst
doc/guides/sample_app_ug/index.rst +1 -0
@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtual
doc/guides/sample_app_ug/intro.rst +4 -0
@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.
doc/guides/sample_app_ug/ioat.rst new file +764 -0
765 lines not on this page · raw message
Signature · 1 line
2.22.0.windows.1
Re: [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usage
In reply to Marcin BaranQuoted 5 lines · Marcin Baran
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
Please fix the dependencies:
https://travis-ci.com/ovsrobot/dpdk/jobs/234891633
examples/meson.build:89:4: ERROR: Problem encountered: Missing dependency "pmd_ioat" for example "ioat"
Quoted 29 lines
v2:
- change printing total stats to be deltas
- update documentation
- fix 1 thread/sw copy/multiple Rx queues packet dropping
- divide patch into several presenting functional changeMarcin Baran (4):
examples/ioat: add software copy support examples/ioat: add rawdev copy mode support examples/ioat: add stats printing for each port doc/guides/: provide IOAT sample app guidePawel Modrak (2):
examples/ioat: create sample app on ioat driver usage examples/ioat: add two threads configurationdoc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1025 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 13 + examples/meson.build | 1 + 8 files changed, 1865 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
Re: [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usage
-----Original Message-----
From: Aaron Conole <[email protected]>
Sent: Friday, September 13, 2019 8:45 PM
To: Baran, MarcinX <[email protected]>
Cc: [email protected]; Richardson, Bruce <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usage
Quoted 4 lines · Marcin Baran
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
Please fix the dependencies:
https://travis-ci.com/ovsrobot/dpdk/jobs/234891633
examples/meson.build:89:4: ERROR: Problem encountered: Missing dependency "pmd_ioat" for example "ioat"
[Marcin]: I will add check for dependency in example's meson.build for v3:
if not dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV')
build = false
subdir_done()
endif
Quoted 28 lines
v2:
- change printing total stats to be deltas
- update documentation
- fix 1 thread/sw copy/multiple Rx queues packet dropping
- divide patch into several presenting functional changeMarcin Baran (4):
examples/ioat: add software copy support
examples/ioat: add rawdev copy mode support
examples/ioat: add stats printing for each port
doc/guides/: provide IOAT sample app guidePawel Modrak (2):
examples/ioat: create sample app on ioat driver usage examples/ioat: add two threads configurationdoc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1025 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 13 + examples/meson.build | 1 + 8 files changed, 1865 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode100644 examples/ioat/Makefile create mode 100644
examples/ioat/ioatfwd.ccreate mode 100644 examples/ioat/meson.build
Re: [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usage
Quoted 18 lines · MarcinX
-----Original Message-----
From: Aaron Conole <[email protected]>
Sent: Friday, September 13, 2019 8:45 PM
To: Baran, MarcinX <[email protected]>
Cc: [email protected]; Richardson, Bruce <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usageQuoted 4 lines · Marcin Baran
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
Please fix the dependencies:
Quoted 8 lines
examples/meson.build:89:4: ERROR: Problem encountered: Missing dependency "
pmd_ioat" for example "ioat"[Marcin]: I will add check for dependency in example's meson.build for v3:
if not
dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV')
build = false
subdir_done()
endif
Seems it didn't work in v3:
https://travis-ci.com/ovsrobot/dpdk/builds/128147569
-Aaron
Quoted 28 lines
Quoted 28 lines
v2:
- change printing total stats to be deltas
- update documentation
- fix 1 thread/sw copy/multiple Rx queues packet dropping
- divide patch into several presenting functional changeMarcin Baran (4):
examples/ioat: add software copy support
examples/ioat: add rawdev copy mode support
examples/ioat: add stats printing for each port
doc/guides/: provide IOAT sample app guidePawel Modrak (2):
examples/ioat: create sample app on ioat driver usage examples/ioat: add two threads configurationdoc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1025 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 13 + examples/meson.build | 1 + 8 files changed, 1865 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode100644 examples/ioat/Makefile create mode 100644
examples/ioat/ioatfwd.ccreate mode 100644 examples/ioat/meson.build
v3 cover examples/ioat: sample app on ioat driver usage
In reply to Marcin BaranA new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
v3:
- add check for meson build whether IOAT is supported
v2:
- change printing total stats to be deltas
- update documentation
- fix 1 thread/sw copy/multiple Rx queues packet dropping
- divide patch into several presenting functional change
Marcin Baran (4):
examples/ioat: add software copy support examples/ioat: add rawdev copy mode support examples/ioat: add stats printing for each port doc/guides/: provide IOAT sample app guide
Pawel Modrak (2):
examples/ioat: create sample app on ioat driver usage examples/ioat: add two threads configuration
doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1025 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 18 + examples/meson.build | 1 + 8 files changed, 1870 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
Signature · 1 line
2.22.0.windows.1
PATCH v3 1/6 examples/ioat: create sample app on ioat driver usage
From: Pawel Modrak <[email protected]>
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The change covers ports initialization, closing connection and argument parsing.
--- examples/Makefile | 3 + examples/ioat/Makefile | 54 +++++ examples/ioat/ioatfwd.c | 439 ++++++++++++++++++++++++++++++++++++++ examples/ioat/meson.build | 18 ++ examples/meson.build | 1 + 5 files changed, 515 insertions(+) create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
examples/Makefile +3 -0
@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify DIRS-y += flow_filtering DIRS-y += helloworld +ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y) +DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat +endif DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline ifeq ($(CONFIG_RTE_LIBRTE_LPM),y) DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
examples/ioat/Makefile new file +54 -0
54 lines not on this page · raw message
examples/ioat/ioatfwd.c new file +439 -0
439 lines not on this page · raw message
examples/ioat/meson.build new file +18 -0
@@ -0,0 +1,18 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if not dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV') + build = false + subdir_done() +endif + +deps += ['pmd_ioat'] + +sources = files( + 'ioatfwd.c' +)
examples/meson.build +1 -0
@@ -16,6 +16,7 @@ all_examples = [ 'eventdev_pipeline', 'exception_path', 'fips_validation', 'flow_classify', 'flow_filtering', 'helloworld', + 'ioat', 'ip_fragmentation', 'ip_pipeline', 'ip_reassembly', 'ipsec-secgw', 'ipv4_multicast', 'kni',
Signature · 1 line
2.22.0.windows.1
PATCH v3 2/6 examples/ioat: add software copy support
In reply to Marcin BaranAdded support for copying packets
using software copy mode and MAC address
changing. The copies are processed using
one lcore.
--- examples/ioat/ioatfwd.c | 211 +++++++++++++++++++++++++++++++++++----- 1 file changed, 188 insertions(+), 23 deletions(-)
examples/ioat/ioatfwd.c +188 -23
@@ -13,7 +13,7 @@#include <rte_rawdev.h> #include <rte_ioat_rawdev.h> -/* size of ring used for software copying between rx and tx. */ +/* size of ring used for software copying between rx and tx. */ #define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1#define MAX_PKT_BURST 32#define MEMPOOL_CACHE_SIZE 512
@@ -89,6 +89,142 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ];struct rte_mempool *ioat_pktmbuf_pool; +static void +update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) +{ +struct rte_ether_hdr *eth; +void *tmp; + + eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *); + +/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but +* it's acceptable cause it gets overwritten by rte_ether_addr_copy +*/ + tmp = ð->d_addr.addr_bytes[0 ]; + *((uint64_t *)tmp) =0x000000000002 + ((uint64_t )dest_portid <<40 ); + +/* src addr */ +rte_ether_addr_copy (&ioat_ports_eth_addr[dest_portid], ð->s_addr); +} + +static inline void +pktmbuf_sw_copy (struct rte_mbuf *src,struct rte_mbuf *dst) +{ +/* Copy packet metadata */ +rte_memcpy (&dst->rearm_data, + &src->rearm_data, + offsetof(struct rte_mbuf , cacheline1) + - offsetof(struct rte_mbuf , rearm_data)); + +/* Copy packet data */ + rte_memcpy(rte_pktmbuf_mtod(dst, char *), + rte_pktmbuf_mtod(src, char *), src->data_len); +} + +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ +static void +ioat_rx_port (struct rxtx_port_config *rx_config) +{ +uint32_t nb_rx, nb_enq, i, j; +struct rte_mbuf *pkts_burst[MAX_PKT_BURST ]; + +for (i =0 ; i < rx_config->nb_queues; i++) { + + nb_rx =rte_eth_rx_burst (rx_config->rxtx_port, i, + pkts_burst,MAX_PKT_BURST ); + +if (nb_rx ==0 ) +continue ; + +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } +} + +/* Transmit packets from IOAT rawdev/rte_ring for one port. */ +static void +ioat_tx_port (struct rxtx_port_config *tx_config) +{ +uint32_t i, nb_dq =0 ; +struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ]; + +/* Deque the mbufs from rx_to_tx_ring. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); + +if (nb_dq ==0 ) +return ; + +/* Update macs if enabled */ +if (mac_updating) { +for (i =0 ; i < nb_dq; i++) +update_mac_addrs (mbufs_dst[i], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, +0 , (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); +} + +/* Main rx and tx loop if only one slave lcore available */ +static void +rxtx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx and tx loop for copy on" +" lcore %u\n" ,rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) { +ioat_rx_port (&cfg.ports[i]); +ioat_tx_port (&cfg.ports[i]); + } +} + +static void start_forwarding_cores (void ) +{ +uint32_t lcore_id =rte_lcore_id (); + +RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , + __func__,rte_lcore_id ()); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); +} +/* Display usage */ static void ioat_usage (const char *prgname)
@@ -102,7 +238,7 @@ ioat_usage(const char *prgname)" - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n" " -c --copy-type CT: type of copy: sw|rawdev\n" " -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n" , - prgname); + prgname); } static int
@@ -161,16 +297,16 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports) argvopt = argv;while ((opt =getopt_long (argc, argvopt, short_options, - lgopts, &option_index)) !=EOF ) { + lgopts, &option_index)) !=EOF ) {switch (opt) { -/* portmask */ +/* portmask */ case 'p' : ioat_enabled_port_mask =ioat_parse_portmask (optarg);if (ioat_enabled_port_mask & ~default_port_mask || - ioat_enabled_port_mask <=0 ) { + ioat_enabled_port_mask <=0 ) {printf ("Invalid portmask, %s, suggest 0x%x\n" , - optarg, default_port_mask); + optarg, default_port_mask);ioat_usage (prgname);return -1 ; }
@@ -204,7 +340,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports) }break ; -/* long options */ +/* long options */ case 0 :break ;
@@ -216,9 +352,9 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)printf ("MAC updating %s\n" , mac_updating ?"enabled" :"disabled" );if (optind >=0 ) - argv[optind-1 ] = prgname; + argv[optind -1 ] = prgname; - ret = optind-1 ; + ret = optind -1 ; optind =1 ;/* reset getopt lib */ return ret; }
@@ -253,6 +389,26 @@ check_link_status(uint32_t port_mask)return retval; } +static void +assign_rings (void ) +{ +uint32_t i; + +for (i =0 ; i < cfg.nb_ports; i++) { +char ring_name[RTE_RING_NAMESIZE ]; + +snprintf (ring_name,sizeof (ring_name),"rx_to_tx_ring_%u" , i); +/* Create ring for inter core communication */ + cfg.ports[i].rx_to_tx_ring =rte_ring_create ( + ring_name, ring_size, +rte_socket_id (),RING_F_SP_ENQ |RING_F_SC_DEQ ); + +if (cfg.ports[i].rx_to_tx_ring ==NULL ) +rte_exit (EXIT_FAILURE ,"Ring create failed: %s\n" , +rte_strerror (rte_errno)); + } +} +/* * Initializes a given port using global settings and with the RX buffers * coming from the mbuf_pool passed as a parameter.
@@ -263,7 +419,7 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)/* configuring port to use RSS for multiple RX queues */ static const struct rte_eth_conf port_conf = { .rxmode = { - .mq_mode =ETH_MQ_RX_RSS , + .mq_mode =ETH_MQ_RX_RSS , .max_rx_pkt_len =RTE_ETHER_MAX_LEN }, .rx_adv_conf = {
@@ -294,14 +450,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues) dev_info.flow_type_rss_offloads;if (dev_info.tx_offload_capa &DEV_TX_OFFLOAD_MBUF_FAST_FREE ) local_port_conf.txmode.offloads |= -DEV_TX_OFFLOAD_MBUF_FAST_FREE ; +DEV_TX_OFFLOAD_MBUF_FAST_FREE ; ret =rte_eth_dev_configure (portid, nb_queues,1 , &local_port_conf);if (ret <0 )rte_exit (EXIT_FAILURE ,"Cannot configure device:" " err=%d, port=%u\n" , ret, portid); ret =rte_eth_dev_adjust_nb_rx_tx_desc (portid, &nb_rxd, - &nb_txd); + &nb_txd);if (ret <0 ) rte_exit(EXIT_FAILURE, "Cannot adjust number of descriptors: err=%d, port=%u\n",
@@ -326,8 +482,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues) txq_conf = dev_info.default_txconf; txq_conf.offloads = local_port_conf.txmode.offloads; ret =rte_eth_tx_queue_setup (portid,0 , nb_txd, -rte_eth_dev_socket_id (portid), - &txq_conf); +rte_eth_dev_socket_id (portid), + &txq_conf);if (ret <0 ) rte_exit(EXIT_FAILURE, "rte_eth_tx_queue_setup:err=%d,port=%u\n",
@@ -335,8 +491,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)/* Initialize TX buffers */ tx_buffer[portid] =rte_zmalloc_socket ("tx_buffer" , -RTE_ETH_TX_BUFFER_SIZE (MAX_PKT_BURST ),0 , -rte_eth_dev_socket_id (portid)); +RTE_ETH_TX_BUFFER_SIZE (MAX_PKT_BURST ),0 , +rte_eth_dev_socket_id (portid));if (tx_buffer[portid] ==NULL ) rte_exit(EXIT_FAILURE, "Cannot allocate buffer for tx on port %u\n",
@@ -354,13 +510,13 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)rte_eth_promiscuous_enable (portid);printf ("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n" , - portid, - ioat_ports_eth_addr[portid].addr_bytes[0 ], - ioat_ports_eth_addr[portid].addr_bytes[1 ], - ioat_ports_eth_addr[portid].addr_bytes[2 ], - ioat_ports_eth_addr[portid].addr_bytes[3 ], - ioat_ports_eth_addr[portid].addr_bytes[4 ], - ioat_ports_eth_addr[portid].addr_bytes[5 ]); + portid, + ioat_ports_eth_addr[portid].addr_bytes[0 ], + ioat_ports_eth_addr[portid].addr_bytes[1 ], + ioat_ports_eth_addr[portid].addr_bytes[2 ], + ioat_ports_eth_addr[portid].addr_bytes[3 ], + ioat_ports_eth_addr[portid].addr_bytes[4 ], + ioat_ports_eth_addr[portid].addr_bytes[5 ]); cfg.ports[cfg.nb_ports].rxtx_port = portid; cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
@@ -428,10 +584,19 @@ main(int argc, char **argv)if (cfg.nb_lcores <1 )rte_exit (EXIT_FAILURE ,"There should be at least one slave lcore.\n" ); + +assign_rings (); + +start_forwarding_cores (); + +/* force_quit is true when we get here */ +rte_eal_mp_wait_lcore (); +for (i =0 ; i < cfg.nb_ports; i++) {printf ("Closing port %d\n" , cfg.ports[i].rxtx_port);rte_eth_dev_stop (cfg.ports[i].rxtx_port);rte_eth_dev_close (cfg.ports[i].rxtx_port); +rte_ring_free (cfg.ports[i].rx_to_tx_ring); }printf ("Bye...\n" );
Signature · 1 line
2.22.0.windows.1
PATCH v3 3/6 examples/ioat: add rawdev copy mode support
In reply to Marcin BaranAdded support for copying packets using
rawdev device. Each port's Rx queue is
assigned DMA channel for copy.
--- examples/ioat/ioatfwd.c | 236 ++++++++++++++++++++++++++++++++-------- 1 file changed, 189 insertions(+), 47 deletions(-)
examples/ioat/ioatfwd.c +189 -47
@@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) rte_pktmbuf_mtod(src, char *), src->data_len); } +static uint32_t +ioat_enqueue_packets (struct rte_mbuf **pkts, +uint32_t nb_rx,uint16_t dev_id) +{ +int ret; +uint32_t i; +struct rte_mbuf *pkts_copy[MAX_PKT_BURST ]; + +const uint64_t addr_offset =RTE_PTR_DIFF (pkts[0 ]->buf_addr, + &pkts[0 ]->rearm_data); + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE ,"Unable to allocate memory.\n" ); + +for (i =0 ; i < nb_rx; i++) { +/* Perform data copy */ + ret =rte_ioat_enqueue_copy (dev_id, + pkts[i]->buf_iova + - addr_offset, + pkts_copy[i]->buf_iova + - addr_offset, +rte_pktmbuf_data_len (pkts[i]) + + addr_offset, + (uintptr_t )pkts[i], + (uintptr_t )pkts_copy[i], +0 /* nofence */ ); + +if (ret !=1 ) +break ; + } + + ret = i; +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i); +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], + nb_rx - i); + + +return ret; +} +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ static void ioat_rx_port (struct rxtx_port_config *rx_config)
@@ -136,32 +180,40 @@ ioat_rx_port(struct rxtx_port_config *rx_config)if (nb_rx ==0 )continue ; -/* Perform packet software copy, free source packets */ -int ret; -struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; - - ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, - (void *)pkts_burst_copy, nb_rx); - -if (unlikely (ret <0 )) -rte_exit (EXIT_FAILURE , -"Unable to allocate memory.\n" ); - -for (j =0 ; j < nb_rx; j++) -pktmbuf_sw_copy (pkts_burst[j], - pkts_burst_copy[j]); - -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)pkts_burst, nb_rx); - - nb_enq =rte_ring_enqueue_burst ( - rx_config->rx_to_tx_ring, - (void *)pkts_burst_copy, nb_rx,NULL ); - -/* Free any not enqueued packets. */ -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)&pkts_burst_copy[nb_enq], - nb_rx - nb_enq); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Perform packet hardware copy */ + nb_enq =ioat_enqueue_packets (pkts_burst, + nb_rx, rx_config->ioat_ids[i]); +if (nb_enq >0 ) +rte_ioat_do_copies (rx_config->ioat_ids[i]); + }else { +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } } }
@@ -169,31 +221,64 @@ ioat_rx_port(struct rxtx_port_config *rx_config)static void ioat_tx_port (struct rxtx_port_config *tx_config) { -uint32_t i, nb_dq =0 ; +uint32_t i, j, nb_dq =0 ; +struct rte_mbuf *mbufs_src[MAX_PKT_BURST ];struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ]; -/* Deque the mbufs from rx_to_tx_ring. */ - nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, - (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Deque the mbufs from IOAT device. */ +for (i =0 ; i < tx_config->nb_queues; i++) { + nb_dq =rte_ioat_completed_copies ( + tx_config->ioat_ids[i],MAX_PKT_BURST , + (void *)mbufs_src, (void *)mbufs_dst); -if (nb_dq ==0 ) -return ; +if (nb_dq ==0 ) +break ; -/* Update macs if enabled */ -if (mac_updating) { -for (i =0 ; i < nb_dq; i++) -update_mac_addrs (mbufs_dst[i], - tx_config->rxtx_port); - } +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)mbufs_src, nb_dq); -const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, -0 , (void *)mbufs_dst, nb_dq); +/* Update macs if enabled */ +if (mac_updating) { +for (j =0 ; j < nb_dq; j++) +update_mac_addrs (mbufs_dst[j], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst ( + tx_config->rxtx_port,0 , + (void *)mbufs_dst, nb_dq); -/* Free any unsent packets. */ -if (unlikely (nb_tx < nb_dq)) -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)&mbufs_dst[nb_tx], - nb_dq - nb_tx); +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } + }else { +/* Deque the mbufs from rx_to_tx_ring. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); + +if (nb_dq ==0 ) +return ; + +/* Update macs if enabled */ +if (mac_updating) { +for (i =0 ; i < nb_dq; i++) +update_mac_addrs (mbufs_dst[i], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, +0 , (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } }/* Main rx and tx loop if only one slave lcore available */
@@ -389,6 +474,52 @@ check_link_status(uint32_t port_mask)return retval; } +static void +configure_rawdev_queue (uint32_t dev_id) +{ +struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size }; +struct rte_rawdev_info info = { .dev_private = &dev_config }; + +if (rte_rawdev_configure (dev_id, &info) !=0 ) { +rte_exit (EXIT_FAILURE , +"Error with rte_rawdev_configure()\n" ); + } +if (rte_rawdev_start (dev_id) !=0 ) { +rte_exit (EXIT_FAILURE , +"Error with rte_rawdev_start()\n" ); + } +} + +static void +assign_rawdevs (void ) +{ +uint16_t nb_rawdev =0 , rdev_id =0 ; +uint32_t i, j; + +for (i =0 ; i < cfg.nb_ports; i++) { +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { +struct rte_rawdev_info rdev_info = {0 }; + +do { +if (rdev_id ==rte_rawdev_count ()) + goto end; +rte_rawdev_info_get (rdev_id++, &rdev_info); + }while (strcmp (rdev_info.driver_name, +IOAT_PMD_RAWDEV_NAME_STR ) !=0 ); + + cfg.ports[i].ioat_ids[j] = rdev_id -1 ; +configure_rawdev_queue (cfg.ports[i].ioat_ids[j]); + ++nb_rawdev; + } + } +end: +if (nb_rawdev < cfg.nb_ports * cfg.ports[0 ].nb_queues) +rte_exit (EXIT_FAILURE , +"Not enough IOAT rawdevs (%u) for all queues (%u).\n" , + nb_rawdev, cfg.nb_ports * cfg.ports[0 ].nb_queues); +RTE_LOG (INFO ,IOAT ,"Number of used rawdevs: %u.\n" , nb_rawdev); +} +static void assign_rings (void ) {
@@ -585,18 +716,29 @@ main(int argc, char **argv)rte_exit (EXIT_FAILURE ,"There should be at least one slave lcore.\n" ); -assign_rings (); +if (copy_mode ==COPY_MODE_IOAT_NUM ) +assign_rawdevs (); +else /* copy_mode == COPY_MODE_SW_NUM */ +assign_rings ();start_forwarding_cores ();/* force_quit is true when we get here */ rte_eal_mp_wait_lcore (); +uint32_t j;for (i =0 ; i < cfg.nb_ports; i++) {printf ("Closing port %d\n" , cfg.ports[i].rxtx_port);rte_eth_dev_stop (cfg.ports[i].rxtx_port);rte_eth_dev_close (cfg.ports[i].rxtx_port); -rte_ring_free (cfg.ports[i].rx_to_tx_ring); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { +printf ("Stopping rawdev %d\n" , + cfg.ports[i].ioat_ids[j]); +rte_rawdev_stop (cfg.ports[i].ioat_ids[j]); + } + }else /* copy_mode == COPY_MODE_SW_NUM */ +rte_ring_free (cfg.ports[i].rx_to_tx_ring); }printf ("Bye...\n" );
Signature · 1 line
2.22.0.windows.1
PATCH v3 4/6 examples/ioat: add two threads configuration
In reply to Marcin BaranFrom: Pawel Modrak <[email protected]>
Added possibility to use two lcores: first for
packet receiving and copying, second for packets
sending.
--- examples/ioat/ioatfwd.c | 92 ++++++++++++++++++++++++++++++----------- 1 file changed, 67 insertions(+), 25 deletions(-)
examples/ioat/ioatfwd.c +67 -25
@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], nb_rx - i); -return ret; }
@@ -226,8 +225,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ];if (copy_mode ==COPY_MODE_IOAT_NUM ) { -/* Deque the mbufs from IOAT device. */ for (i =0 ; i < tx_config->nb_queues; i++) { +/* Deque the mbufs from IOAT device. */ nb_dq =rte_ioat_completed_copies ( tx_config->ioat_ids[i],MAX_PKT_BURST , (void *)mbufs_src, (void *)mbufs_dst);
@@ -256,31 +255,64 @@ ioat_tx_port(struct rxtx_port_config *tx_config) nb_dq - nb_tx); } }else { -/* Deque the mbufs from rx_to_tx_ring. */ - nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, - (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); - -if (nb_dq ==0 ) -return ; - -/* Update macs if enabled */ -if (mac_updating) { -for (i =0 ; i < nb_dq; i++) -update_mac_addrs (mbufs_dst[i], - tx_config->rxtx_port); - } +for (i =0 ; i < tx_config->nb_queues; i++) { +/* Deque the mbufs from IOAT device. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); -const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, -0 , (void *)mbufs_dst, nb_dq); +if (nb_dq ==0 ) +return ; -/* Free any unsent packets. */ -if (unlikely (nb_tx < nb_dq)) -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)&mbufs_dst[nb_tx], - nb_dq - nb_tx); +/* Update macs if enabled */ +if (mac_updating) { +for (j =0 ; j < nb_dq; j++) +update_mac_addrs (mbufs_dst[j], + tx_config->rxtx_port); + } + +const uint16_t nb_tx = +rte_eth_tx_burst (tx_config->rxtx_port,0 , + (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } } } +/* Main rx processing loop for IOAT rawdev. */ +static void +rx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx loop for copy on lcore %u\n" , +rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) +ioat_rx_port (&cfg.ports[i]); +} + +/* Main tx processing loop for hardware copy. */ +static void +tx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main tx loop for copy on lcore %u\n" , +rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) +ioat_tx_port (&cfg.ports[i]); +} +/* Main rx and tx loop if only one slave lcore available */ static void rxtx_main_loop (void )
@@ -305,9 +337,19 @@ static void start_forwarding_cores(void)RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , __func__,rte_lcore_id ()); - lcore_id =rte_get_next_lcore (lcore_id, true, true); -rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, -NULL , lcore_id); +if (cfg.nb_lcores ==1 ) { + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); + }else if (cfg.nb_lcores >1 ) { + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rx_main_loop, +NULL , lcore_id); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)tx_main_loop,NULL , + lcore_id); + } }/* Display usage */
Signature · 1 line
2.22.0.windows.1
PATCH v3 5/6 examples/ioat: add stats printing for each port
In reply to Marcin BaranAdded printing stats of ports each second. The stats printing is done using master core. The information provided informs about packets received, dropped and send as well as statistics of rawdev devices.
--- examples/ioat/ioatfwd.c | 245 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 241 insertions(+), 4 deletions(-)
examples/ioat/ioatfwd.c +241 -4
@@ -48,10 +48,27 @@ struct rxtx_transmission_config {uint16_t nb_lcores; }; +/* per-port statistics struct */ +struct ioat_port_statistics { +uint64_t rx[RTE_MAX_ETHPORTS ]; +uint64_t tx[RTE_MAX_ETHPORTS ]; +uint64_t tx_dropped[RTE_MAX_ETHPORTS ]; +uint64_t copy_dropped[RTE_MAX_ETHPORTS ]; +}; +struct ioat_port_statistics port_statistics; + +struct total_statistics { +uint64_t total_packets_dropped; +uint64_t total_packets_tx; +uint64_t total_packets_rx; +uint64_t total_successful_enqueues; +uint64_t total_failed_enqueues; +}; + typedef enum copy_mode_t { #define COPY_MODE_SW "sw" COPY_MODE_SW_NUM, -#define COPY_MODE_IOAT "rawdev" +#define COPY_MODE_IOAT "hw" COPY_MODE_IOAT_NUM, COPY_MODE_INVALID_NUM, COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
@@ -89,6 +106,204 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ];struct rte_mempool *ioat_pktmbuf_pool; +/* Print out statistics for one port. */ +static void +print_port_stats (uint16_t port_id) +{ +printf ("\nStatistics for port %u ------------------------------" +"\nPackets sent: %34" PRIu64 +"\nPackets received: %30" PRIu64 +"\nPackets dropped on tx: %25" PRIu64 +"\nPackets dropped on copy: %23" PRIu64, + port_id, + port_statistics.tx[port_id], + port_statistics.rx[port_id], + port_statistics.tx_dropped[port_id], + port_statistics.copy_dropped[port_id]); +} + +/* Print out statistics for one IOAT rawdev device. */ +static void +print_rawdev_stats (uint32_t dev_id,uint64_t *xstats, +unsigned int *ids_xstats,uint16_t nb_xstats, +struct rte_rawdev_xstats_name *names_xstats) +{ +uint16_t i; + +printf ("\nIOAT channel %u" , dev_id); +for (i =0 ; i < nb_xstats; i++) +printf ("\n\t %s: %*" PRIu64, + names_xstats[ids_xstats[i]].name, + (int )(37 -strlen (names_xstats[ids_xstats[i]].name)), + xstats[i]); +} + +static void +print_total_stats (struct total_statistics *ts) +{ +printf ("\nAggregate statistics ===============================" +"\nTotal packets Tx: %24" PRIu64" [pps]" +"\nTotal packets Rx: %24" PRIu64" [pps]" +"\nTotal packets dropped: %19" PRIu64" [pps]" , + ts->total_packets_tx, + ts->total_packets_rx, + ts->total_packets_dropped); + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +printf ("\nTotal IOAT successful enqueues: %8" PRIu64" [enq/s]" +"\nTotal IOAT failed enqueues: %12" PRIu64" [enq/s]" , + ts->total_successful_enqueues, + ts->total_failed_enqueues); + } + +printf ("\n====================================================\n" ); +} + +/* Print out statistics on packets dropped. */ +static void +print_stats (char *prgname) +{ +struct total_statistics ts, delta_ts; +uint32_t i, port_id, dev_id; +struct rte_rawdev_xstats_name *names_xstats; +uint64_t *xstats; +unsigned int *ids_xstats, nb_xstats; +char status_string[120 ];/* to print at the top of the output */ +int status_strlen; + + +const char clr[] = {27 ,'[' ,'2' ,'J' ,'\0' }; +const char topLeft[] = {27 ,'[' ,'1' ,';' ,'1' ,'H' ,'\0' }; + + status_strlen =snprintf (status_string,sizeof (status_string), +"%s, " , prgname); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Worker Threads = %d, " , +rte_lcore_count () >2 ?2 :1 ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Copy Mode = %s,\n" , copy_mode ==COPY_MODE_SW_NUM ? +COPY_MODE_SW :COPY_MODE_IOAT ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Updating MAC = %s, " , mac_updating ? +"enabled" :"disabled" ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Rx Queues = %d, " , nb_queues); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Ring Size = %d\n" , ring_size); + +/* Allocate memory for xstats names and values */ + nb_xstats =rte_rawdev_xstats_names_get ( + cfg.ports[0 ].ioat_ids[0 ],NULL ,0 ); + + names_xstats =malloc (sizeof (*names_xstats) * nb_xstats); +if (names_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat names memory\n" ); + } +rte_rawdev_xstats_names_get (cfg.ports[0 ].ioat_ids[0 ], + names_xstats, nb_xstats); + + ids_xstats =malloc (sizeof (*ids_xstats) *2 ); +if (ids_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat ids_xstats memory\n" ); + } + + xstats =malloc (sizeof (*xstats) *2 ); +if (xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat memory\n" ); + } + +/* Get failed/successful enqueues stats index */ + ids_xstats[0 ] = ids_xstats[1 ] = nb_xstats; +for (i =0 ; i < nb_xstats; i++) { +if (!strcmp (names_xstats[i].name,"failed_enqueues" )) + ids_xstats[0 ] = i; +else if (!strcmp (names_xstats[i].name,"successful_enqueues" )) + ids_xstats[1 ] = i; +if (ids_xstats[0 ] < nb_xstats && ids_xstats[1 ] < nb_xstats) +break ; + } +if (ids_xstats[0 ] == nb_xstats || ids_xstats[1 ] == nb_xstats) { +rte_exit (EXIT_FAILURE , +"Error getting failed/successful enqueues stats index\n" ); + } + +memset (&ts,0 ,sizeof (struct total_statistics )); + +while (!force_quit) { +/* Sleep for 1 second each round - init sleep allows reading +* messages from app startup. +*/ +sleep (1 ); + +/* Clear screen and move to top left */ +printf ("%s%s" , clr, topLeft); + +memset (&delta_ts,0 ,sizeof (struct total_statistics )); + +printf ("%s" , status_string); + +for (i =0 ; i < cfg.nb_ports; i++) { + port_id = cfg.ports[i].rxtx_port; +print_port_stats (port_id); + + delta_ts.total_packets_dropped += + port_statistics.tx_dropped[port_id] + + port_statistics.copy_dropped[port_id]; + delta_ts.total_packets_tx += + port_statistics.tx[port_id]; + delta_ts.total_packets_rx += + port_statistics.rx[port_id]; + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +uint32_t j; + +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { + dev_id = cfg.ports[i].ioat_ids[j]; +rte_rawdev_xstats_get (dev_id, + ids_xstats, xstats,2 ); + +print_rawdev_stats (dev_id, xstats, + ids_xstats,2 , names_xstats); + + delta_ts.total_failed_enqueues += + xstats[ids_xstats[0 ]]; + delta_ts.total_successful_enqueues += + xstats[ids_xstats[1 ]]; + } + } + } + + delta_ts.total_packets_tx -= ts.total_packets_tx; + delta_ts.total_packets_rx -= ts.total_packets_rx; + delta_ts.total_packets_dropped -= ts.total_packets_dropped; + delta_ts.total_failed_enqueues -= ts.total_failed_enqueues; + delta_ts.total_successful_enqueues -= + ts.total_successful_enqueues; + +printf ("\n" ); +print_total_stats (&delta_ts); + + ts.total_packets_tx += delta_ts.total_packets_tx; + ts.total_packets_rx += delta_ts.total_packets_rx; + ts.total_packets_dropped += delta_ts.total_packets_dropped; + ts.total_failed_enqueues += delta_ts.total_failed_enqueues; + ts.total_successful_enqueues += + delta_ts.total_successful_enqueues; + } + +free (names_xstats); +free (xstats); +free (ids_xstats); +} +static void update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) {
@@ -179,6 +394,8 @@ ioat_rx_port(struct rxtx_port_config *rx_config)if (nb_rx ==0 )continue ; + port_statistics.rx[rx_config->rxtx_port] += nb_rx; +if (copy_mode ==COPY_MODE_IOAT_NUM ) {/* Perform packet hardware copy */ nb_enq = ioat_enqueue_packets(pkts_burst,
@@ -213,6 +430,9 @@ ioat_rx_port(struct rxtx_port_config *rx_config) (void *)&pkts_burst_copy[nb_enq], nb_rx - nb_enq); } + + port_statistics.copy_dropped[rx_config->rxtx_port] += + (nb_rx - nb_enq); } }
@@ -248,6 +468,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config) tx_config->rxtx_port,0 , (void *)mbufs_dst, nb_dq); + port_statistics.tx[tx_config->rxtx_port] += nb_tx; +/* Free any unsent packets. */ if (unlikely (nb_tx < nb_dq)) rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -274,6 +496,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)rte_eth_tx_burst (tx_config->rxtx_port,0 , (void *)mbufs_dst, nb_dq); + port_statistics.tx[tx_config->rxtx_port] += nb_tx; +/* Free any unsent packets. */ if (unlikely (nb_tx < nb_dq)) rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -363,7 +587,7 @@ ioat_usage(const char *prgname)" When enabled:\n" " - The source MAC address is replaced by the TX port MAC address\n" " - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n" -" -c --copy-type CT: type of copy: sw|rawdev\n" +" -c --copy-type CT: type of copy: sw|hw\n" " -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n" , prgname); }
@@ -400,7 +624,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)static const char short_options[] ="p:" /* portmask */ "q:" /* number of RX queues per port */ -"c:" /* copy type (sw|rawdev) */ +"c:" /* copy type (sw|hw) */ "s:" /* ring size */ ;
@@ -452,7 +676,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)case 'c' : copy_mode =ioat_parse_copy_mode (optarg);if (copy_mode ==COPY_MODE_INVALID_NUM ) { -printf ("Invalid copy type. Use: sw, rawdev\n" ); +printf ("Invalid copy type. Use: sw, hw\n" );ioat_usage (prgname);return -1 ; }
@@ -673,6 +897,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)rte_eth_tx_buffer_init (tx_buffer[portid],MAX_PKT_BURST ); + ret =rte_eth_tx_buffer_set_err_callback (tx_buffer[portid], + rte_eth_tx_buffer_count_callback, + &port_statistics.tx_dropped[portid]); +if (ret <0 ) +rte_exit (EXIT_FAILURE , +"Cannot set error callback for tx buffer on port %u\n" , + portid); +/* Start device */ ret =rte_eth_dev_start (portid);if (ret <0 )
@@ -749,6 +981,9 @@ main(int argc, char **argv)RTE_ETH_FOREACH_DEV (portid)port_init (portid, ioat_pktmbuf_pool, nb_queues); +/* Initialize port xstats */ +memset (&port_statistics,0 ,sizeof (port_statistics)); +while (!check_link_status (ioat_enabled_port_mask) && !force_quit)sleep (1 );
@@ -764,6 +999,8 @@ main(int argc, char **argv)assign_rings ();start_forwarding_cores (); +/* master core prints stats while other cores forward */ +print_stats (argv[0 ]);/* force_quit is true when we get here */ rte_eal_mp_wait_lcore ();
Signature · 1 line
2.22.0.windows.1
PATCH v3 6/6 doc/guides/: provide IOAT sample app guide
In reply to Marcin BaranAdded guide for IOAT sample app usage and
code description.
--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++++++++++ 3 files changed, 769 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst
doc/guides/sample_app_ug/index.rst +1 -0
@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtual
doc/guides/sample_app_ug/intro.rst +4 -0
@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.
doc/guides/sample_app_ug/ioat.rst new file +764 -0
765 lines not on this page · raw message
Signature · 1 line
2.22.0.windows.1
v4 cover examples/ioat: sample app on ioat driver usage
In reply to Marcin BaranA new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
v4:
- fix meson build support check
v3:
- add check for meson build whether IOAT is supported
v2:
- change printing total stats to be deltas
- update documentation
- fix 1 thread/sw copy/multiple Rx queues packet dropping
- divide patch into several presenting functional change
Marcin Baran (4):
examples/ioat: add software copy support examples/ioat: add rawdev copy mode support examples/ioat: add stats printing for each port doc/guides/: provide IOAT sample app guide
Pawel Modrak (2):
examples/ioat: create sample app on ioat driver usage examples/ioat: add two threads configuration
doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1025 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 23 + examples/meson.build | 1 + 8 files changed, 1875 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
Signature · 1 line
2.22.0.windows.1
PATCH v4 1/6 examples/ioat: create sample app on ioat driver usage
From: Pawel Modrak <[email protected]>
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The change covers ports initialization, closing connection and argument parsing.
--- examples/Makefile | 3 + examples/ioat/Makefile | 54 +++++ examples/ioat/ioatfwd.c | 439 ++++++++++++++++++++++++++++++++++++++ examples/ioat/meson.build | 23 ++ examples/meson.build | 1 + 5 files changed, 520 insertions(+) create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
examples/Makefile +3 -0
@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify DIRS-y += flow_filtering DIRS-y += helloworld +ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y) +DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat +endif DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline ifeq ($(CONFIG_RTE_LIBRTE_LPM),y) DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
examples/ioat/Makefile new file +54 -0
54 lines not on this page · raw message
examples/ioat/ioatfwd.c new file +439 -0
439 lines not on this page · raw message
examples/ioat/meson.build new file +23 -0
@@ -0,0 +1,23 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if not dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV') + build = false + subdir_done() +endif + +if dpdk_conf.get('RTE_LIBRTE_PMD_IOAT_RAWDEV', 0) != 1 + build = false + subdir_done() +endif + +deps += ['pmd_ioat'] + +sources = files( + 'ioatfwd.c' +)
examples/meson.build +1 -0
@@ -16,6 +16,7 @@ all_examples = [ 'eventdev_pipeline', 'exception_path', 'fips_validation', 'flow_classify', 'flow_filtering', 'helloworld', + 'ioat', 'ip_fragmentation', 'ip_pipeline', 'ip_reassembly', 'ipsec-secgw', 'ipv4_multicast', 'kni',
Signature · 1 line
2.22.0.windows.1
Quoted 11 lines · Marcin Baran
From: Pawel Modrak <[email protected]>
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The change covers ports initialization, closing connection and argument parsing.
Signed-off-by Pawel Modrak <[email protected]>Signed-off-by Marcin Baran <[email protected]>---
<snip>
Quoted 13 lines
examples/ioat/meson.build new file +10
-0@@ -0,0 +1,23 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if not dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV') + build = false
examples/ioat/meson.build line 20
Quoted 10 lines
examples/ioat/meson.build new file +10
-0+ subdir_done() +endif + +if dpdk_conf.get('RTE_LIBRTE_PMD_IOAT_RAWDEV', 0) != 1 + build = false + subdir_done() +endif + +deps += ['pmd_ioat'] +
Since commit '54dcfdee85aa ("drivers/raw: standardize naming")' the naming has changed a little so this needs to be "rawdev_ioat" rather than "pmd_ioat". I believe this is causing some of the CI error reports.
Regards,
/Bruce
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Thursday, September 19, 2019 4:44 PM
To: Baran, MarcinX <[email protected]>
Cc: [email protected]; Modrak, PawelX <[email protected]>
Subject: Re: [PATCH v4 1/6] examples/ioat: create sample app on ioat driver usage
Quoted 10 lines · Marcin Baran
From: Pawel Modrak <[email protected]>
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The change covers ports initialization, closing connection and argument parsing.
Signed-off-by Pawel Modrak <[email protected]>Signed-off-by Marcin Baran <[email protected]>---
<snip>
Quoted 13 lines
examples/ioat/meson.build new file +10
-0@@ -0,0 +1,23 @@ +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2019 Intel +Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an +already-installed # DPDK instance, use 'make' + +if not dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV') + build = false
examples/ioat/meson.build line 20
Quoted 10 lines
examples/ioat/meson.build new file +10
-0+ subdir_done() +endif + +if dpdk_conf.get('RTE_LIBRTE_PMD_IOAT_RAWDEV', 0) != 1 + build = false + subdir_done() +endif + +deps += ['pmd_ioat'] +
Since commit '54dcfdee85aa ("drivers/raw: standardize naming")' the naming has changed a little so this needs to be "rawdev_ioat" rather than "pmd_ioat". I believe this is causing some of the CI error reports. Regards,
/Bruce
[MARCIN] Yes, I noticed that just now. I am generating new patches with the fix.
PATCH v4 2/6 examples/ioat: add software copy support
In reply to Marcin BaranAdded support for copying packets
using software copy mode and MAC address
changing. The copies are processed using
one lcore.
--- examples/ioat/ioatfwd.c | 211 +++++++++++++++++++++++++++++++++++----- 1 file changed, 188 insertions(+), 23 deletions(-)
examples/ioat/ioatfwd.c +188 -23
@@ -13,7 +13,7 @@#include <rte_rawdev.h> #include <rte_ioat_rawdev.h> -/* size of ring used for software copying between rx and tx. */ +/* size of ring used for software copying between rx and tx. */ #define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1#define MAX_PKT_BURST 32#define MEMPOOL_CACHE_SIZE 512
@@ -89,6 +89,142 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ];struct rte_mempool *ioat_pktmbuf_pool; +static void +update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) +{ +struct rte_ether_hdr *eth; +void *tmp; + + eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *); + +/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but +* it's acceptable cause it gets overwritten by rte_ether_addr_copy +*/ + tmp = ð->d_addr.addr_bytes[0 ]; + *((uint64_t *)tmp) =0x000000000002 + ((uint64_t )dest_portid <<40 ); + +/* src addr */ +rte_ether_addr_copy (&ioat_ports_eth_addr[dest_portid], ð->s_addr); +} + +static inline void +pktmbuf_sw_copy (struct rte_mbuf *src,struct rte_mbuf *dst) +{ +/* Copy packet metadata */ +rte_memcpy (&dst->rearm_data, + &src->rearm_data, + offsetof(struct rte_mbuf , cacheline1) + - offsetof(struct rte_mbuf , rearm_data)); + +/* Copy packet data */ + rte_memcpy(rte_pktmbuf_mtod(dst, char *), + rte_pktmbuf_mtod(src, char *), src->data_len); +} + +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ +static void +ioat_rx_port (struct rxtx_port_config *rx_config) +{ +uint32_t nb_rx, nb_enq, i, j; +struct rte_mbuf *pkts_burst[MAX_PKT_BURST ]; + +for (i =0 ; i < rx_config->nb_queues; i++) { + + nb_rx =rte_eth_rx_burst (rx_config->rxtx_port, i, + pkts_burst,MAX_PKT_BURST ); + +if (nb_rx ==0 ) +continue ; + +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } +} + +/* Transmit packets from IOAT rawdev/rte_ring for one port. */ +static void +ioat_tx_port (struct rxtx_port_config *tx_config) +{ +uint32_t i, nb_dq =0 ; +struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ]; + +/* Deque the mbufs from rx_to_tx_ring. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); + +if (nb_dq ==0 ) +return ; + +/* Update macs if enabled */ +if (mac_updating) { +for (i =0 ; i < nb_dq; i++) +update_mac_addrs (mbufs_dst[i], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, +0 , (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); +} + +/* Main rx and tx loop if only one slave lcore available */ +static void +rxtx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx and tx loop for copy on" +" lcore %u\n" ,rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) { +ioat_rx_port (&cfg.ports[i]); +ioat_tx_port (&cfg.ports[i]); + } +} + +static void start_forwarding_cores (void ) +{ +uint32_t lcore_id =rte_lcore_id (); + +RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , + __func__,rte_lcore_id ()); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); +} +/* Display usage */ static void ioat_usage (const char *prgname)
@@ -102,7 +238,7 @@ ioat_usage(const char *prgname)" - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n" " -c --copy-type CT: type of copy: sw|rawdev\n" " -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n" , - prgname); + prgname); } static int
@@ -161,16 +297,16 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports) argvopt = argv;while ((opt =getopt_long (argc, argvopt, short_options, - lgopts, &option_index)) !=EOF ) { + lgopts, &option_index)) !=EOF ) {switch (opt) { -/* portmask */ +/* portmask */ case 'p' : ioat_enabled_port_mask =ioat_parse_portmask (optarg);if (ioat_enabled_port_mask & ~default_port_mask || - ioat_enabled_port_mask <=0 ) { + ioat_enabled_port_mask <=0 ) {printf ("Invalid portmask, %s, suggest 0x%x\n" , - optarg, default_port_mask); + optarg, default_port_mask);ioat_usage (prgname);return -1 ; }
@@ -204,7 +340,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports) }break ; -/* long options */ +/* long options */ case 0 :break ;
@@ -216,9 +352,9 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)printf ("MAC updating %s\n" , mac_updating ?"enabled" :"disabled" );if (optind >=0 ) - argv[optind-1 ] = prgname; + argv[optind -1 ] = prgname; - ret = optind-1 ; + ret = optind -1 ; optind =1 ;/* reset getopt lib */ return ret; }
@@ -253,6 +389,26 @@ check_link_status(uint32_t port_mask)return retval; } +static void +assign_rings (void ) +{ +uint32_t i; + +for (i =0 ; i < cfg.nb_ports; i++) { +char ring_name[RTE_RING_NAMESIZE ]; + +snprintf (ring_name,sizeof (ring_name),"rx_to_tx_ring_%u" , i); +/* Create ring for inter core communication */ + cfg.ports[i].rx_to_tx_ring =rte_ring_create ( + ring_name, ring_size, +rte_socket_id (),RING_F_SP_ENQ |RING_F_SC_DEQ ); + +if (cfg.ports[i].rx_to_tx_ring ==NULL ) +rte_exit (EXIT_FAILURE ,"Ring create failed: %s\n" , +rte_strerror (rte_errno)); + } +} +/* * Initializes a given port using global settings and with the RX buffers * coming from the mbuf_pool passed as a parameter.
@@ -263,7 +419,7 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)/* configuring port to use RSS for multiple RX queues */ static const struct rte_eth_conf port_conf = { .rxmode = { - .mq_mode =ETH_MQ_RX_RSS , + .mq_mode =ETH_MQ_RX_RSS , .max_rx_pkt_len =RTE_ETHER_MAX_LEN }, .rx_adv_conf = {
@@ -294,14 +450,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues) dev_info.flow_type_rss_offloads;if (dev_info.tx_offload_capa &DEV_TX_OFFLOAD_MBUF_FAST_FREE ) local_port_conf.txmode.offloads |= -DEV_TX_OFFLOAD_MBUF_FAST_FREE ; +DEV_TX_OFFLOAD_MBUF_FAST_FREE ; ret =rte_eth_dev_configure (portid, nb_queues,1 , &local_port_conf);if (ret <0 )rte_exit (EXIT_FAILURE ,"Cannot configure device:" " err=%d, port=%u\n" , ret, portid); ret =rte_eth_dev_adjust_nb_rx_tx_desc (portid, &nb_rxd, - &nb_txd); + &nb_txd);if (ret <0 ) rte_exit(EXIT_FAILURE, "Cannot adjust number of descriptors: err=%d, port=%u\n",
@@ -326,8 +482,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues) txq_conf = dev_info.default_txconf; txq_conf.offloads = local_port_conf.txmode.offloads; ret =rte_eth_tx_queue_setup (portid,0 , nb_txd, -rte_eth_dev_socket_id (portid), - &txq_conf); +rte_eth_dev_socket_id (portid), + &txq_conf);if (ret <0 ) rte_exit(EXIT_FAILURE, "rte_eth_tx_queue_setup:err=%d,port=%u\n",
@@ -335,8 +491,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)/* Initialize TX buffers */ tx_buffer[portid] =rte_zmalloc_socket ("tx_buffer" , -RTE_ETH_TX_BUFFER_SIZE (MAX_PKT_BURST ),0 , -rte_eth_dev_socket_id (portid)); +RTE_ETH_TX_BUFFER_SIZE (MAX_PKT_BURST ),0 , +rte_eth_dev_socket_id (portid));if (tx_buffer[portid] ==NULL ) rte_exit(EXIT_FAILURE, "Cannot allocate buffer for tx on port %u\n",
@@ -354,13 +510,13 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)rte_eth_promiscuous_enable (portid);printf ("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n" , - portid, - ioat_ports_eth_addr[portid].addr_bytes[0 ], - ioat_ports_eth_addr[portid].addr_bytes[1 ], - ioat_ports_eth_addr[portid].addr_bytes[2 ], - ioat_ports_eth_addr[portid].addr_bytes[3 ], - ioat_ports_eth_addr[portid].addr_bytes[4 ], - ioat_ports_eth_addr[portid].addr_bytes[5 ]); + portid, + ioat_ports_eth_addr[portid].addr_bytes[0 ], + ioat_ports_eth_addr[portid].addr_bytes[1 ], + ioat_ports_eth_addr[portid].addr_bytes[2 ], + ioat_ports_eth_addr[portid].addr_bytes[3 ], + ioat_ports_eth_addr[portid].addr_bytes[4 ], + ioat_ports_eth_addr[portid].addr_bytes[5 ]); cfg.ports[cfg.nb_ports].rxtx_port = portid; cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
@@ -428,10 +584,19 @@ main(int argc, char **argv)if (cfg.nb_lcores <1 )rte_exit (EXIT_FAILURE ,"There should be at least one slave lcore.\n" ); + +assign_rings (); + +start_forwarding_cores (); + +/* force_quit is true when we get here */ +rte_eal_mp_wait_lcore (); +for (i =0 ; i < cfg.nb_ports; i++) {printf ("Closing port %d\n" , cfg.ports[i].rxtx_port);rte_eth_dev_stop (cfg.ports[i].rxtx_port);rte_eth_dev_close (cfg.ports[i].rxtx_port); +rte_ring_free (cfg.ports[i].rx_to_tx_ring); }printf ("Bye...\n" );
Signature · 1 line
2.22.0.windows.1
PATCH v4 3/6 examples/ioat: add rawdev copy mode support
In reply to Marcin BaranAdded support for copying packets using
rawdev device. Each port's Rx queue is
assigned DMA channel for copy.
--- examples/ioat/ioatfwd.c | 236 ++++++++++++++++++++++++++++++++-------- 1 file changed, 189 insertions(+), 47 deletions(-)
examples/ioat/ioatfwd.c +189 -47
@@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) rte_pktmbuf_mtod(src, char *), src->data_len); } +static uint32_t +ioat_enqueue_packets (struct rte_mbuf **pkts, +uint32_t nb_rx,uint16_t dev_id) +{ +int ret; +uint32_t i; +struct rte_mbuf *pkts_copy[MAX_PKT_BURST ]; + +const uint64_t addr_offset =RTE_PTR_DIFF (pkts[0 ]->buf_addr, + &pkts[0 ]->rearm_data); + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE ,"Unable to allocate memory.\n" ); + +for (i =0 ; i < nb_rx; i++) { +/* Perform data copy */ + ret =rte_ioat_enqueue_copy (dev_id, + pkts[i]->buf_iova + - addr_offset, + pkts_copy[i]->buf_iova + - addr_offset, +rte_pktmbuf_data_len (pkts[i]) + + addr_offset, + (uintptr_t )pkts[i], + (uintptr_t )pkts_copy[i], +0 /* nofence */ ); + +if (ret !=1 ) +break ; + } + + ret = i; +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i); +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], + nb_rx - i); + + +return ret; +} +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ static void ioat_rx_port (struct rxtx_port_config *rx_config)
@@ -136,32 +180,40 @@ ioat_rx_port(struct rxtx_port_config *rx_config)if (nb_rx ==0 )continue ; -/* Perform packet software copy, free source packets */ -int ret; -struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; - - ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, - (void *)pkts_burst_copy, nb_rx); - -if (unlikely (ret <0 )) -rte_exit (EXIT_FAILURE , -"Unable to allocate memory.\n" ); - -for (j =0 ; j < nb_rx; j++) -pktmbuf_sw_copy (pkts_burst[j], - pkts_burst_copy[j]); - -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)pkts_burst, nb_rx); - - nb_enq =rte_ring_enqueue_burst ( - rx_config->rx_to_tx_ring, - (void *)pkts_burst_copy, nb_rx,NULL ); - -/* Free any not enqueued packets. */ -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)&pkts_burst_copy[nb_enq], - nb_rx - nb_enq); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Perform packet hardware copy */ + nb_enq =ioat_enqueue_packets (pkts_burst, + nb_rx, rx_config->ioat_ids[i]); +if (nb_enq >0 ) +rte_ioat_do_copies (rx_config->ioat_ids[i]); + }else { +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } } }
@@ -169,31 +221,64 @@ ioat_rx_port(struct rxtx_port_config *rx_config)static void ioat_tx_port (struct rxtx_port_config *tx_config) { -uint32_t i, nb_dq =0 ; +uint32_t i, j, nb_dq =0 ; +struct rte_mbuf *mbufs_src[MAX_PKT_BURST ];struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ]; -/* Deque the mbufs from rx_to_tx_ring. */ - nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, - (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Deque the mbufs from IOAT device. */ +for (i =0 ; i < tx_config->nb_queues; i++) { + nb_dq =rte_ioat_completed_copies ( + tx_config->ioat_ids[i],MAX_PKT_BURST , + (void *)mbufs_src, (void *)mbufs_dst); -if (nb_dq ==0 ) -return ; +if (nb_dq ==0 ) +break ; -/* Update macs if enabled */ -if (mac_updating) { -for (i =0 ; i < nb_dq; i++) -update_mac_addrs (mbufs_dst[i], - tx_config->rxtx_port); - } +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)mbufs_src, nb_dq); -const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, -0 , (void *)mbufs_dst, nb_dq); +/* Update macs if enabled */ +if (mac_updating) { +for (j =0 ; j < nb_dq; j++) +update_mac_addrs (mbufs_dst[j], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst ( + tx_config->rxtx_port,0 , + (void *)mbufs_dst, nb_dq); -/* Free any unsent packets. */ -if (unlikely (nb_tx < nb_dq)) -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)&mbufs_dst[nb_tx], - nb_dq - nb_tx); +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } + }else { +/* Deque the mbufs from rx_to_tx_ring. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); + +if (nb_dq ==0 ) +return ; + +/* Update macs if enabled */ +if (mac_updating) { +for (i =0 ; i < nb_dq; i++) +update_mac_addrs (mbufs_dst[i], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, +0 , (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } }/* Main rx and tx loop if only one slave lcore available */
@@ -389,6 +474,52 @@ check_link_status(uint32_t port_mask)return retval; } +static void +configure_rawdev_queue (uint32_t dev_id) +{ +struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size }; +struct rte_rawdev_info info = { .dev_private = &dev_config }; + +if (rte_rawdev_configure (dev_id, &info) !=0 ) { +rte_exit (EXIT_FAILURE , +"Error with rte_rawdev_configure()\n" ); + } +if (rte_rawdev_start (dev_id) !=0 ) { +rte_exit (EXIT_FAILURE , +"Error with rte_rawdev_start()\n" ); + } +} + +static void +assign_rawdevs (void ) +{ +uint16_t nb_rawdev =0 , rdev_id =0 ; +uint32_t i, j; + +for (i =0 ; i < cfg.nb_ports; i++) { +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { +struct rte_rawdev_info rdev_info = {0 }; + +do { +if (rdev_id ==rte_rawdev_count ()) + goto end; +rte_rawdev_info_get (rdev_id++, &rdev_info); + }while (strcmp (rdev_info.driver_name, +IOAT_PMD_RAWDEV_NAME_STR ) !=0 ); + + cfg.ports[i].ioat_ids[j] = rdev_id -1 ; +configure_rawdev_queue (cfg.ports[i].ioat_ids[j]); + ++nb_rawdev; + } + } +end: +if (nb_rawdev < cfg.nb_ports * cfg.ports[0 ].nb_queues) +rte_exit (EXIT_FAILURE , +"Not enough IOAT rawdevs (%u) for all queues (%u).\n" , + nb_rawdev, cfg.nb_ports * cfg.ports[0 ].nb_queues); +RTE_LOG (INFO ,IOAT ,"Number of used rawdevs: %u.\n" , nb_rawdev); +} +static void assign_rings (void ) {
@@ -585,18 +716,29 @@ main(int argc, char **argv)rte_exit (EXIT_FAILURE ,"There should be at least one slave lcore.\n" ); -assign_rings (); +if (copy_mode ==COPY_MODE_IOAT_NUM ) +assign_rawdevs (); +else /* copy_mode == COPY_MODE_SW_NUM */ +assign_rings ();start_forwarding_cores ();/* force_quit is true when we get here */ rte_eal_mp_wait_lcore (); +uint32_t j;for (i =0 ; i < cfg.nb_ports; i++) {printf ("Closing port %d\n" , cfg.ports[i].rxtx_port);rte_eth_dev_stop (cfg.ports[i].rxtx_port);rte_eth_dev_close (cfg.ports[i].rxtx_port); -rte_ring_free (cfg.ports[i].rx_to_tx_ring); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { +printf ("Stopping rawdev %d\n" , + cfg.ports[i].ioat_ids[j]); +rte_rawdev_stop (cfg.ports[i].ioat_ids[j]); + } + }else /* copy_mode == COPY_MODE_SW_NUM */ +rte_ring_free (cfg.ports[i].rx_to_tx_ring); }printf ("Bye...\n" );
Signature · 1 line
2.22.0.windows.1
PATCH v4 4/6 examples/ioat: add two threads configuration
In reply to Marcin BaranFrom: Pawel Modrak <[email protected]>
Added possibility to use two lcores: first for
packet receiving and copying, second for packets
sending.
--- examples/ioat/ioatfwd.c | 92 ++++++++++++++++++++++++++++++----------- 1 file changed, 67 insertions(+), 25 deletions(-)
examples/ioat/ioatfwd.c +67 -25
@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], nb_rx - i); -return ret; }
@@ -226,8 +225,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ];if (copy_mode ==COPY_MODE_IOAT_NUM ) { -/* Deque the mbufs from IOAT device. */ for (i =0 ; i < tx_config->nb_queues; i++) { +/* Deque the mbufs from IOAT device. */ nb_dq =rte_ioat_completed_copies ( tx_config->ioat_ids[i],MAX_PKT_BURST , (void *)mbufs_src, (void *)mbufs_dst);
@@ -256,31 +255,64 @@ ioat_tx_port(struct rxtx_port_config *tx_config) nb_dq - nb_tx); } }else { -/* Deque the mbufs from rx_to_tx_ring. */ - nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, - (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); - -if (nb_dq ==0 ) -return ; - -/* Update macs if enabled */ -if (mac_updating) { -for (i =0 ; i < nb_dq; i++) -update_mac_addrs (mbufs_dst[i], - tx_config->rxtx_port); - } +for (i =0 ; i < tx_config->nb_queues; i++) { +/* Deque the mbufs from IOAT device. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); -const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, -0 , (void *)mbufs_dst, nb_dq); +if (nb_dq ==0 ) +return ; -/* Free any unsent packets. */ -if (unlikely (nb_tx < nb_dq)) -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)&mbufs_dst[nb_tx], - nb_dq - nb_tx); +/* Update macs if enabled */ +if (mac_updating) { +for (j =0 ; j < nb_dq; j++) +update_mac_addrs (mbufs_dst[j], + tx_config->rxtx_port); + } + +const uint16_t nb_tx = +rte_eth_tx_burst (tx_config->rxtx_port,0 , + (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } } } +/* Main rx processing loop for IOAT rawdev. */ +static void +rx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx loop for copy on lcore %u\n" , +rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) +ioat_rx_port (&cfg.ports[i]); +} + +/* Main tx processing loop for hardware copy. */ +static void +tx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main tx loop for copy on lcore %u\n" , +rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) +ioat_tx_port (&cfg.ports[i]); +} +/* Main rx and tx loop if only one slave lcore available */ static void rxtx_main_loop (void )
@@ -305,9 +337,19 @@ static void start_forwarding_cores(void)RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , __func__,rte_lcore_id ()); - lcore_id =rte_get_next_lcore (lcore_id, true, true); -rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, -NULL , lcore_id); +if (cfg.nb_lcores ==1 ) { + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); + }else if (cfg.nb_lcores >1 ) { + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rx_main_loop, +NULL , lcore_id); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)tx_main_loop,NULL , + lcore_id); + } }/* Display usage */
Signature · 1 line
2.22.0.windows.1
PATCH v4 5/6 examples/ioat: add stats printing for each port
In reply to Marcin BaranAdded printing stats of ports each second. The stats printing is done using master core. The information provided informs about packets received, dropped and send as well as statistics of rawdev devices.
--- examples/ioat/ioatfwd.c | 245 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 241 insertions(+), 4 deletions(-)
examples/ioat/ioatfwd.c +241 -4
@@ -48,10 +48,27 @@ struct rxtx_transmission_config {uint16_t nb_lcores; }; +/* per-port statistics struct */ +struct ioat_port_statistics { +uint64_t rx[RTE_MAX_ETHPORTS ]; +uint64_t tx[RTE_MAX_ETHPORTS ]; +uint64_t tx_dropped[RTE_MAX_ETHPORTS ]; +uint64_t copy_dropped[RTE_MAX_ETHPORTS ]; +}; +struct ioat_port_statistics port_statistics; + +struct total_statistics { +uint64_t total_packets_dropped; +uint64_t total_packets_tx; +uint64_t total_packets_rx; +uint64_t total_successful_enqueues; +uint64_t total_failed_enqueues; +}; + typedef enum copy_mode_t { #define COPY_MODE_SW "sw" COPY_MODE_SW_NUM, -#define COPY_MODE_IOAT "rawdev" +#define COPY_MODE_IOAT "hw" COPY_MODE_IOAT_NUM, COPY_MODE_INVALID_NUM, COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
@@ -89,6 +106,204 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ];struct rte_mempool *ioat_pktmbuf_pool; +/* Print out statistics for one port. */ +static void +print_port_stats (uint16_t port_id) +{ +printf ("\nStatistics for port %u ------------------------------" +"\nPackets sent: %34" PRIu64 +"\nPackets received: %30" PRIu64 +"\nPackets dropped on tx: %25" PRIu64 +"\nPackets dropped on copy: %23" PRIu64, + port_id, + port_statistics.tx[port_id], + port_statistics.rx[port_id], + port_statistics.tx_dropped[port_id], + port_statistics.copy_dropped[port_id]); +} + +/* Print out statistics for one IOAT rawdev device. */ +static void +print_rawdev_stats (uint32_t dev_id,uint64_t *xstats, +unsigned int *ids_xstats,uint16_t nb_xstats, +struct rte_rawdev_xstats_name *names_xstats) +{ +uint16_t i; + +printf ("\nIOAT channel %u" , dev_id); +for (i =0 ; i < nb_xstats; i++) +printf ("\n\t %s: %*" PRIu64, + names_xstats[ids_xstats[i]].name, + (int )(37 -strlen (names_xstats[ids_xstats[i]].name)), + xstats[i]); +} + +static void +print_total_stats (struct total_statistics *ts) +{ +printf ("\nAggregate statistics ===============================" +"\nTotal packets Tx: %24" PRIu64" [pps]" +"\nTotal packets Rx: %24" PRIu64" [pps]" +"\nTotal packets dropped: %19" PRIu64" [pps]" , + ts->total_packets_tx, + ts->total_packets_rx, + ts->total_packets_dropped); + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +printf ("\nTotal IOAT successful enqueues: %8" PRIu64" [enq/s]" +"\nTotal IOAT failed enqueues: %12" PRIu64" [enq/s]" , + ts->total_successful_enqueues, + ts->total_failed_enqueues); + } + +printf ("\n====================================================\n" ); +} + +/* Print out statistics on packets dropped. */ +static void +print_stats (char *prgname) +{ +struct total_statistics ts, delta_ts; +uint32_t i, port_id, dev_id; +struct rte_rawdev_xstats_name *names_xstats; +uint64_t *xstats; +unsigned int *ids_xstats, nb_xstats; +char status_string[120 ];/* to print at the top of the output */ +int status_strlen; + + +const char clr[] = {27 ,'[' ,'2' ,'J' ,'\0' }; +const char topLeft[] = {27 ,'[' ,'1' ,';' ,'1' ,'H' ,'\0' }; + + status_strlen =snprintf (status_string,sizeof (status_string), +"%s, " , prgname); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Worker Threads = %d, " , +rte_lcore_count () >2 ?2 :1 ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Copy Mode = %s,\n" , copy_mode ==COPY_MODE_SW_NUM ? +COPY_MODE_SW :COPY_MODE_IOAT ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Updating MAC = %s, " , mac_updating ? +"enabled" :"disabled" ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Rx Queues = %d, " , nb_queues); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Ring Size = %d\n" , ring_size); + +/* Allocate memory for xstats names and values */ + nb_xstats =rte_rawdev_xstats_names_get ( + cfg.ports[0 ].ioat_ids[0 ],NULL ,0 ); + + names_xstats =malloc (sizeof (*names_xstats) * nb_xstats); +if (names_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat names memory\n" ); + } +rte_rawdev_xstats_names_get (cfg.ports[0 ].ioat_ids[0 ], + names_xstats, nb_xstats); + + ids_xstats =malloc (sizeof (*ids_xstats) *2 ); +if (ids_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat ids_xstats memory\n" ); + } + + xstats =malloc (sizeof (*xstats) *2 ); +if (xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat memory\n" ); + } + +/* Get failed/successful enqueues stats index */ + ids_xstats[0 ] = ids_xstats[1 ] = nb_xstats; +for (i =0 ; i < nb_xstats; i++) { +if (!strcmp (names_xstats[i].name,"failed_enqueues" )) + ids_xstats[0 ] = i; +else if (!strcmp (names_xstats[i].name,"successful_enqueues" )) + ids_xstats[1 ] = i; +if (ids_xstats[0 ] < nb_xstats && ids_xstats[1 ] < nb_xstats) +break ; + } +if (ids_xstats[0 ] == nb_xstats || ids_xstats[1 ] == nb_xstats) { +rte_exit (EXIT_FAILURE , +"Error getting failed/successful enqueues stats index\n" ); + } + +memset (&ts,0 ,sizeof (struct total_statistics )); + +while (!force_quit) { +/* Sleep for 1 second each round - init sleep allows reading +* messages from app startup. +*/ +sleep (1 ); + +/* Clear screen and move to top left */ +printf ("%s%s" , clr, topLeft); + +memset (&delta_ts,0 ,sizeof (struct total_statistics )); + +printf ("%s" , status_string); + +for (i =0 ; i < cfg.nb_ports; i++) { + port_id = cfg.ports[i].rxtx_port; +print_port_stats (port_id); + + delta_ts.total_packets_dropped += + port_statistics.tx_dropped[port_id] + + port_statistics.copy_dropped[port_id]; + delta_ts.total_packets_tx += + port_statistics.tx[port_id]; + delta_ts.total_packets_rx += + port_statistics.rx[port_id]; + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +uint32_t j; + +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { + dev_id = cfg.ports[i].ioat_ids[j]; +rte_rawdev_xstats_get (dev_id, + ids_xstats, xstats,2 ); + +print_rawdev_stats (dev_id, xstats, + ids_xstats,2 , names_xstats); + + delta_ts.total_failed_enqueues += + xstats[ids_xstats[0 ]]; + delta_ts.total_successful_enqueues += + xstats[ids_xstats[1 ]]; + } + } + } + + delta_ts.total_packets_tx -= ts.total_packets_tx; + delta_ts.total_packets_rx -= ts.total_packets_rx; + delta_ts.total_packets_dropped -= ts.total_packets_dropped; + delta_ts.total_failed_enqueues -= ts.total_failed_enqueues; + delta_ts.total_successful_enqueues -= + ts.total_successful_enqueues; + +printf ("\n" ); +print_total_stats (&delta_ts); + + ts.total_packets_tx += delta_ts.total_packets_tx; + ts.total_packets_rx += delta_ts.total_packets_rx; + ts.total_packets_dropped += delta_ts.total_packets_dropped; + ts.total_failed_enqueues += delta_ts.total_failed_enqueues; + ts.total_successful_enqueues += + delta_ts.total_successful_enqueues; + } + +free (names_xstats); +free (xstats); +free (ids_xstats); +} +static void update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) {
@@ -179,6 +394,8 @@ ioat_rx_port(struct rxtx_port_config *rx_config)if (nb_rx ==0 )continue ; + port_statistics.rx[rx_config->rxtx_port] += nb_rx; +if (copy_mode ==COPY_MODE_IOAT_NUM ) {/* Perform packet hardware copy */ nb_enq = ioat_enqueue_packets(pkts_burst,
@@ -213,6 +430,9 @@ ioat_rx_port(struct rxtx_port_config *rx_config) (void *)&pkts_burst_copy[nb_enq], nb_rx - nb_enq); } + + port_statistics.copy_dropped[rx_config->rxtx_port] += + (nb_rx - nb_enq); } }
@@ -248,6 +468,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config) tx_config->rxtx_port,0 , (void *)mbufs_dst, nb_dq); + port_statistics.tx[tx_config->rxtx_port] += nb_tx; +/* Free any unsent packets. */ if (unlikely (nb_tx < nb_dq)) rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -274,6 +496,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)rte_eth_tx_burst (tx_config->rxtx_port,0 , (void *)mbufs_dst, nb_dq); + port_statistics.tx[tx_config->rxtx_port] += nb_tx; +/* Free any unsent packets. */ if (unlikely (nb_tx < nb_dq)) rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -363,7 +587,7 @@ ioat_usage(const char *prgname)" When enabled:\n" " - The source MAC address is replaced by the TX port MAC address\n" " - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n" -" -c --copy-type CT: type of copy: sw|rawdev\n" +" -c --copy-type CT: type of copy: sw|hw\n" " -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n" , prgname); }
@@ -400,7 +624,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)static const char short_options[] ="p:" /* portmask */ "q:" /* number of RX queues per port */ -"c:" /* copy type (sw|rawdev) */ +"c:" /* copy type (sw|hw) */ "s:" /* ring size */ ;
@@ -452,7 +676,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)case 'c' : copy_mode =ioat_parse_copy_mode (optarg);if (copy_mode ==COPY_MODE_INVALID_NUM ) { -printf ("Invalid copy type. Use: sw, rawdev\n" ); +printf ("Invalid copy type. Use: sw, hw\n" );ioat_usage (prgname);return -1 ; }
@@ -673,6 +897,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)rte_eth_tx_buffer_init (tx_buffer[portid],MAX_PKT_BURST ); + ret =rte_eth_tx_buffer_set_err_callback (tx_buffer[portid], + rte_eth_tx_buffer_count_callback, + &port_statistics.tx_dropped[portid]); +if (ret <0 ) +rte_exit (EXIT_FAILURE , +"Cannot set error callback for tx buffer on port %u\n" , + portid); +/* Start device */ ret =rte_eth_dev_start (portid);if (ret <0 )
@@ -749,6 +981,9 @@ main(int argc, char **argv)RTE_ETH_FOREACH_DEV (portid)port_init (portid, ioat_pktmbuf_pool, nb_queues); +/* Initialize port xstats */ +memset (&port_statistics,0 ,sizeof (port_statistics)); +while (!check_link_status (ioat_enabled_port_mask) && !force_quit)sleep (1 );
@@ -764,6 +999,8 @@ main(int argc, char **argv)assign_rings ();start_forwarding_cores (); +/* master core prints stats while other cores forward */ +print_stats (argv[0 ]);/* force_quit is true when we get here */ rte_eal_mp_wait_lcore ();
Signature · 1 line
2.22.0.windows.1
PATCH v4 6/6 doc/guides/: provide IOAT sample app guide
In reply to Marcin BaranAdded guide for IOAT sample app usage and
code description.
--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++++++++++ 3 files changed, 769 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst
doc/guides/sample_app_ug/index.rst +1 -0
@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtual
doc/guides/sample_app_ug/intro.rst +4 -0
@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.
doc/guides/sample_app_ug/ioat.rst new file +764 -0
765 lines not on this page · raw message
Signature · 1 line
2.22.0.windows.1
v5 cover examples/ioat: sample app on ioat driver usage
In reply to Marcin BaranA new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
v5:
- change dependency name from 'pmd_ioat' to 'rawdev_ioat' fixing build errors (branch diverged from master)
v4:
- fix meson build support check
v3:
- add check for meson build whether IOAT is supported
v2:
- change printing total stats to be deltas
- update documentation
- fix 1 thread/sw copy/multiple Rx queues packet dropping
- divide patch into several presenting functional change
Marcin Baran (4):
examples/ioat: add software copy support examples/ioat: add rawdev copy mode support examples/ioat: add stats printing for each port doc/guides/: provide IOAT sample app guide
Pawel Modrak (2):
examples/ioat: create sample app on ioat driver usage examples/ioat: add two threads configuration
doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1025 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 15 + examples/meson.build | 1 + 8 files changed, 1867 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
Signature · 1 line
2.22.0.windows.1
PATCH v5 1/6 examples/ioat: create sample app on ioat driver usage
From: Pawel Modrak <[email protected]>
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The change covers ports initialization, closing connection and argument parsing.
--- examples/Makefile | 3 + examples/ioat/Makefile | 54 +++++ examples/ioat/ioatfwd.c | 439 ++++++++++++++++++++++++++++++++++++++ examples/ioat/meson.build | 15 ++ examples/meson.build | 1 + 5 files changed, 512 insertions(+) create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
examples/Makefile +3 -0
@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify DIRS-y += flow_filtering DIRS-y += helloworld +ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y) +DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat +endif DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline ifeq ($(CONFIG_RTE_LIBRTE_LPM),y) DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
examples/ioat/Makefile new file +54 -0
54 lines not on this page · raw message
examples/ioat/ioatfwd.c new file +439 -0
439 lines not on this page · raw message
examples/ioat/meson.build new file +15 -0
@@ -0,0 +1,15 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +build = dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV') + +deps += ['rawdev_ioat'] + +sources = files( + 'ioatfwd.c' +)
examples/meson.build +1 -0
@@ -16,6 +16,7 @@ all_examples = [ 'eventdev_pipeline', 'exception_path', 'fips_validation', 'flow_classify', 'flow_filtering', 'helloworld', + 'ioat', 'ip_fragmentation', 'ip_pipeline', 'ip_reassembly', 'ipsec-secgw', 'ipv4_multicast', 'kni',
Signature · 1 line
2.22.0.windows.1
Quoted 11 lines · Marcin Baran
From: Pawel Modrak <[email protected]>
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The change covers ports initialization, closing connection and argument parsing.
Signed-off-by Pawel Modrak <[email protected]>Signed-off-by Marcin Baran <[email protected]>---
PATCH v5 2/6 examples/ioat: add software copy support
In reply to Marcin BaranAdded support for copying packets
using software copy mode and MAC address
changing. The copies are processed using
one lcore.
--- examples/ioat/ioatfwd.c | 211 +++++++++++++++++++++++++++++++++++----- 1 file changed, 188 insertions(+), 23 deletions(-)
examples/ioat/ioatfwd.c +188 -23
@@ -13,7 +13,7 @@#include <rte_rawdev.h> #include <rte_ioat_rawdev.h> -/* size of ring used for software copying between rx and tx. */ +/* size of ring used for software copying between rx and tx. */ #define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1#define MAX_PKT_BURST 32#define MEMPOOL_CACHE_SIZE 512
@@ -89,6 +89,142 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ];struct rte_mempool *ioat_pktmbuf_pool; +static void +update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) +{ +struct rte_ether_hdr *eth; +void *tmp; + + eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *); + +/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but +* it's acceptable cause it gets overwritten by rte_ether_addr_copy +*/ + tmp = ð->d_addr.addr_bytes[0 ]; + *((uint64_t *)tmp) =0x000000000002 + ((uint64_t )dest_portid <<40 ); + +/* src addr */ +rte_ether_addr_copy (&ioat_ports_eth_addr[dest_portid], ð->s_addr); +} + +static inline void +pktmbuf_sw_copy (struct rte_mbuf *src,struct rte_mbuf *dst) +{ +/* Copy packet metadata */ +rte_memcpy (&dst->rearm_data, + &src->rearm_data, + offsetof(struct rte_mbuf , cacheline1) + - offsetof(struct rte_mbuf , rearm_data)); + +/* Copy packet data */ + rte_memcpy(rte_pktmbuf_mtod(dst, char *), + rte_pktmbuf_mtod(src, char *), src->data_len); +} + +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ +static void +ioat_rx_port (struct rxtx_port_config *rx_config) +{ +uint32_t nb_rx, nb_enq, i, j; +struct rte_mbuf *pkts_burst[MAX_PKT_BURST ]; + +for (i =0 ; i < rx_config->nb_queues; i++) { + + nb_rx =rte_eth_rx_burst (rx_config->rxtx_port, i, + pkts_burst,MAX_PKT_BURST ); + +if (nb_rx ==0 ) +continue ; + +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } +} + +/* Transmit packets from IOAT rawdev/rte_ring for one port. */ +static void +ioat_tx_port (struct rxtx_port_config *tx_config) +{ +uint32_t i, nb_dq =0 ; +struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ]; + +/* Deque the mbufs from rx_to_tx_ring. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); + +if (nb_dq ==0 ) +return ; + +/* Update macs if enabled */ +if (mac_updating) { +for (i =0 ; i < nb_dq; i++) +update_mac_addrs (mbufs_dst[i], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, +0 , (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); +} + +/* Main rx and tx loop if only one slave lcore available */ +static void +rxtx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx and tx loop for copy on" +" lcore %u\n" ,rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) { +ioat_rx_port (&cfg.ports[i]); +ioat_tx_port (&cfg.ports[i]); + } +} + +static void start_forwarding_cores (void ) +{ +uint32_t lcore_id =rte_lcore_id (); + +RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , + __func__,rte_lcore_id ()); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); +} +/* Display usage */ static void ioat_usage (const char *prgname)
@@ -102,7 +238,7 @@ ioat_usage(const char *prgname)" - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n" " -c --copy-type CT: type of copy: sw|rawdev\n" " -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n" , - prgname); + prgname); } static int
@@ -161,16 +297,16 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports) argvopt = argv;while ((opt =getopt_long (argc, argvopt, short_options, - lgopts, &option_index)) !=EOF ) { + lgopts, &option_index)) !=EOF ) {switch (opt) { -/* portmask */ +/* portmask */ case 'p' : ioat_enabled_port_mask =ioat_parse_portmask (optarg);if (ioat_enabled_port_mask & ~default_port_mask || - ioat_enabled_port_mask <=0 ) { + ioat_enabled_port_mask <=0 ) {printf ("Invalid portmask, %s, suggest 0x%x\n" , - optarg, default_port_mask); + optarg, default_port_mask);ioat_usage (prgname);return -1 ; }
@@ -204,7 +340,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports) }break ; -/* long options */ +/* long options */ case 0 :break ;
@@ -216,9 +352,9 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)printf ("MAC updating %s\n" , mac_updating ?"enabled" :"disabled" );if (optind >=0 ) - argv[optind-1 ] = prgname; + argv[optind -1 ] = prgname; - ret = optind-1 ; + ret = optind -1 ; optind =1 ;/* reset getopt lib */ return ret; }
@@ -253,6 +389,26 @@ check_link_status(uint32_t port_mask)return retval; } +static void +assign_rings (void ) +{ +uint32_t i; + +for (i =0 ; i < cfg.nb_ports; i++) { +char ring_name[RTE_RING_NAMESIZE ]; + +snprintf (ring_name,sizeof (ring_name),"rx_to_tx_ring_%u" , i); +/* Create ring for inter core communication */ + cfg.ports[i].rx_to_tx_ring =rte_ring_create ( + ring_name, ring_size, +rte_socket_id (),RING_F_SP_ENQ |RING_F_SC_DEQ ); + +if (cfg.ports[i].rx_to_tx_ring ==NULL ) +rte_exit (EXIT_FAILURE ,"Ring create failed: %s\n" , +rte_strerror (rte_errno)); + } +} +/* * Initializes a given port using global settings and with the RX buffers * coming from the mbuf_pool passed as a parameter.
@@ -263,7 +419,7 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)/* configuring port to use RSS for multiple RX queues */ static const struct rte_eth_conf port_conf = { .rxmode = { - .mq_mode =ETH_MQ_RX_RSS , + .mq_mode =ETH_MQ_RX_RSS , .max_rx_pkt_len =RTE_ETHER_MAX_LEN }, .rx_adv_conf = {
@@ -294,14 +450,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues) dev_info.flow_type_rss_offloads;if (dev_info.tx_offload_capa &DEV_TX_OFFLOAD_MBUF_FAST_FREE ) local_port_conf.txmode.offloads |= -DEV_TX_OFFLOAD_MBUF_FAST_FREE ; +DEV_TX_OFFLOAD_MBUF_FAST_FREE ; ret =rte_eth_dev_configure (portid, nb_queues,1 , &local_port_conf);if (ret <0 )rte_exit (EXIT_FAILURE ,"Cannot configure device:" " err=%d, port=%u\n" , ret, portid); ret =rte_eth_dev_adjust_nb_rx_tx_desc (portid, &nb_rxd, - &nb_txd); + &nb_txd);if (ret <0 ) rte_exit(EXIT_FAILURE, "Cannot adjust number of descriptors: err=%d, port=%u\n",
@@ -326,8 +482,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues) txq_conf = dev_info.default_txconf; txq_conf.offloads = local_port_conf.txmode.offloads; ret =rte_eth_tx_queue_setup (portid,0 , nb_txd, -rte_eth_dev_socket_id (portid), - &txq_conf); +rte_eth_dev_socket_id (portid), + &txq_conf);if (ret <0 ) rte_exit(EXIT_FAILURE, "rte_eth_tx_queue_setup:err=%d,port=%u\n",
@@ -335,8 +491,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)/* Initialize TX buffers */ tx_buffer[portid] =rte_zmalloc_socket ("tx_buffer" , -RTE_ETH_TX_BUFFER_SIZE (MAX_PKT_BURST ),0 , -rte_eth_dev_socket_id (portid)); +RTE_ETH_TX_BUFFER_SIZE (MAX_PKT_BURST ),0 , +rte_eth_dev_socket_id (portid));if (tx_buffer[portid] ==NULL ) rte_exit(EXIT_FAILURE, "Cannot allocate buffer for tx on port %u\n",
@@ -354,13 +510,13 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)rte_eth_promiscuous_enable (portid);printf ("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n" , - portid, - ioat_ports_eth_addr[portid].addr_bytes[0 ], - ioat_ports_eth_addr[portid].addr_bytes[1 ], - ioat_ports_eth_addr[portid].addr_bytes[2 ], - ioat_ports_eth_addr[portid].addr_bytes[3 ], - ioat_ports_eth_addr[portid].addr_bytes[4 ], - ioat_ports_eth_addr[portid].addr_bytes[5 ]); + portid, + ioat_ports_eth_addr[portid].addr_bytes[0 ], + ioat_ports_eth_addr[portid].addr_bytes[1 ], + ioat_ports_eth_addr[portid].addr_bytes[2 ], + ioat_ports_eth_addr[portid].addr_bytes[3 ], + ioat_ports_eth_addr[portid].addr_bytes[4 ], + ioat_ports_eth_addr[portid].addr_bytes[5 ]); cfg.ports[cfg.nb_ports].rxtx_port = portid; cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
@@ -428,10 +584,19 @@ main(int argc, char **argv)if (cfg.nb_lcores <1 )rte_exit (EXIT_FAILURE ,"There should be at least one slave lcore.\n" ); + +assign_rings (); + +start_forwarding_cores (); + +/* force_quit is true when we get here */ +rte_eal_mp_wait_lcore (); +for (i =0 ; i < cfg.nb_ports; i++) {printf ("Closing port %d\n" , cfg.ports[i].rxtx_port);rte_eth_dev_stop (cfg.ports[i].rxtx_port);rte_eth_dev_close (cfg.ports[i].rxtx_port); +rte_ring_free (cfg.ports[i].rx_to_tx_ring); }printf ("Bye...\n" );
Signature · 1 line
2.22.0.windows.1
Re: [dpdk-dev] [PATCH v5 2/6] examples/ioat: add software copy support
Quoted 177 lines · Marcin Baran
Added support for copying packets
using software copy mode and MAC address
changing. The copies are processed using
one lcore.Signed-off-by Marcin Baran <[email protected]>Signed-off-by Pawel Modrak <[email protected]>--- examples/ioat/ioatfwd.c | 211 +++++++++++++++++++++++++++++++++++----- 1 file changed, 188 insertions(+), 23 deletions(-)examples/ioat/ioatfwd.c +138
-2@@ -13,7 +13,7 @@#include <rte_rawdev.h> #include <rte_ioat_rawdev.h> -/* size of ring used for software copying between rx and tx. */ +/* size of ring used for software copying between rx and tx. */ #define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1#define MAX_PKT_BURST 32#define MEMPOOL_CACHE_SIZE 512@@ -89,6 +89,142 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ];struct rte_mempool *ioat_pktmbuf_pool; +static void +update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) +{ +struct rte_ether_hdr *eth; +void *tmp; + + eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *); + +/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but +* it's acceptable cause it gets overwritten by rte_ether_addr_copy +*/ + tmp = ð->d_addr.addr_bytes[0 ]; + *((uint64_t *)tmp) =0x000000000002 + ((uint64_t )dest_portid <<40 ); + +/* src addr */ +rte_ether_addr_copy (&ioat_ports_eth_addr[dest_portid], ð->s_addr); +} + +static inline void +pktmbuf_sw_copy (struct rte_mbuf *src,struct rte_mbuf *dst) +{ +/* Copy packet metadata */ +rte_memcpy (&dst->rearm_data, + &src->rearm_data, + offsetof(struct rte_mbuf , cacheline1) + - offsetof(struct rte_mbuf , rearm_data)); + +/* Copy packet data */ + rte_memcpy(rte_pktmbuf_mtod(dst, char *), + rte_pktmbuf_mtod(src, char *), src->data_len); +} + +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ +static void +ioat_rx_port (struct rxtx_port_config *rx_config) +{ +uint32_t nb_rx, nb_enq, i, j; +struct rte_mbuf *pkts_burst[MAX_PKT_BURST ]; + +for (i =0 ; i < rx_config->nb_queues; i++) { + + nb_rx =rte_eth_rx_burst (rx_config->rxtx_port, i, + pkts_burst,MAX_PKT_BURST ); + +if (nb_rx ==0 ) +continue ; + +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } +} + +/* Transmit packets from IOAT rawdev/rte_ring for one port. */ +static void +ioat_tx_port (struct rxtx_port_config *tx_config) +{ +uint32_t i, nb_dq =0 ; +struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ]; + +/* Deque the mbufs from rx_to_tx_ring. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); + +if (nb_dq ==0 ) +return ; + +/* Update macs if enabled */ +if (mac_updating) { +for (i =0 ; i < nb_dq; i++) +update_mac_addrs (mbufs_dst[i], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, +0 , (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); +} + +/* Main rx and tx loop if only one slave lcore available */ +static void +rxtx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx and tx loop for copy on" +" lcore %u\n" ,rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) { +ioat_rx_port (&cfg.ports[i]); +ioat_tx_port (&cfg.ports[i]); + } +} + +static void start_forwarding_cores (void ) +{ +uint32_t lcore_id =rte_lcore_id (); + +RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , + __func__,rte_lcore_id ()); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); +} +/* Display usage */ static void ioat_usage (const char *prgname)@@ -102,7 +238,7 @@ ioat_usage(const char *prgname)" - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n" " -c --copy-type CT: type of copy: sw|rawdev\n" " -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n" , - prgname); + prgname); } static int
examples/ioat/ioatfwd.c ioat_parse_args(int argc, char **argv, unsigned int nb_ports) · line 300
Quoted 5 lines
examples/ioat/ioatfwd.c +1
-1@@ -161,16 +297,16 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports) argvopt = argv;while ((opt =getopt_long (argc, argvopt, short_options, - lgopts, &option_index)) !=EOF ) { + lgopts, &option_index)) !=EOF ) {
These and the other whitespace changes in this patch should be fixed in patch 1 rather than having them as modifications here.
Without these whitespace changes merged into original patch:
Baran, MarcinX
Acked-by Bruce RichardsonRe: [dpdk-dev] [PATCH v5 2/6] examples/ioat: add software copy support
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Friday, September 27, 2019 12:01 PM
To: Baran, MarcinX <[email protected]>
Cc: [email protected]; Modrak, PawelX <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v5 2/6] examples/ioat: add software copy support
Quoted 169 lines · Marcin Baran
Added support for copying packets
using software copy mode and MAC address changing. The copies are processed using one lcore.Signed-off-by Marcin Baran <[email protected]>Signed-off-by Pawel Modrak <[email protected]>--- examples/ioat/ioatfwd.c | 211 +++++++++++++++++++++++++++++++++++----- 1 file changed, 188 insertions(+), 23 deletions(-)examples/ioat/ioatfwd.c index renamed from examples/ioat/ioatfwd.c +0
-0
977ea6a61..3a092c6cf100644examples/ioat/ioatfwd.c +1
-1@@ -13,7 +13,7 @@#include <rte_rawdev.h> #include <rte_ioat_rawdev.h> -/* size of ring used for software copying between rx and tx. */ +/* size of ring used for software copying between rx and tx. */ #define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1 #define MAX_PKT_BURST 32#define MEMPOOL_CACHE_SIZE 512 @@ -89,6 +89,142 @@ static structrte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS ];static struct
rte_eth_dev_tx_buffer*tx_buffer[RTE_MAX_ETHPORTS];
structrte_mempool*ioat_pktmbuf_pool;examples/ioat/ioatfwd.c +130
-1+static void +update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) { +struct rte_ether_hdr *eth; +void *tmp; + + eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *); + +/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but +* it's acceptable cause it gets overwritten by rte_ether_addr_copy +*/ + tmp = ð->d_addr.addr_bytes[0 ]; + *((uint64_t *)tmp) =0x000000000002 + ((uint64_t )dest_portid <<40 ); + +/* src addr */ +rte_ether_addr_copy (&ioat_ports_eth_addr[dest_portid], +ð->s_addr); } + +static inline void +pktmbuf_sw_copy (struct rte_mbuf *src,struct rte_mbuf *dst) { +/* Copy packet metadata */ +rte_memcpy (&dst->rearm_data, + &src->rearm_data, + offsetof(struct rte_mbuf , cacheline1) + - offsetof(struct rte_mbuf , rearm_data)); + +/* Copy packet data */ + rte_memcpy(rte_pktmbuf_mtod(dst, char *), + rte_pktmbuf_mtod(src, char *), src->data_len); } + +/* Receive packets on one port and enqueue to IOAT rawdev or +rte_ring. */ static void ioat_rx_port (struct rxtx_port_config +*rx_config) { +uint32_t nb_rx, nb_enq, i, j; +struct rte_mbuf *pkts_burst[MAX_PKT_BURST ]; + +for (i =0 ; i < rx_config->nb_queues; i++) { + + nb_rx =rte_eth_rx_burst (rx_config->rxtx_port, i, + pkts_burst,MAX_PKT_BURST ); + +if (nb_rx ==0 ) +continue ; + +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } +} + +/* Transmit packets from IOAT rawdev/rte_ring for one port. */ static +void ioat_tx_port (struct rxtx_port_config *tx_config) { +uint32_t i, nb_dq =0 ; +struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ]; + +/* Deque the mbufs from rx_to_tx_ring. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); + +if (nb_dq ==0 ) +return ; + +/* Update macs if enabled */ +if (mac_updating) { +for (i =0 ; i < nb_dq; i++) +update_mac_addrs (mbufs_dst[i], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, +0 , (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); +} + +/* Main rx and tx loop if only one slave lcore available */ static +void +rxtx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx and tx loop for copy on" +" lcore %u\n" ,rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) { +ioat_rx_port (&cfg.ports[i]); +ioat_tx_port (&cfg.ports[i]); + } +} + +static void start_forwarding_cores (void ) { +uint32_t lcore_id =rte_lcore_id (); + +RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , + __func__,rte_lcore_id ()); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); +} +/* Display usage */ static void ioat_usage (const char *prgname)@@ -102,7 +238,7 @@ ioat_usage(const char *prgname)" - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n" " -c --copy-type CT: type of copy: sw|rawdev\n" " -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n" , - prgname); + prgname); } static int
examples/ioat/ioatfwd.c ioat_parse_args(int argc, char **argv, unsigned int nb_ports) · line 300
Quoted 5 lines
examples/ioat/ioatfwd.c +1
-1@@ -161,16 +297,16 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports) argvopt = argv;while ((opt =getopt_long (argc, argvopt, short_options, - lgopts, &option_index)) !=EOF ) { + lgopts, &option_index)) !=EOF ) {
These and the other whitespace changes in this patch should be fixed in patch 1 rather than having them as modifications here.
Without these whitespace changes merged into original patch:
[Marcin]: Whitespace changes moved to previous commit for v6.
PATCH v5 3/6 examples/ioat: add rawdev copy mode support
In reply to Marcin BaranAdded support for copying packets using
rawdev device. Each port's Rx queue is
assigned DMA channel for copy.
--- examples/ioat/ioatfwd.c | 236 ++++++++++++++++++++++++++++++++-------- 1 file changed, 189 insertions(+), 47 deletions(-)
examples/ioat/ioatfwd.c +189 -47
@@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) rte_pktmbuf_mtod(src, char *), src->data_len); } +static uint32_t +ioat_enqueue_packets (struct rte_mbuf **pkts, +uint32_t nb_rx,uint16_t dev_id) +{ +int ret; +uint32_t i; +struct rte_mbuf *pkts_copy[MAX_PKT_BURST ]; + +const uint64_t addr_offset =RTE_PTR_DIFF (pkts[0 ]->buf_addr, + &pkts[0 ]->rearm_data); + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE ,"Unable to allocate memory.\n" ); + +for (i =0 ; i < nb_rx; i++) { +/* Perform data copy */ + ret =rte_ioat_enqueue_copy (dev_id, + pkts[i]->buf_iova + - addr_offset, + pkts_copy[i]->buf_iova + - addr_offset, +rte_pktmbuf_data_len (pkts[i]) + + addr_offset, + (uintptr_t )pkts[i], + (uintptr_t )pkts_copy[i], +0 /* nofence */ ); + +if (ret !=1 ) +break ; + } + + ret = i; +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i); +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], + nb_rx - i); + + +return ret; +} +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ static void ioat_rx_port (struct rxtx_port_config *rx_config)
@@ -136,32 +180,40 @@ ioat_rx_port(struct rxtx_port_config *rx_config)if (nb_rx ==0 )continue ; -/* Perform packet software copy, free source packets */ -int ret; -struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; - - ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, - (void *)pkts_burst_copy, nb_rx); - -if (unlikely (ret <0 )) -rte_exit (EXIT_FAILURE , -"Unable to allocate memory.\n" ); - -for (j =0 ; j < nb_rx; j++) -pktmbuf_sw_copy (pkts_burst[j], - pkts_burst_copy[j]); - -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)pkts_burst, nb_rx); - - nb_enq =rte_ring_enqueue_burst ( - rx_config->rx_to_tx_ring, - (void *)pkts_burst_copy, nb_rx,NULL ); - -/* Free any not enqueued packets. */ -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)&pkts_burst_copy[nb_enq], - nb_rx - nb_enq); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Perform packet hardware copy */ + nb_enq =ioat_enqueue_packets (pkts_burst, + nb_rx, rx_config->ioat_ids[i]); +if (nb_enq >0 ) +rte_ioat_do_copies (rx_config->ioat_ids[i]); + }else { +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } } }
@@ -169,31 +221,64 @@ ioat_rx_port(struct rxtx_port_config *rx_config)static void ioat_tx_port (struct rxtx_port_config *tx_config) { -uint32_t i, nb_dq =0 ; +uint32_t i, j, nb_dq =0 ; +struct rte_mbuf *mbufs_src[MAX_PKT_BURST ];struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ]; -/* Deque the mbufs from rx_to_tx_ring. */ - nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, - (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Deque the mbufs from IOAT device. */ +for (i =0 ; i < tx_config->nb_queues; i++) { + nb_dq =rte_ioat_completed_copies ( + tx_config->ioat_ids[i],MAX_PKT_BURST , + (void *)mbufs_src, (void *)mbufs_dst); -if (nb_dq ==0 ) -return ; +if (nb_dq ==0 ) +break ; -/* Update macs if enabled */ -if (mac_updating) { -for (i =0 ; i < nb_dq; i++) -update_mac_addrs (mbufs_dst[i], - tx_config->rxtx_port); - } +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)mbufs_src, nb_dq); -const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, -0 , (void *)mbufs_dst, nb_dq); +/* Update macs if enabled */ +if (mac_updating) { +for (j =0 ; j < nb_dq; j++) +update_mac_addrs (mbufs_dst[j], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst ( + tx_config->rxtx_port,0 , + (void *)mbufs_dst, nb_dq); -/* Free any unsent packets. */ -if (unlikely (nb_tx < nb_dq)) -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)&mbufs_dst[nb_tx], - nb_dq - nb_tx); +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } + }else { +/* Deque the mbufs from rx_to_tx_ring. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); + +if (nb_dq ==0 ) +return ; + +/* Update macs if enabled */ +if (mac_updating) { +for (i =0 ; i < nb_dq; i++) +update_mac_addrs (mbufs_dst[i], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, +0 , (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } }/* Main rx and tx loop if only one slave lcore available */
@@ -389,6 +474,52 @@ check_link_status(uint32_t port_mask)return retval; } +static void +configure_rawdev_queue (uint32_t dev_id) +{ +struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size }; +struct rte_rawdev_info info = { .dev_private = &dev_config }; + +if (rte_rawdev_configure (dev_id, &info) !=0 ) { +rte_exit (EXIT_FAILURE , +"Error with rte_rawdev_configure()\n" ); + } +if (rte_rawdev_start (dev_id) !=0 ) { +rte_exit (EXIT_FAILURE , +"Error with rte_rawdev_start()\n" ); + } +} + +static void +assign_rawdevs (void ) +{ +uint16_t nb_rawdev =0 , rdev_id =0 ; +uint32_t i, j; + +for (i =0 ; i < cfg.nb_ports; i++) { +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { +struct rte_rawdev_info rdev_info = {0 }; + +do { +if (rdev_id ==rte_rawdev_count ()) + goto end; +rte_rawdev_info_get (rdev_id++, &rdev_info); + }while (strcmp (rdev_info.driver_name, +IOAT_PMD_RAWDEV_NAME_STR ) !=0 ); + + cfg.ports[i].ioat_ids[j] = rdev_id -1 ; +configure_rawdev_queue (cfg.ports[i].ioat_ids[j]); + ++nb_rawdev; + } + } +end: +if (nb_rawdev < cfg.nb_ports * cfg.ports[0 ].nb_queues) +rte_exit (EXIT_FAILURE , +"Not enough IOAT rawdevs (%u) for all queues (%u).\n" , + nb_rawdev, cfg.nb_ports * cfg.ports[0 ].nb_queues); +RTE_LOG (INFO ,IOAT ,"Number of used rawdevs: %u.\n" , nb_rawdev); +} +static void assign_rings (void ) {
@@ -585,18 +716,29 @@ main(int argc, char **argv)rte_exit (EXIT_FAILURE ,"There should be at least one slave lcore.\n" ); -assign_rings (); +if (copy_mode ==COPY_MODE_IOAT_NUM ) +assign_rawdevs (); +else /* copy_mode == COPY_MODE_SW_NUM */ +assign_rings ();start_forwarding_cores ();/* force_quit is true when we get here */ rte_eal_mp_wait_lcore (); +uint32_t j;for (i =0 ; i < cfg.nb_ports; i++) {printf ("Closing port %d\n" , cfg.ports[i].rxtx_port);rte_eth_dev_stop (cfg.ports[i].rxtx_port);rte_eth_dev_close (cfg.ports[i].rxtx_port); -rte_ring_free (cfg.ports[i].rx_to_tx_ring); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { +printf ("Stopping rawdev %d\n" , + cfg.ports[i].ioat_ids[j]); +rte_rawdev_stop (cfg.ports[i].ioat_ids[j]); + } + }else /* copy_mode == COPY_MODE_SW_NUM */ +rte_ring_free (cfg.ports[i].rx_to_tx_ring); }printf ("Bye...\n" );
Signature · 1 line
2.22.0.windows.1
Re: [dpdk-dev] [PATCH v5 3/6] examples/ioat: add rawdev copy mode support
Quoted 119 lines · Marcin Baran
Added support for copying packets using
rawdev device. Each port's Rx queue is
assigned DMA channel for copy.Signed-off-by Marcin Baran <[email protected]>Signed-off-by Pawel Modrak <[email protected]>--- examples/ioat/ioatfwd.c | 236 ++++++++++++++++++++++++++++++++-------- 1 file changed, 189 insertions(+), 47 deletions(-)examples/ioat/ioatfwd.c +68
-26@@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) rte_pktmbuf_mtod(src, char *), src->data_len); } +static uint32_t +ioat_enqueue_packets (struct rte_mbuf **pkts, +uint32_t nb_rx,uint16_t dev_id) +{ +int ret; +uint32_t i; +struct rte_mbuf *pkts_copy[MAX_PKT_BURST ]; + +const uint64_t addr_offset =RTE_PTR_DIFF (pkts[0 ]->buf_addr, + &pkts[0 ]->rearm_data); + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE ,"Unable to allocate memory.\n" ); + +for (i =0 ; i < nb_rx; i++) { +/* Perform data copy */ + ret =rte_ioat_enqueue_copy (dev_id, + pkts[i]->buf_iova + - addr_offset, + pkts_copy[i]->buf_iova + - addr_offset, +rte_pktmbuf_data_len (pkts[i]) + + addr_offset, + (uintptr_t )pkts[i], + (uintptr_t )pkts_copy[i], +0 /* nofence */ ); + +if (ret !=1 ) +break ; + } + + ret = i; +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i); +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], + nb_rx - i); + + +return ret; +} +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ static void ioat_rx_port (struct rxtx_port_config *rx_config)@@ -136,32 +180,40 @@ ioat_rx_port(struct rxtx_port_config *rx_config)if (nb_rx ==0 )continue ; -/* Perform packet software copy, free source packets */ -int ret; -struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; - - ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, - (void *)pkts_burst_copy, nb_rx); - -if (unlikely (ret <0 )) -rte_exit (EXIT_FAILURE , -"Unable to allocate memory.\n" ); - -for (j =0 ; j < nb_rx; j++) -pktmbuf_sw_copy (pkts_burst[j], - pkts_burst_copy[j]); - -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)pkts_burst, nb_rx); - - nb_enq =rte_ring_enqueue_burst ( - rx_config->rx_to_tx_ring, - (void *)pkts_burst_copy, nb_rx,NULL ); - -/* Free any not enqueued packets. */ -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)&pkts_burst_copy[nb_enq], - nb_rx - nb_enq); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Perform packet hardware copy */ + nb_enq =ioat_enqueue_packets (pkts_burst, + nb_rx, rx_config->ioat_ids[i]); +if (nb_enq >0 ) +rte_ioat_do_copies (rx_config->ioat_ids[i]); + }else { +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx);
examples/ioat/ioatfwd.c ioat_rx_port(struct rxtx_port_config *rx_config) · line 216
Quoted 10 lines
examples/ioat/ioatfwd.c +10
-0+ + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + }
Would the diff in this patch be smaller if you switched the order of the branches so that the SW copy leg, which was added first, was processed first? You could even add in a dummy branch in patch 2, so that the indentation for that section remains unchanged.
/Bruce
Re: [dpdk-dev] [PATCH v5 3/6] examples/ioat: add rawdev copy mode support
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Friday, September 27, 2019 12:06 PM
To: Baran, MarcinX <[email protected]>
Cc: [email protected]; Modrak, PawelX <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v5 3/6] examples/ioat: add rawdev copy mode support
Quoted 118 lines · Marcin Baran
Added support for copying packets using rawdev device. Each port's Rx queue is assigned DMA channel for copy.
Signed-off-by Marcin Baran <[email protected]>Signed-off-by Pawel Modrak <[email protected]>--- examples/ioat/ioatfwd.c | 236 ++++++++++++++++++++++++++++++++-------- 1 file changed, 189 insertions(+), 47 deletions(-)examples/ioat/ioatfwd.c index renamed from examples/ioat/ioatfwd.c +0
-0
3a092c6cf..c66ce7e49100644examples/ioat/ioatfwd.c +44
-0@@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) rte_pktmbuf_mtod(src, char *), src->data_len); } +static uint32_t +ioat_enqueue_packets (struct rte_mbuf **pkts, +uint32_t nb_rx,uint16_t dev_id) +{ +int ret; +uint32_t i; +struct rte_mbuf *pkts_copy[MAX_PKT_BURST ]; + +const uint64_t addr_offset =RTE_PTR_DIFF (pkts[0 ]->buf_addr, + &pkts[0 ]->rearm_data); + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE ,"Unable to allocate memory.\n" ); + +for (i =0 ; i < nb_rx; i++) { +/* Perform data copy */ + ret =rte_ioat_enqueue_copy (dev_id, + pkts[i]->buf_iova + - addr_offset, + pkts_copy[i]->buf_iova + - addr_offset, +rte_pktmbuf_data_len (pkts[i]) + + addr_offset, + (uintptr_t )pkts[i], + (uintptr_t )pkts_copy[i], +0 /* nofence */ ); + +if (ret !=1 ) +break ; + } + + ret = i; +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i); +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], + nb_rx - i); + + +return ret; +} +/* Receive packets on one port and enqueue to IOAT rawdev or
rte_ring. */ static voidioat_rx_port(struct rxtx_port_config *rx_config)@@ -136,32 +180,40 @@ioat_rx_port(struct rxtx_port_config *rx_config)examples/ioat/ioatfwd.c +24
-26if (nb_rx ==0 )continue ; -/* Perform packet software copy, free source packets */ -int ret; -struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; - - ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, - (void *)pkts_burst_copy, nb_rx); - -if (unlikely (ret <0 )) -rte_exit (EXIT_FAILURE , -"Unable to allocate memory.\n" ); - -for (j =0 ; j < nb_rx; j++) -pktmbuf_sw_copy (pkts_burst[j], - pkts_burst_copy[j]); - -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)pkts_burst, nb_rx); - - nb_enq =rte_ring_enqueue_burst ( - rx_config->rx_to_tx_ring, - (void *)pkts_burst_copy, nb_rx,NULL ); - -/* Free any not enqueued packets. */ -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)&pkts_burst_copy[nb_enq], - nb_rx - nb_enq); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Perform packet hardware copy */ + nb_enq =ioat_enqueue_packets (pkts_burst, + nb_rx, rx_config->ioat_ids[i]); +if (nb_enq >0 ) +rte_ioat_do_copies (rx_config->ioat_ids[i]); + }else { +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx);
examples/ioat/ioatfwd.c ioat_rx_port(struct rxtx_port_config *rx_config) · line 216
Quoted 10 lines
examples/ioat/ioatfwd.c +10
-0+ + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + }
Would the diff in this patch be smaller if you switched the order of the branches so that the SW copy leg, which was added first, was processed first? You could even add in a dummy branch in patch 2, so that the indentation for that section remains unchanged.
/Bruce
[Marcin] Switched the order and added dummy branch in patch 2. Also changed ioat_tx_port() function the same way in v6.
PATCH v5 4/6 examples/ioat: add two threads configuration
In reply to Marcin BaranFrom: Pawel Modrak <[email protected]>
Added possibility to use two lcores: first for
packet receiving and copying, second for packets
sending.
--- examples/ioat/ioatfwd.c | 92 ++++++++++++++++++++++++++++++----------- 1 file changed, 67 insertions(+), 25 deletions(-)
examples/ioat/ioatfwd.c +67 -25
@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], nb_rx - i); -return ret; }
@@ -226,8 +225,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ];if (copy_mode ==COPY_MODE_IOAT_NUM ) { -/* Deque the mbufs from IOAT device. */ for (i =0 ; i < tx_config->nb_queues; i++) { +/* Deque the mbufs from IOAT device. */ nb_dq =rte_ioat_completed_copies ( tx_config->ioat_ids[i],MAX_PKT_BURST , (void *)mbufs_src, (void *)mbufs_dst);
@@ -256,31 +255,64 @@ ioat_tx_port(struct rxtx_port_config *tx_config) nb_dq - nb_tx); } }else { -/* Deque the mbufs from rx_to_tx_ring. */ - nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, - (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); - -if (nb_dq ==0 ) -return ; - -/* Update macs if enabled */ -if (mac_updating) { -for (i =0 ; i < nb_dq; i++) -update_mac_addrs (mbufs_dst[i], - tx_config->rxtx_port); - } +for (i =0 ; i < tx_config->nb_queues; i++) { +/* Deque the mbufs from IOAT device. */ + nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, + (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); -const uint16_t nb_tx =rte_eth_tx_burst (tx_config->rxtx_port, -0 , (void *)mbufs_dst, nb_dq); +if (nb_dq ==0 ) +return ; -/* Free any unsent packets. */ -if (unlikely (nb_tx < nb_dq)) -rte_mempool_put_bulk (ioat_pktmbuf_pool, - (void *)&mbufs_dst[nb_tx], - nb_dq - nb_tx); +/* Update macs if enabled */ +if (mac_updating) { +for (j =0 ; j < nb_dq; j++) +update_mac_addrs (mbufs_dst[j], + tx_config->rxtx_port); + } + +const uint16_t nb_tx = +rte_eth_tx_burst (tx_config->rxtx_port,0 , + (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } } } +/* Main rx processing loop for IOAT rawdev. */ +static void +rx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx loop for copy on lcore %u\n" , +rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) +ioat_rx_port (&cfg.ports[i]); +} + +/* Main tx processing loop for hardware copy. */ +static void +tx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main tx loop for copy on lcore %u\n" , +rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) +ioat_tx_port (&cfg.ports[i]); +} +/* Main rx and tx loop if only one slave lcore available */ static void rxtx_main_loop (void )
@@ -305,9 +337,19 @@ static void start_forwarding_cores(void)RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , __func__,rte_lcore_id ()); - lcore_id =rte_get_next_lcore (lcore_id, true, true); -rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, -NULL , lcore_id); +if (cfg.nb_lcores ==1 ) { + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); + }else if (cfg.nb_lcores >1 ) { + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rx_main_loop, +NULL , lcore_id); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)tx_main_loop,NULL , + lcore_id); + } }/* Display usage */
Signature · 1 line
2.22.0.windows.1
Re: [dpdk-dev] [PATCH v5 4/6] examples/ioat: add two threads configuration
Quoted 42 lines · Marcin Baran
From: Pawel Modrak <[email protected]>
Added possibility to use two lcores: first for
packet receiving and copying, second for packets
sending.Signed-off-by Pawel Modrak <[email protected]>Signed-off-by Marcin Baran <[email protected]>--- examples/ioat/ioatfwd.c | 92 ++++++++++++++++++++++++++++++----------- 1 file changed, 67 insertions(+), 25 deletions(-)examples/ioat/ioatfwd.c +1
-6@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], nb_rx - i); -return ret; }@@ -226,8 +225,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ];if (copy_mode ==COPY_MODE_IOAT_NUM ) { -/* Deque the mbufs from IOAT device. */ for (i =0 ; i < tx_config->nb_queues; i++) { +/* Deque the mbufs from IOAT device. */ nb_dq =rte_ioat_completed_copies ( tx_config->ioat_ids[i],MAX_PKT_BURST , (void *)mbufs_src, (void *)mbufs_dst);@@ -256,31 +255,64 @@ ioat_tx_port(struct rxtx_port_config *tx_config) nb_dq - nb_tx); } }else { -/* Deque the mbufs from rx_to_tx_ring. */ - nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, - (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); -
examples/ioat/ioatfwd.c ioat_tx_port(struct rxtx_port_config *tx_config) · line 258
Quoted 10 lines
examples/ioat/ioatfwd.c +1
-9-if (nb_dq ==0 ) -return ; - -/* Update macs if enabled */ -if (mac_updating) { -for (i =0 ; i < nb_dq; i++) -update_mac_addrs (mbufs_dst[i], - tx_config->rxtx_port); - } +for (i =0 ; i < tx_config->nb_queues; i++) {
This commit refers to adding support for 2 threads rather than 1, but this change seems related to having multiple queues rather than threads. Shoudl it be in a previous patch?
/Bruce
Re: [dpdk-dev] [PATCH v5 4/6] examples/ioat: add two threads configuration
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Friday, September 27, 2019 12:09 PM
To: Baran, MarcinX <[email protected]>
Cc: [email protected]; Modrak, PawelX <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v5 4/6] examples/ioat: add two threads configuration
Quoted 42 lines · Marcin Baran
From: Pawel Modrak <[email protected]>
Added possibility to use two lcores: first for packet receiving and copying, second for packets sending.
Signed-off-by Pawel Modrak <[email protected]>Signed-off-by Marcin Baran <[email protected]>--- examples/ioat/ioatfwd.c | 92 ++++++++++++++++++++++++++++++----------- 1 file changed, 67 insertions(+), 25 deletions(-)examples/ioat/ioatfwd.c index renamed from examples/ioat/ioatfwd.c +0
-0
c66ce7e49..4c51db6bd100644examples/ioat/ioatfwd.c +1
-2@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], nb_rx - i); -return ret; }@@ -226,8 +225,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ];if (copy_mode ==COPY_MODE_IOAT_NUM ) { -/* Deque the mbufs from IOAT device. */ for (i =0 ; i < tx_config->nb_queues; i++) { +/* Deque the mbufs from IOAT device. */ nb_dq =rte_ioat_completed_copies ( tx_config->ioat_ids[i],MAX_PKT_BURST , (void *)mbufs_src, (void *)mbufs_dst); @@ -256,31 +255,64 @@
ioat_tx_port(struct rxtx_port_config *tx_config)examples/ioat/ioatfwd.c +0
-4nb_dq - nb_tx); } }else { -/* Deque the mbufs from rx_to_tx_ring. */ - nb_dq =rte_ring_dequeue_burst (tx_config->rx_to_tx_ring, - (void *)mbufs_dst,MAX_PKT_BURST ,NULL ); -
examples/ioat/ioatfwd.c ioat_tx_port(struct rxtx_port_config *tx_config) · line 258
Quoted 10 lines
examples/ioat/ioatfwd.c +1
-9-if (nb_dq ==0 ) -return ; - -/* Update macs if enabled */ -if (mac_updating) { -for (i =0 ; i < nb_dq; i++) -update_mac_addrs (mbufs_dst[i], - tx_config->rxtx_port); - } +for (i =0 ; i < tx_config->nb_queues; i++) {
This commit refers to adding support for 2 threads rather than 1, but this change seems related to having multiple queues rather than threads. Shoudl it be in a previous patch?
/Bruce
[Marcin] Moved the change to 2nd patch for v6.
PATCH v5 5/6 examples/ioat: add stats printing for each port
In reply to Marcin BaranAdded printing stats of ports each second. The stats printing is done using master core. The information provided informs about packets received, dropped and send as well as statistics of rawdev devices.
--- examples/ioat/ioatfwd.c | 245 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 241 insertions(+), 4 deletions(-)
examples/ioat/ioatfwd.c +241 -4
@@ -48,10 +48,27 @@ struct rxtx_transmission_config {uint16_t nb_lcores; }; +/* per-port statistics struct */ +struct ioat_port_statistics { +uint64_t rx[RTE_MAX_ETHPORTS ]; +uint64_t tx[RTE_MAX_ETHPORTS ]; +uint64_t tx_dropped[RTE_MAX_ETHPORTS ]; +uint64_t copy_dropped[RTE_MAX_ETHPORTS ]; +}; +struct ioat_port_statistics port_statistics; + +struct total_statistics { +uint64_t total_packets_dropped; +uint64_t total_packets_tx; +uint64_t total_packets_rx; +uint64_t total_successful_enqueues; +uint64_t total_failed_enqueues; +}; + typedef enum copy_mode_t { #define COPY_MODE_SW "sw" COPY_MODE_SW_NUM, -#define COPY_MODE_IOAT "rawdev" +#define COPY_MODE_IOAT "hw" COPY_MODE_IOAT_NUM, COPY_MODE_INVALID_NUM, COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
@@ -89,6 +106,204 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ];struct rte_mempool *ioat_pktmbuf_pool; +/* Print out statistics for one port. */ +static void +print_port_stats (uint16_t port_id) +{ +printf ("\nStatistics for port %u ------------------------------" +"\nPackets sent: %34" PRIu64 +"\nPackets received: %30" PRIu64 +"\nPackets dropped on tx: %25" PRIu64 +"\nPackets dropped on copy: %23" PRIu64, + port_id, + port_statistics.tx[port_id], + port_statistics.rx[port_id], + port_statistics.tx_dropped[port_id], + port_statistics.copy_dropped[port_id]); +} + +/* Print out statistics for one IOAT rawdev device. */ +static void +print_rawdev_stats (uint32_t dev_id,uint64_t *xstats, +unsigned int *ids_xstats,uint16_t nb_xstats, +struct rte_rawdev_xstats_name *names_xstats) +{ +uint16_t i; + +printf ("\nIOAT channel %u" , dev_id); +for (i =0 ; i < nb_xstats; i++) +printf ("\n\t %s: %*" PRIu64, + names_xstats[ids_xstats[i]].name, + (int )(37 -strlen (names_xstats[ids_xstats[i]].name)), + xstats[i]); +} + +static void +print_total_stats (struct total_statistics *ts) +{ +printf ("\nAggregate statistics ===============================" +"\nTotal packets Tx: %24" PRIu64" [pps]" +"\nTotal packets Rx: %24" PRIu64" [pps]" +"\nTotal packets dropped: %19" PRIu64" [pps]" , + ts->total_packets_tx, + ts->total_packets_rx, + ts->total_packets_dropped); + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +printf ("\nTotal IOAT successful enqueues: %8" PRIu64" [enq/s]" +"\nTotal IOAT failed enqueues: %12" PRIu64" [enq/s]" , + ts->total_successful_enqueues, + ts->total_failed_enqueues); + } + +printf ("\n====================================================\n" ); +} + +/* Print out statistics on packets dropped. */ +static void +print_stats (char *prgname) +{ +struct total_statistics ts, delta_ts; +uint32_t i, port_id, dev_id; +struct rte_rawdev_xstats_name *names_xstats; +uint64_t *xstats; +unsigned int *ids_xstats, nb_xstats; +char status_string[120 ];/* to print at the top of the output */ +int status_strlen; + + +const char clr[] = {27 ,'[' ,'2' ,'J' ,'\0' }; +const char topLeft[] = {27 ,'[' ,'1' ,';' ,'1' ,'H' ,'\0' }; + + status_strlen =snprintf (status_string,sizeof (status_string), +"%s, " , prgname); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Worker Threads = %d, " , +rte_lcore_count () >2 ?2 :1 ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Copy Mode = %s,\n" , copy_mode ==COPY_MODE_SW_NUM ? +COPY_MODE_SW :COPY_MODE_IOAT ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Updating MAC = %s, " , mac_updating ? +"enabled" :"disabled" ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Rx Queues = %d, " , nb_queues); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Ring Size = %d\n" , ring_size); + +/* Allocate memory for xstats names and values */ + nb_xstats =rte_rawdev_xstats_names_get ( + cfg.ports[0 ].ioat_ids[0 ],NULL ,0 ); + + names_xstats =malloc (sizeof (*names_xstats) * nb_xstats); +if (names_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat names memory\n" ); + } +rte_rawdev_xstats_names_get (cfg.ports[0 ].ioat_ids[0 ], + names_xstats, nb_xstats); + + ids_xstats =malloc (sizeof (*ids_xstats) *2 ); +if (ids_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat ids_xstats memory\n" ); + } + + xstats =malloc (sizeof (*xstats) *2 ); +if (xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat memory\n" ); + } + +/* Get failed/successful enqueues stats index */ + ids_xstats[0 ] = ids_xstats[1 ] = nb_xstats; +for (i =0 ; i < nb_xstats; i++) { +if (!strcmp (names_xstats[i].name,"failed_enqueues" )) + ids_xstats[0 ] = i; +else if (!strcmp (names_xstats[i].name,"successful_enqueues" )) + ids_xstats[1 ] = i; +if (ids_xstats[0 ] < nb_xstats && ids_xstats[1 ] < nb_xstats) +break ; + } +if (ids_xstats[0 ] == nb_xstats || ids_xstats[1 ] == nb_xstats) { +rte_exit (EXIT_FAILURE , +"Error getting failed/successful enqueues stats index\n" ); + } + +memset (&ts,0 ,sizeof (struct total_statistics )); + +while (!force_quit) { +/* Sleep for 1 second each round - init sleep allows reading +* messages from app startup. +*/ +sleep (1 ); + +/* Clear screen and move to top left */ +printf ("%s%s" , clr, topLeft); + +memset (&delta_ts,0 ,sizeof (struct total_statistics )); + +printf ("%s" , status_string); + +for (i =0 ; i < cfg.nb_ports; i++) { + port_id = cfg.ports[i].rxtx_port; +print_port_stats (port_id); + + delta_ts.total_packets_dropped += + port_statistics.tx_dropped[port_id] + + port_statistics.copy_dropped[port_id]; + delta_ts.total_packets_tx += + port_statistics.tx[port_id]; + delta_ts.total_packets_rx += + port_statistics.rx[port_id]; + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +uint32_t j; + +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { + dev_id = cfg.ports[i].ioat_ids[j]; +rte_rawdev_xstats_get (dev_id, + ids_xstats, xstats,2 ); + +print_rawdev_stats (dev_id, xstats, + ids_xstats,2 , names_xstats); + + delta_ts.total_failed_enqueues += + xstats[ids_xstats[0 ]]; + delta_ts.total_successful_enqueues += + xstats[ids_xstats[1 ]]; + } + } + } + + delta_ts.total_packets_tx -= ts.total_packets_tx; + delta_ts.total_packets_rx -= ts.total_packets_rx; + delta_ts.total_packets_dropped -= ts.total_packets_dropped; + delta_ts.total_failed_enqueues -= ts.total_failed_enqueues; + delta_ts.total_successful_enqueues -= + ts.total_successful_enqueues; + +printf ("\n" ); +print_total_stats (&delta_ts); + + ts.total_packets_tx += delta_ts.total_packets_tx; + ts.total_packets_rx += delta_ts.total_packets_rx; + ts.total_packets_dropped += delta_ts.total_packets_dropped; + ts.total_failed_enqueues += delta_ts.total_failed_enqueues; + ts.total_successful_enqueues += + delta_ts.total_successful_enqueues; + } + +free (names_xstats); +free (xstats); +free (ids_xstats); +} +static void update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) {
@@ -179,6 +394,8 @@ ioat_rx_port(struct rxtx_port_config *rx_config)if (nb_rx ==0 )continue ; + port_statistics.rx[rx_config->rxtx_port] += nb_rx; +if (copy_mode ==COPY_MODE_IOAT_NUM ) {/* Perform packet hardware copy */ nb_enq = ioat_enqueue_packets(pkts_burst,
@@ -213,6 +430,9 @@ ioat_rx_port(struct rxtx_port_config *rx_config) (void *)&pkts_burst_copy[nb_enq], nb_rx - nb_enq); } + + port_statistics.copy_dropped[rx_config->rxtx_port] += + (nb_rx - nb_enq); } }
@@ -248,6 +468,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config) tx_config->rxtx_port,0 , (void *)mbufs_dst, nb_dq); + port_statistics.tx[tx_config->rxtx_port] += nb_tx; +/* Free any unsent packets. */ if (unlikely (nb_tx < nb_dq)) rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -274,6 +496,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)rte_eth_tx_burst (tx_config->rxtx_port,0 , (void *)mbufs_dst, nb_dq); + port_statistics.tx[tx_config->rxtx_port] += nb_tx; +/* Free any unsent packets. */ if (unlikely (nb_tx < nb_dq)) rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -363,7 +587,7 @@ ioat_usage(const char *prgname)" When enabled:\n" " - The source MAC address is replaced by the TX port MAC address\n" " - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n" -" -c --copy-type CT: type of copy: sw|rawdev\n" +" -c --copy-type CT: type of copy: sw|hw\n" " -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n" , prgname); }
@@ -400,7 +624,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)static const char short_options[] ="p:" /* portmask */ "q:" /* number of RX queues per port */ -"c:" /* copy type (sw|rawdev) */ +"c:" /* copy type (sw|hw) */ "s:" /* ring size */ ;
@@ -452,7 +676,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)case 'c' : copy_mode =ioat_parse_copy_mode (optarg);if (copy_mode ==COPY_MODE_INVALID_NUM ) { -printf ("Invalid copy type. Use: sw, rawdev\n" ); +printf ("Invalid copy type. Use: sw, hw\n" );ioat_usage (prgname);return -1 ; }
@@ -673,6 +897,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)rte_eth_tx_buffer_init (tx_buffer[portid],MAX_PKT_BURST ); + ret =rte_eth_tx_buffer_set_err_callback (tx_buffer[portid], + rte_eth_tx_buffer_count_callback, + &port_statistics.tx_dropped[portid]); +if (ret <0 ) +rte_exit (EXIT_FAILURE , +"Cannot set error callback for tx buffer on port %u\n" , + portid); +/* Start device */ ret =rte_eth_dev_start (portid);if (ret <0 )
@@ -749,6 +981,9 @@ main(int argc, char **argv)RTE_ETH_FOREACH_DEV (portid)port_init (portid, ioat_pktmbuf_pool, nb_queues); +/* Initialize port xstats */ +memset (&port_statistics,0 ,sizeof (port_statistics)); +while (!check_link_status (ioat_enabled_port_mask) && !force_quit)sleep (1 );
@@ -764,6 +999,8 @@ main(int argc, char **argv)assign_rings ();start_forwarding_cores (); +/* master core prints stats while other cores forward */ +print_stats (argv[0 ]);/* force_quit is true when we get here */ rte_eal_mp_wait_lcore ();
Signature · 1 line
2.22.0.windows.1
Re: [dpdk-dev] [PATCH v5 5/6] examples/ioat: add stats printing for each port
Quoted 17 lines · Marcin Baran
Added printing stats of ports each second. The stats printing is done using master core. The information provided informs about packets received, dropped and send as well as statistics of rawdev devices.
Signed-off-by Marcin Baran <[email protected]>Signed-off-by Pawel Modrak <[email protected]>--- examples/ioat/ioatfwd.c | 245 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 241 insertions(+), 4 deletions(-)examples/ioat/ioatfwd.c +0
-0
examples/ioat/ioatfwd.c struct rxtx_transmission_config { · line 50
Quoted 3 lines
examples/ioat/ioatfwd.c +0
-0@@ -48,10 +48,27 @@ struct rxtx_transmission_config {uint16_t nb_lcores; };
<snip stats code>
examples/ioat/ioatfwd.c ioat_usage(const char *prgname) · line 593
Quoted 9 lines
examples/ioat/ioatfwd.c +1
-1@@ -363,7 +587,7 @@ ioat_usage(const char *prgname)" When enabled:\n" " - The source MAC address is replaced by the TX port MAC address\n" " - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n" -" -c --copy-type CT: type of copy: sw|rawdev\n" +" -c --copy-type CT: type of copy: sw|hw\n" " -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n" , prgname); }
This change is not relevant to this patch, and should be corrected in the original patch of the series. Try to avoid adding a line and then modifying it in a subsequent patch in the same set.
/Bruce
Re: [dpdk-dev] [PATCH v5 5/6] examples/ioat: add stats printing for each port
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Friday, September 27, 2019 12:13 PM
To: Baran, MarcinX <[email protected]>
Cc: [email protected]; Modrak, PawelX <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v5 5/6] examples/ioat: add stats printing for each port
Quoted 17 lines · Marcin Baran
Added printing stats of ports each second.
The stats printing is done using master core.
The information provided informs about packets received, dropped and send as well as statistics of rawdev devices.Signed-off-by Marcin Baran <[email protected]>Signed-off-by Pawel Modrak <[email protected]>--- examples/ioat/ioatfwd.c | 245 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 241 insertions(+), 4 deletions(-)examples/ioat/ioatfwd.c index renamed from examples/ioat/ioatfwd.c +0
-0
4c51db6bd..c6b994832100644
examples/ioat/ioatfwd.c struct rxtx_transmission_config { · line 50
Quoted 3 lines
examples/ioat/ioatfwd.c +0
-0@@ -48,10 +48,27 @@ struct rxtx_transmission_config {uint16_t nb_lcores; };
<snip stats code>
examples/ioat/ioatfwd.c ioat_usage(const char *prgname) · line 593
Quoted 9 lines
examples/ioat/ioatfwd.c +1
-1@@ -363,7 +587,7 @@ ioat_usage(const char *prgname)" When enabled:\n" " - The source MAC address is replaced by the TX port MAC address\n" " - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n" -" -c --copy-type CT: type of copy: sw|rawdev\n" +" -c --copy-type CT: type of copy: sw|hw\n" " -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n" , prgname); }
This change is not relevant to this patch, and should be corrected in the original patch of the series. Try to avoid adding a line and then modifying it in a subsequent patch in the same set.
/Bruce
[Marcin] Moved change to first patch.
PATCH v5 6/6 doc/guides/: provide IOAT sample app guide
In reply to Marcin BaranAdded guide for IOAT sample app usage and
code description.
--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++++++++++ 3 files changed, 769 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst
doc/guides/sample_app_ug/index.rst +1 -0
@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtual
doc/guides/sample_app_ug/intro.rst +4 -0
@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.
doc/guides/sample_app_ug/ioat.rst new file +764 -0
765 lines not on this page · raw message
Signature · 1 line
2.22.0.windows.1
Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
Quoted 44 lines · Marcin Baran
Added guide for IOAT sample app usage and
code description.Signed-off-by Marcin Baran <[email protected]>--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++++++++++ 3 files changed, 769 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rstdoc/guides/sample_app_ug/index.rst +1
-0@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtualdoc/guides/sample_app_ug/intro.rst +4
-0@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.doc/guides/sample_app_ug/ioat.rst new file +0
-0
doc/guides/sample_app_ug/ioat.rst line 6
Quoted 6 lines
doc/guides/sample_app_ug/ioat.rst new file +6
-0@@ -0,0 +1,764 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Intel Corporation. + +Sample Application of packet copying using Intel\ |reg| QuickData Technology +============================================================================ +
Title is too long, you can drop the "Sample Application" at minimum, since this is part of the example applications guide document. Call the section "Packet Copying Using ..."
In order to get the proper (R) symbol, you also need to add an include to the file. Therefore add the following line just after the copyright:
.. include:: <isonum.txt>
/Bruce
Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Friday, September 27, 2019 12:36 PM
To: Baran, MarcinX <[email protected]>
Cc: [email protected]
Subject: Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
Quoted 48 lines · Marcin Baran
Added guide for IOAT sample app usage and code description.
Signed-off-by Marcin Baran <[email protected]>--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++++++++++ 3 files changed, 769 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rstdoc/guides/sample_app_ug/index.rst +1
-0@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtualdoc/guides/sample_app_ug/intro.rst +5
-0@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev +driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.doc/guides/sample_app_ug/ioat.rst new file +0
-0
doc/guides/sample_app_ug/ioat.rst line 8
Quoted 8 lines
doc/guides/sample_app_ug/ioat.rst new file +8
-0@@ -0,0 +1,764 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Intel Corporation. + +Sample Application of packet copying using Intel\ |reg| QuickData +Technology +===================================================================== +======= +
Title is too long, you can drop the "Sample Application" at minimum, since this is part of the example applications guide document. Call the section "Packet Copying Using ..."
In order to get the proper (R) symbol, you also need to add an include to the file. Therefore add the following line just after the copyright:
.. include:: <isonum.txt>
/Bruce
[Marcin] Dropped "Sample Application" and added include.
Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
In reply to Marcin BaranAdding John and Marko on CC as documentation maintainers.
Quoted 812 lines · Marcin Baran
Added guide for IOAT sample app usage and
code description.Signed-off-by Marcin Baran <[email protected]>--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++++++++++ 3 files changed, 769 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rstdoc/guides/sample_app_ug/index.rst +1
-0@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtualdoc/guides/sample_app_ug/intro.rst +4
-0@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.doc/guides/sample_app_ug/ioat.rst new file +764
-0@@ -0,0 +1,764 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Intel Corporation. + +Sample Application of packet copying using Intel\ |reg| QuickData Technology +============================================================================ + +Overview +-------- + +This sample is intended as a demonstration of the basic components of a DPDK +forwarding application and example of how to use IOAT driver API to make +packets copies. + +Also while forwarding, the MAC addresses are affected as follows: + +* The source MAC address is replaced by the TX port MAC address + +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID + +This application can be used to compare performance of using software packet +copy with copy done using a DMA device for different sizes of packets. +The example will print out statistics each second. The stats shows +received/send packets and packets dropped or failed to copy. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``ioat`` sub-directory. + + +Running the Application +----------------------- + +In order to run the hardware copy application, the copying device +needs to be bound to user-space IO driver. + +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData Technology* +guide for information on using the driver. + +The application requires a number of command line options: + +.. code-block:: console + + ./build/ioatfwd [EAL options] -- -p MASK [-q NQ] [-s RS] [-c <sw|hw>] + [--[no-]mac-updating] + +where, + +* p MASK: A hexadecimal bitmask of the ports to configure + +* q NQ: Number of Rx queues used per port equivalent to CBDMA channels + per port + +* c CT: Performed packet copy type: software (sw) or hardware using + DMA (hw) + +* s RS: Size of IOAT rawdev ring for hardware copy mode or rte_ring for + software copy mode + +* --[no-]mac-updating: Whether MAC address of packets should be changed + or not + +The application can be launched in various configurations depending on +provided parameters. Each port can use up to 2 lcores: one of them receives +incoming traffic and makes a copy of each packet. The second lcore then +updates MAC address and sends the copy. If one lcore per port is used, +both operations are done sequentially. For each configuration an additional +lcore is needed since master lcore in use which is responsible for +configuration, statistics printing and safe deinitialization of all ports +and devices. + +The application can use a maximum of 8 ports. + +To run the application in a Linux environment with 3 lcores (one of them +is master lcore), 1 port (port 0), software copying and MAC updating issue +the command: + +.. code-block:: console + + $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw + +To run the application in a Linux environment with 2 lcores (one of them +is master lcore), 2 ports (ports 0 and 1), hardware copying and no MAC +updating issue the command: + +.. code-block:: console + + $ ./build/ioatfwd -l 0-1 -n 1 -- -p 0x3 --no-mac-updating -c hw + +Refer to the *DPDK Getting Started Guide* for general information on +running applications and the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +The following sections provide an explanation of the main components of the +code. + +All DPDK library functions used in the sample code are prefixed with +``rte_`` and are explained in detail in the *DPDK API Documentation*. + + +The Main Function +~~~~~~~~~~~~~~~~~ + +The ``main()`` function performs the initialization and calls the execution +threads for each lcore. + +The first task is to initialize the Environment Abstraction Layer (EAL). +The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()`` +function. The value returned is the number of parsed arguments: + +.. code-block:: c + + /* init EAL */ + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n"); + + +The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers) +used by the application: + +.. code-block:: c + + nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd + + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE), + MIN_POOL_SIZE); + + /* Create the mbuf pool */ + ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs, + MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE, + rte_socket_id()); + if (ioat_pktmbuf_pool == NULL) + rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n"); + +Mbufs are the packet buffer structure used by DPDK. They are explained in +detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*. + +The ``main()`` function also initializes the ports: + +.. code-block:: c + + /* Initialise each port */ + RTE_ETH_FOREACH_DEV(portid) { + port_init(portid, ioat_pktmbuf_pool); + } + +Each port is configured using ``port_init()``: + +.. code-block:: c + + /* + * Initializes a given port using global settings and with the RX buffers + * coming from the mbuf_pool passed as a parameter. + */ + static inline void + port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues) + { + /* configuring port to use RSS for multiple RX queues */ + static const struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = RTE_ETHER_MAX_LEN + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_PROTO_MASK, + } + } + }; + + struct rte_eth_rxconf rxq_conf; + struct rte_eth_txconf txq_conf; + struct rte_eth_conf local_port_conf = port_conf; + struct rte_eth_dev_info dev_info; + int ret, i; + + /* Skip ports that are not enabled */ + if ((ioat_enabled_port_mask & (1 << portid)) == 0) { + printf("Skipping disabled port %u\n", portid); + return; + } + + /* Init port */ + printf("Initializing port %u... ", portid); + fflush(stdout); + rte_eth_dev_info_get(portid, &dev_info); + local_port_conf.rx_adv_conf.rss_conf.rss_hf &= + dev_info.flow_type_rss_offloads; + if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE) + local_port_conf.txmode.offloads |= + DEV_TX_OFFLOAD_MBUF_FAST_FREE; + ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Cannot configure device:" + " err=%d, port=%u\n", ret, portid); + + ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd, + &nb_txd); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot adjust number of descriptors: err=%d, port=%u\n", + ret, portid); + + rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]); + + /* Init Rx queues */ + rxq_conf = dev_info.default_rxconf; + rxq_conf.offloads = local_port_conf.rxmode.offloads; + for (i = 0; i < nb_queues; i++) { + ret = rte_eth_rx_queue_setup(portid, i, nb_rxd, + rte_eth_dev_socket_id(portid), &rxq_conf, + mbuf_pool); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n", + ret, portid, i); + } + + /* Init one TX queue on each port */ + txq_conf = dev_info.default_txconf; + txq_conf.offloads = local_port_conf.txmode.offloads; + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, + rte_eth_dev_socket_id(portid), + &txq_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_tx_queue_setup:err=%d,port=%u\n", + ret, portid); + + /* Initialize TX buffers */ + tx_buffer[portid] = rte_zmalloc_socket("tx_buffer", + RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0, + rte_eth_dev_socket_id(portid)); + if (tx_buffer[portid] == NULL) + rte_exit(EXIT_FAILURE, + "Cannot allocate buffer for tx on port %u\n", + portid); + + rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST); + + ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid], + rte_eth_tx_buffer_count_callback, + &port_statistics.tx_dropped[portid]); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot set error callback for tx buffer on port %u\n", + portid); + + /* Start device */ + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_dev_start:err=%d, port=%u\n", + ret, portid); + + rte_eth_promiscuous_enable(portid); + + printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n", + portid, + ioat_ports_eth_addr[portid].addr_bytes[0], + ioat_ports_eth_addr[portid].addr_bytes[1], + ioat_ports_eth_addr[portid].addr_bytes[2], + ioat_ports_eth_addr[portid].addr_bytes[3], + ioat_ports_eth_addr[portid].addr_bytes[4], + ioat_ports_eth_addr[portid].addr_bytes[5]); + + cfg.ports[cfg.nb_ports].rxtx_port = portid; + cfg.ports[cfg.nb_ports++].nb_queues = nb_queues; + } + +The Ethernet ports are configured with local settings using the +``rte_eth_dev_configure()`` function and the ``port_conf`` struct. +The RSS is enabled so that multiple Rx queues could be used for +packet receiving and copying by multiple CBDMA channels per port: + +.. code-block:: c + + /* configuring port to use RSS for multiple RX queues */ + static const struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = RTE_ETHER_MAX_LEN + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_PROTO_MASK, + } + } + }; + +For this example the ports are set up with the number of Rx queues provided +with -q option and 1 Tx queue using the ``rte_eth_rx_queue_setup()`` +and ``rte_eth_tx_queue_setup()`` functions. + +The Ethernet port is then started: + +.. code-block:: c + + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n", + ret, portid); + + +Finally the Rx port is set in promiscuous mode: + +.. code-block:: c + + rte_eth_promiscuous_enable(portid); + + +After that each port application assigns resources needed. + +.. code-block:: c + + check_link_status(ioat_enabled_port_mask); + + if (!cfg.nb_ports) { + rte_exit(EXIT_FAILURE, + "All available ports are disabled. Please set portmask.\n"); + } + + /* Check if there is enough lcores for all ports. */ + cfg.nb_lcores = rte_lcore_count() - 1; + if (cfg.nb_lcores < 1) + rte_exit(EXIT_FAILURE, + "There should be at least one slave lcore.\n"); + + ret = 0; + + if (copy_mode == COPY_MODE_IOAT_NUM) { + assign_rawdevs(); + } else /* copy_mode == COPY_MODE_SW_NUM */ { + assign_rings(); + } + +A link status is checked of each port enabled by port mask +using ``check_link_status()`` function. + +.. code-block:: c + + /* check link status, return true if at least one port is up */ + static int + check_link_status(uint32_t port_mask) + { + uint16_t portid; + struct rte_eth_link link; + int retval = 0; + + printf("\nChecking link status\n"); + RTE_ETH_FOREACH_DEV(portid) { + if ((port_mask & (1 << portid)) == 0) + continue; + + memset(&link, 0, sizeof(link)); + rte_eth_link_get(portid, &link); + + /* Print link status */ + if (link.link_status) { + printf( + "Port %d Link Up. Speed %u Mbps - %s\n", + portid, link.link_speed, + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + ("full-duplex") : ("half-duplex\n")); + retval = 1; + } else + printf("Port %d Link Down\n", portid); + } + return retval; + } + +Depending on mode set (whether copy should be done by software or by hardware) +special structures are assigned to each port. If software copy was chosen, +application have to assign ring structures for packet exchanging between lcores +assigned to ports. + +.. code-block:: c + + static void + assign_rings(void) + { + uint32_t i; + + for (i = 0; i < cfg.nb_ports; i++) { + char ring_name[20]; + + snprintf(ring_name, 20, "rx_to_tx_ring_%u", i); + /* Create ring for inter core communication */ + cfg.ports[i].rx_to_tx_ring = rte_ring_create( + ring_name, ring_size, + rte_socket_id(), RING_F_SP_ENQ); + + if (cfg.ports[i].rx_to_tx_ring == NULL) + rte_exit(EXIT_FAILURE, "%s\n", + rte_strerror(rte_errno)); + } + } + + +When using hardware copy each Rx queue of the port is assigned an +IOAT device (``assign_rawdevs()``) using IOAT Rawdev Driver API +functions: + +.. code-block:: c + + static void + assign_rawdevs(void) + { + uint16_t nb_rawdev = 0, rdev_id = 0; + uint32_t i, j; + + for (i = 0; i < cfg.nb_ports; i++) { + for (j = 0; j < cfg.ports[i].nb_queues; j++) { + struct rte_rawdev_info rdev_info = { 0 }; + + do { + if (rdev_id == rte_rawdev_count()) + goto end; + rte_rawdev_info_get(rdev_id++, &rdev_info); + } while (strcmp(rdev_info.driver_name, + IOAT_PMD_RAWDEV_NAME_STR) != 0); + + cfg.ports[i].ioat_ids[j] = rdev_id - 1; + configure_rawdev_queue(cfg.ports[i].ioat_ids[j]); + ++nb_rawdev; + } + } + end: + if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues) + rte_exit(EXIT_FAILURE, + "Not enough IOAT rawdevs (%u) for all queues (%u).\n", + nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues); + RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev); + } + + +The initialization of hardware device is done by ``rte_rawdev_configure()`` +function and ``rte_rawdev_info`` struct. After configuration the device is +started using ``rte_rawdev_start()`` function. Each of the above operations +is done in ``configure_rawdev_queue()``. + +.. code-block:: c + + static void + configure_rawdev_queue(uint32_t dev_id) + { + struct rte_rawdev_info info = { .dev_private = &dev_config }; + + /* Configure hardware copy device */ + dev_config.ring_size = ring_size; + + if (rte_rawdev_configure(dev_id, &info) != 0) { + rte_exit(EXIT_FAILURE, + "Error with rte_rawdev_configure()\n"); + } + rte_rawdev_info_get(dev_id, &info); + if (dev_config.ring_size != ring_size) { + rte_exit(EXIT_FAILURE, + "Error, ring size is not %d (%d)\n", + ring_size, (int)dev_config.ring_size); + } + if (rte_rawdev_start(dev_id) != 0) { + rte_exit(EXIT_FAILURE, + "Error with rte_rawdev_start()\n"); + } + } + +If initialization is successful memory for hardware device +statistics is allocated. + +Finally ``main()`` functions starts all processing lcores and starts +printing stats in a loop on master lcore. The application can be +interrupted and closed using ``Ctrl-C``. The master lcore waits for +all slave processes to finish, deallocates resources and exits. + +The processing lcores launching function are described below. + +The Lcores Launching Functions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As described above ``main()`` function invokes ``start_forwarding_cores()`` +function in order to start processing for each lcore: + +.. code-block:: c + + static void start_forwarding_cores(void) + { + uint32_t lcore_id = rte_lcore_id(); + + RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n", + __func__, rte_lcore_id()); + + if (cfg.nb_lcores == 1) { + lcore_id = rte_get_next_lcore(lcore_id, true, true); + rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop, + NULL, lcore_id); + } else if (cfg.nb_lcores > 1) { + lcore_id = rte_get_next_lcore(lcore_id, true, true); + rte_eal_remote_launch((lcore_function_t *)rx_main_loop, + NULL, lcore_id); + + lcore_id = rte_get_next_lcore(lcore_id, true, true); + rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL, + lcore_id); + } + } + +The function launches Rx/Tx processing functions on configured lcores +for each port using ``rte_eal_remote_launch()``. The configured ports, +their number and number of assigned lcores are stored in user-defined +``rxtx_transmission_config`` struct that is initialized before launching +tasks: + +.. code-block:: c + + struct rxtx_transmission_config { + struct rxtx_port_config ports[RTE_MAX_ETHPORTS]; + uint16_t nb_ports; + uint16_t nb_lcores; + }; + +The Lcores Processing Functions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For receiving packets on each port an ``ioat_rx_port()`` function is used. +The function receives packets on each configured Rx queue. Depending on mode +the user chose, it will enqueue packets to IOAT rawdev channels and then invoke +copy process (hardware copy), or perform software copy of each packet using +``pktmbuf_sw_copy()`` function and enqueue them to 1 rte_ring: + +.. code-block:: c + + /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ + static void + ioat_rx_port(struct rxtx_port_config *rx_config) + { + uint32_t nb_rx, nb_enq, i, j; + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; + for (i = 0; i < rx_config->nb_queues; i++) { + + nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i, + pkts_burst, MAX_PKT_BURST); + + if (nb_rx == 0) + continue; + + port_statistics.rx[rx_config->rxtx_port] += nb_rx; + + if (copy_mode == COPY_MODE_IOAT_NUM) { + /* Perform packet hardware copy */ + nb_enq = ioat_enqueue_packets(pkts_burst, + nb_rx, rx_config->ioat_ids[i]); + if (nb_enq > 0) + rte_ioat_do_copies(rx_config->ioat_ids[i]); + } else { + /* Perform packet software copy, free source packets */ + int ret; + struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST]; + + ret = rte_mempool_get_bulk(ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + + if (unlikely(ret < 0)) + rte_exit(EXIT_FAILURE, + "Unable to allocate memory.\n"); + + for (j = 0; j < nb_rx; j++) + pktmbuf_sw_copy(pkts_burst[j], + pkts_burst_copy[j]); + + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq = rte_ring_enqueue_burst( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx, NULL); + + /* Free any not enqueued packets. */ + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } + + port_statistics.copy_dropped[rx_config->rxtx_port] += + (nb_rx - nb_enq); + } + } + +The packets are received in burst mode using ``rte_eth_rx_burst()`` +function. When using hardware copy mode the packets are enqueued in +copying device's buffer using ``ioat_enqueue_packets()`` which calls +``rte_ioat_enqueue_copy()``. When all received packets are in the +buffer the copies are invoked by calling ``rte_ioat_do_copies()``. +Function ``rte_ioat_enqueue_copy()`` operates on physical address of +the packet. Structure ``rte_mbuf`` contains only physical address to +start of the data buffer (``buf_iova``). Thus the address is shifted +by ``addr_offset`` value in order to get pointer to ``rearm_data`` +member of ``rte_mbuf``. That way the packet is copied all at once +(with data and metadata). + +.. code-block:: c + + static uint32_t + ioat_enqueue_packets(struct rte_mbuf **pkts, + uint32_t nb_rx, uint16_t dev_id) + { + int ret; + uint32_t i; + struct rte_mbuf *pkts_copy[MAX_PKT_BURST]; + + const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr, + &pkts[0]->rearm_data); + + ret = rte_mempool_get_bulk(ioat_pktmbuf_pool, + (void *)pkts_copy, nb_rx); + + if (unlikely(ret < 0)) + rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n"); + + for (i = 0; i < nb_rx; i++) { + /* Perform data copy */ + ret = rte_ioat_enqueue_copy(dev_id, + pkts[i]->buf_iova + - addr_offset, + pkts_copy[i]->buf_iova + - addr_offset, + rte_pktmbuf_data_len(pkts[i]) + + addr_offset, + (uintptr_t)pkts[i], + (uintptr_t)pkts_copy[i], + 0 /* nofence */); + + if (ret != 1) + break; + } + + ret = i; + /* Free any not enqueued packets. */ + rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i); + rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i], + nb_rx - i); + + return ret; + } + + +All done copies are processed by ``ioat_tx_port()`` function. When using +hardware copy mode the function invokes ``rte_ioat_completed_copies()`` +on each assigned IOAT channel to gather copied packets. If software copy +mode is used the function dequeues copied packets from the rte_ring. Then each +packet MAC address is changed if it was enabled. After that copies are sent +in burst mode using `` rte_eth_tx_burst()``. + + +.. code-block:: c + + /* Transmit packets from IOAT rawdev/rte_ring for one port. */ + static void + ioat_tx_port(struct rxtx_port_config *tx_config) + { + uint32_t i, j, nb_dq = 0; + struct rte_mbuf *mbufs_src[MAX_PKT_BURST]; + struct rte_mbuf *mbufs_dst[MAX_PKT_BURST]; + + if (copy_mode == COPY_MODE_IOAT_NUM) { + for (i = 0; i < tx_config->nb_queues; i++) { + /* Deque the mbufs from IOAT device. */ + nb_dq = rte_ioat_completed_copies( + tx_config->ioat_ids[i], MAX_PKT_BURST, + (void *)mbufs_src, (void *)mbufs_dst); + + if (nb_dq == 0) + break; + + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)mbufs_src, nb_dq); + + /* Update macs if enabled */ + if (mac_updating) { + for (j = 0; j < nb_dq; j++) + update_mac_addrs(mbufs_dst[j], + tx_config->rxtx_port); + } + + const uint16_t nb_tx = rte_eth_tx_burst( + tx_config->rxtx_port, 0, + (void *)mbufs_dst, nb_dq); + + port_statistics.tx[tx_config->rxtx_port] += nb_tx; + + /* Free any unsent packets. */ + if (unlikely(nb_tx < nb_dq)) + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } + } + else { + for (i = 0; i < tx_config->nb_queues; i++) { + /* Deque the mbufs from IOAT device. */ + nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring, + (void *)mbufs_dst, MAX_PKT_BURST, NULL); + + if (nb_dq == 0) + return; + + /* Update macs if enabled */ + if (mac_updating) { + for (j = 0; j < nb_dq; j++) + update_mac_addrs(mbufs_dst[j], + tx_config->rxtx_port); + } + + const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port, + 0, (void *)mbufs_dst, nb_dq); + + port_statistics.tx[tx_config->rxtx_port] += nb_tx; + + /* Free any unsent packets. */ + if (unlikely(nb_tx < nb_dq)) + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } + } + } + +The Packet Copying Functions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In order to perform packet copy there is a user-defined function +``pktmbuf_sw_copy()`` used. It copies a whole packet by copying +metadata from source packet to new mbuf, and then copying a data +chunk of source packet. Both memory copies are done using +``rte_memcpy()``: + +.. code-block:: c + + static inline void + pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) + { + /* Copy packet metadata */ + rte_memcpy(&dst->rearm_data, + &src->rearm_data, + offsetof(struct rte_mbuf, cacheline1) + - offsetof(struct rte_mbuf, rearm_data)); + + /* Copy packet data */ + rte_memcpy(rte_pktmbuf_mtod(dst, char *), + rte_pktmbuf_mtod(src, char *), src->data_len); + } + +The metadata in this example is copied from ``rearm_data`` member of +``rte_mbuf`` struct up to ``cacheline1``. + +In order to understand why software packet copying is done as shown +above please refer to the "Mbuf Library" section of the +*DPDK Programmer's Guide*.Signature · 1 line
2.22.0.windows.1
Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
In reply to Marcin BaranQuoted 75 lines · Marcin Baran
Added guide for IOAT sample app usage and
code description.Signed-off-by Marcin Baran <[email protected]>--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++++++++++ 3 files changed, 769 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rstdoc/guides/sample_app_ug/index.rst +1
-0@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtualdoc/guides/sample_app_ug/intro.rst +4
-0@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.doc/guides/sample_app_ug/ioat.rst new file +31
-0@@ -0,0 +1,764 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Intel Corporation. + +Sample Application of packet copying using Intel\ |reg| QuickData Technology +============================================================================ + +Overview +-------- + +This sample is intended as a demonstration of the basic components of a DPDK +forwarding application and example of how to use IOAT driver API to make +packets copies. + +Also while forwarding, the MAC addresses are affected as follows: + +* The source MAC address is replaced by the TX port MAC address + +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID + +This application can be used to compare performance of using software packet +copy with copy done using a DMA device for different sizes of packets. +The example will print out statistics each second. The stats shows +received/send packets and packets dropped or failed to copy. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``ioat`` sub-directory. +
doc/guides/sample_app_ug/ioat.rst line 41
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ +Running the Application +----------------------- + +In order to run the hardware copy application, the copying device +needs to be bound to user-space IO driver. + +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData Technology* +guide for information on using the driver. +
The document is not called that, as the IOAT guide is just part of the overall rawdev document. So I suggest you just reference the rawdev guide.
doc/guides/sample_app_ug/ioat.rst line 51
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+The application requires a number of command line options: + +.. code-block:: console + + ./build/ioatfwd [EAL options] -- -p MASK [-q NQ] [-s RS] [-c <sw|hw>] + [--[no-]mac-updating] + +where, + +* p MASK: A hexadecimal bitmask of the ports to configure
Is this a mandatory parameter, or does the app use all detected ports by default, e.g. like testpmd?
doc/guides/sample_app_ug/ioat.rst line 57
Quoted 6 lines
doc/guides/sample_app_ug/ioat.rst new file +6
-0+ +* q NQ: Number of Rx queues used per port equivalent to CBDMA channels + per port + +* c CT: Performed packet copy type: software (sw) or hardware using + DMA (hw)
What is the default? Same for next two parameters.
doc/guides/sample_app_ug/ioat.rst line 66
Quoted 9 lines
doc/guides/sample_app_ug/ioat.rst new file +9
-0+ +* s RS: Size of IOAT rawdev ring for hardware copy mode or rte_ring for + software copy mode + +* --[no-]mac-updating: Whether MAC address of packets should be changed + or not + +The application can be launched in various configurations depending on +provided parameters. Each port can use up to 2 lcores: one of them receives
The app uses 2 data plane cores, total, rather than 2 per-port, I believe. It would be good to explain the difference here that with 2 cores the copies are done on one core, and the mac updates on the second one.
doc/guides/sample_app_ug/ioat.rst line 70
Quoted 4 lines
doc/guides/sample_app_ug/ioat.rst new file +4
-0+incoming traffic and makes a copy of each packet. The second lcore then +updates MAC address and sends the copy. If one lcore per port is used, +both operations are done sequentially. For each configuration an additional +lcore is needed since master lcore in use which is responsible for
... since the master lcore does not handle traffic but is responsible for
doc/guides/sample_app_ug/ioat.rst line 72
Quoted 2 lines
doc/guides/sample_app_ug/ioat.rst new file +2
-0+configuration, statistics printing and safe deinitialization of all ports +and devices.
s/deinitialization/shutdown/
doc/guides/sample_app_ug/ioat.rst line 74
Quoted 2 lines
doc/guides/sample_app_ug/ioat.rst new file +2
-0+ +The application can use a maximum of 8 ports.
Is this a hard limit in the app, if so explain why. I see the stats arrays are limited by "RTE_MAX_ETHPORTS".
doc/guides/sample_app_ug/ioat.rst line 78
Quoted 4 lines
doc/guides/sample_app_ug/ioat.rst new file +4
-0+ +To run the application in a Linux environment with 3 lcores (one of them +is master lcore), 1 port (port 0), software copying and MAC updating issue +the command:
s/1 port/a single port/
s/one of them is master lcore/the master lcore, plus two forwarding cores/
Similar comments would apply to text immediately below too.
Quoted 187 lines
doc/guides/sample_app_ug/ioat.rst new file +187
-0+ +.. code-block:: console + + $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw + +To run the application in a Linux environment with 2 lcores (one of them +is master lcore), 2 ports (ports 0 and 1), hardware copying and no MAC +updating issue the command: + +.. code-block:: console + + $ ./build/ioatfwd -l 0-1 -n 1 -- -p 0x3 --no-mac-updating -c hw + +Refer to the *DPDK Getting Started Guide* for general information on +running applications and the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +The following sections provide an explanation of the main components of the +code. + +All DPDK library functions used in the sample code are prefixed with +``rte_`` and are explained in detail in the *DPDK API Documentation*. + + +The Main Function +~~~~~~~~~~~~~~~~~ + +The ``main()`` function performs the initialization and calls the execution +threads for each lcore. + +The first task is to initialize the Environment Abstraction Layer (EAL). +The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()`` +function. The value returned is the number of parsed arguments: + +.. code-block:: c + + /* init EAL */ + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n"); + + +The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers) +used by the application: + +.. code-block:: c + + nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd + + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE), + MIN_POOL_SIZE); + + /* Create the mbuf pool */ + ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs, + MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE, + rte_socket_id()); + if (ioat_pktmbuf_pool == NULL) + rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n"); + +Mbufs are the packet buffer structure used by DPDK. They are explained in +detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*. + +The ``main()`` function also initializes the ports: + +.. code-block:: c + + /* Initialise each port */ + RTE_ETH_FOREACH_DEV(portid) { + port_init(portid, ioat_pktmbuf_pool); + } + +Each port is configured using ``port_init()``: + +.. code-block:: c + + /* + * Initializes a given port using global settings and with the RX buffers + * coming from the mbuf_pool passed as a parameter. + */ + static inline void + port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues) + { + /* configuring port to use RSS for multiple RX queues */ + static const struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = RTE_ETHER_MAX_LEN + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_PROTO_MASK, + } + } + }; + + struct rte_eth_rxconf rxq_conf; + struct rte_eth_txconf txq_conf; + struct rte_eth_conf local_port_conf = port_conf; + struct rte_eth_dev_info dev_info; + int ret, i; + + /* Skip ports that are not enabled */ + if ((ioat_enabled_port_mask & (1 << portid)) == 0) { + printf("Skipping disabled port %u\n", portid); + return; + } + + /* Init port */ + printf("Initializing port %u... ", portid); + fflush(stdout); + rte_eth_dev_info_get(portid, &dev_info); + local_port_conf.rx_adv_conf.rss_conf.rss_hf &= + dev_info.flow_type_rss_offloads; + if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE) + local_port_conf.txmode.offloads |= + DEV_TX_OFFLOAD_MBUF_FAST_FREE; + ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Cannot configure device:" + " err=%d, port=%u\n", ret, portid); + + ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd, + &nb_txd); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot adjust number of descriptors: err=%d, port=%u\n", + ret, portid); + + rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]); + + /* Init Rx queues */ + rxq_conf = dev_info.default_rxconf; + rxq_conf.offloads = local_port_conf.rxmode.offloads; + for (i = 0; i < nb_queues; i++) { + ret = rte_eth_rx_queue_setup(portid, i, nb_rxd, + rte_eth_dev_socket_id(portid), &rxq_conf, + mbuf_pool); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n", + ret, portid, i); + } + + /* Init one TX queue on each port */ + txq_conf = dev_info.default_txconf; + txq_conf.offloads = local_port_conf.txmode.offloads; + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, + rte_eth_dev_socket_id(portid), + &txq_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_tx_queue_setup:err=%d,port=%u\n", + ret, portid); + + /* Initialize TX buffers */ + tx_buffer[portid] = rte_zmalloc_socket("tx_buffer", + RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0, + rte_eth_dev_socket_id(portid)); + if (tx_buffer[portid] == NULL) + rte_exit(EXIT_FAILURE, + "Cannot allocate buffer for tx on port %u\n", + portid); + + rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST); + + ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid], + rte_eth_tx_buffer_count_callback, + &port_statistics.tx_dropped[portid]); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot set error callback for tx buffer on port %u\n", + portid); + + /* Start device */ + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_dev_start:err=%d, port=%u\n", + ret, portid); + + rte_eth_promiscuous_enable(portid); + + printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n", + portid, + ioat_ports_eth_addr[portid].addr_bytes[0],
doc/guides/sample_app_ug/ioat.rst line 275
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ ioat_ports_eth_addr[portid].addr_bytes[1], + ioat_ports_eth_addr[portid].addr_bytes[2], + ioat_ports_eth_addr[portid].addr_bytes[3], + ioat_ports_eth_addr[portid].addr_bytes[4], + ioat_ports_eth_addr[portid].addr_bytes[5]); + + cfg.ports[cfg.nb_ports].rxtx_port = portid; + cfg.ports[cfg.nb_ports++].nb_queues = nb_queues; + } +
This code is probably quite similar to that in other sample apps, so I don't think we need to include the full function here. It makes updating the code more difficult, so just refer to the function as doing the port init and leave it at that, I think. The snippets below give enough detail.
Quoted 60 lines
doc/guides/sample_app_ug/ioat.rst new file +60
-0+The Ethernet ports are configured with local settings using the +``rte_eth_dev_configure()`` function and the ``port_conf`` struct. +The RSS is enabled so that multiple Rx queues could be used for +packet receiving and copying by multiple CBDMA channels per port: + +.. code-block:: c + + /* configuring port to use RSS for multiple RX queues */ + static const struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = RTE_ETHER_MAX_LEN + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_PROTO_MASK, + } + } + }; + +For this example the ports are set up with the number of Rx queues provided +with -q option and 1 Tx queue using the ``rte_eth_rx_queue_setup()`` +and ``rte_eth_tx_queue_setup()`` functions. + +The Ethernet port is then started: + +.. code-block:: c + + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n", + ret, portid); + + +Finally the Rx port is set in promiscuous mode: + +.. code-block:: c + + rte_eth_promiscuous_enable(portid); + + +After that each port application assigns resources needed. + +.. code-block:: c + + check_link_status(ioat_enabled_port_mask); + + if (!cfg.nb_ports) { + rte_exit(EXIT_FAILURE, + "All available ports are disabled. Please set portmask.\n"); + } + + /* Check if there is enough lcores for all ports. */ + cfg.nb_lcores = rte_lcore_count() - 1; + if (cfg.nb_lcores < 1) + rte_exit(EXIT_FAILURE, + "There should be at least one slave lcore.\n"); + + ret = 0;
doc/guides/sample_app_ug/ioat.rst line 345
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ + if (copy_mode == COPY_MODE_IOAT_NUM) { + assign_rawdevs(); + } else /* copy_mode == COPY_MODE_SW_NUM */ { + assign_rings(); + } + +A link status is checked of each port enabled by port mask +using ``check_link_status()`` function. +
I don't think this block needs to be covered. No need to go into everything in detail, just focus on the key parts of the app that are unique to it, i.e. the copying and passing mbufs between threads parts.
Quoted 32 lines
doc/guides/sample_app_ug/ioat.rst new file +32
-0+.. code-block:: c + + /* check link status, return true if at least one port is up */ + static int + check_link_status(uint32_t port_mask) + { + uint16_t portid; + struct rte_eth_link link; + int retval = 0; + + printf("\nChecking link status\n"); + RTE_ETH_FOREACH_DEV(portid) { + if ((port_mask & (1 << portid)) == 0) + continue; + + memset(&link, 0, sizeof(link)); + rte_eth_link_get(portid, &link); + + /* Print link status */ + if (link.link_status) { + printf( + "Port %d Link Up. Speed %u Mbps - %s\n", + portid, link.link_speed, + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + ("full-duplex") : ("half-duplex\n")); + retval = 1; + } else + printf("Port %d Link Down\n", portid); + } + return retval; + } +
Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Friday, September 27, 2019 1:02 PM
To: Baran, MarcinX <[email protected]>
Cc: [email protected]
Subject: Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
Quoted 81 lines · Marcin Baran
Added guide for IOAT sample app usage and code description.
Signed-off-by Marcin Baran <[email protected]>--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++++++++++ 3 files changed, 769 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rstdoc/guides/sample_app_ug/index.rst +1
-0@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtualdoc/guides/sample_app_ug/intro.rst +5
-0@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev +driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.doc/guides/sample_app_ug/ioat.rst new file +33
-0@@ -0,0 +1,764 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Intel Corporation. + +Sample Application of packet copying using Intel\ |reg| QuickData +Technology +===================================================================== +======= + +Overview +-------- + +This sample is intended as a demonstration of the basic components of +a DPDK forwarding application and example of how to use IOAT driver +API to make packets copies. + +Also while forwarding, the MAC addresses are affected as follows: + +* The source MAC address is replaced by the TX port MAC address + +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID + +This application can be used to compare performance of using software +packet copy with copy done using a DMA device for different sizes of packets. +The example will print out statistics each second. The stats shows +received/send packets and packets dropped or failed to copy. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``ioat`` sub-directory. +
doc/guides/sample_app_ug/ioat.rst line 43
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ +Running the Application +----------------------- + +In order to run the hardware copy application, the copying device +needs to be bound to user-space IO driver. + +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData +Technology* guide for information on using the driver. +
The document is not called that, as the IOAT guide is just part of the overall rawdev document. So I suggest you just reference the rawdev guide. [Marcin] I wanted to refer to the ioat guide in /doc/guides/rawdevs/ioat.rst which has that title. Is there another document or I referenced this one incorrectly?
doc/guides/sample_app_ug/ioat.rst line 51
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+The application requires a number of command line options: + +.. code-block:: console + + ./build/ioatfwd [EAL options] -- -p MASK [-q NQ] [-s RS] [-c <sw|hw>] + [--[no-]mac-updating] + +where, + +* p MASK: A hexadecimal bitmask of the ports to configure
Is this a mandatory parameter, or does the app use all detected ports by default, e.g. like testpmd? [Marcin] Optional, the app use all detected ports, added default value comment and tagged -p as optional.
doc/guides/sample_app_ug/ioat.rst line 57
Quoted 6 lines
doc/guides/sample_app_ug/ioat.rst new file +6
-0+ +* q NQ: Number of Rx queues used per port equivalent to CBDMA channels + per port + +* c CT: Performed packet copy type: software (sw) or hardware using + DMA (hw)
What is the default? Same for next two parameters. [Marcin] Added default values description.
doc/guides/sample_app_ug/ioat.rst
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ +* s RS: Size of IOAT rawdev ring for hardware copy mode or rte_ring for + software copy mode + +* --[no-]mac-updating: Whether MAC address of packets should be changed + or not + +The application can be launched in various configurations depending +on provided parameters. Each port can use up to 2 lcores: one of them +receives
The app uses 2 data plane cores, total, rather than 2 per-port, I believe.
It would be good to explain the difference here that with 2 cores the copies are done on one core, and the mac updates on the second one. [Marcin] Changed the description accordingly.
doc/guides/sample_app_ug/ioat.rst line 74
Quoted 5 lines
doc/guides/sample_app_ug/ioat.rst new file +5
-0+incoming traffic and makes a copy of each packet. The second lcore +then updates MAC address and sends the copy. If one lcore per port is +used, both operations are done sequentially. For each configuration +an additional lcore is needed since master lcore in use which is +responsible for
... since the master lcore does not handle traffic but is responsible for [Marcin] Changed the description accordingly.
doc/guides/sample_app_ug/ioat.rst line 76
Quoted 2 lines
doc/guides/sample_app_ug/ioat.rst new file +2
-0+configuration, statistics printing and safe deinitialization of all +ports and devices.
s/deinitialization/shutdown/
[Marcin] Changed the description accordingly.
doc/guides/sample_app_ug/ioat.rst line 78
Quoted 2 lines
doc/guides/sample_app_ug/ioat.rst new file +2
-0+ +The application can use a maximum of 8 ports.
Is this a hard limit in the app, if so explain why. I see the stats arrays are limited by "RTE_MAX_ETHPORTS". [Marcin] The limit was set for simplicity but also because on testing board there was 16 CBDMA channels total so when there are 8 ports used, they can be set to work with more than one Rx queue each.
doc/guides/sample_app_ug/ioat.rst line 86
Quoted 4 lines
doc/guides/sample_app_ug/ioat.rst new file +4
-0+ +To run the application in a Linux environment with 3 lcores (one of +them is master lcore), 1 port (port 0), software copying and MAC +updating issue the command:
s/1 port/a single port/
s/one of them is master lcore/the master lcore, plus two forwarding cores/
Similar comments would apply to text immediately below too. [Marcin] Changed the description accordingly.
Quoted 187 lines
doc/guides/sample_app_ug/ioat.rst new file +187
-0+ +.. code-block:: console + + $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw + +To run the application in a Linux environment with 2 lcores (one of +them is master lcore), 2 ports (ports 0 and 1), hardware copying and +no MAC updating issue the command: + +.. code-block:: console + + $ ./build/ioatfwd -l 0-1 -n 1 -- -p 0x3 --no-mac-updating -c hw + +Refer to the *DPDK Getting Started Guide* for general information on +running applications and the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +The following sections provide an explanation of the main components +of the code. + +All DPDK library functions used in the sample code are prefixed with +``rte_`` and are explained in detail in the *DPDK API Documentation*. + + +The Main Function +~~~~~~~~~~~~~~~~~ + +The ``main()`` function performs the initialization and calls the +execution threads for each lcore. + +The first task is to initialize the Environment Abstraction Layer (EAL). +The ``argc`` and ``argv`` arguments are provided to the +``rte_eal_init()`` function. The value returned is the number of parsed arguments: + +.. code-block:: c + + /* init EAL */ + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n"); + + +The ``main()`` also allocates a mempool to hold the mbufs (Message +Buffers) used by the application: + +.. code-block:: c + + nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd + + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE), + MIN_POOL_SIZE); + + /* Create the mbuf pool */ + ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs, + MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE, + rte_socket_id()); + if (ioat_pktmbuf_pool == NULL) + rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n"); + +Mbufs are the packet buffer structure used by DPDK. They are +explained in detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*. + +The ``main()`` function also initializes the ports: + +.. code-block:: c + + /* Initialise each port */ + RTE_ETH_FOREACH_DEV(portid) { + port_init(portid, ioat_pktmbuf_pool); + } + +Each port is configured using ``port_init()``: + +.. code-block:: c + + /* + * Initializes a given port using global settings and with the RX buffers + * coming from the mbuf_pool passed as a parameter. + */ + static inline void + port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues) + { + /* configuring port to use RSS for multiple RX queues */ + static const struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = RTE_ETHER_MAX_LEN + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_PROTO_MASK, + } + } + }; + + struct rte_eth_rxconf rxq_conf; + struct rte_eth_txconf txq_conf; + struct rte_eth_conf local_port_conf = port_conf; + struct rte_eth_dev_info dev_info; + int ret, i; + + /* Skip ports that are not enabled */ + if ((ioat_enabled_port_mask & (1 << portid)) == 0) { + printf("Skipping disabled port %u\n", portid); + return; + } + + /* Init port */ + printf("Initializing port %u... ", portid); + fflush(stdout); + rte_eth_dev_info_get(portid, &dev_info); + local_port_conf.rx_adv_conf.rss_conf.rss_hf &= + dev_info.flow_type_rss_offloads; + if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE) + local_port_conf.txmode.offloads |= + DEV_TX_OFFLOAD_MBUF_FAST_FREE; + ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Cannot configure device:" + " err=%d, port=%u\n", ret, portid); + + ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd, + &nb_txd); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot adjust number of descriptors: err=%d, port=%u\n", + ret, portid); + + rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]); + + /* Init Rx queues */ + rxq_conf = dev_info.default_rxconf; + rxq_conf.offloads = local_port_conf.rxmode.offloads; + for (i = 0; i < nb_queues; i++) { + ret = rte_eth_rx_queue_setup(portid, i, nb_rxd, + rte_eth_dev_socket_id(portid), &rxq_conf, + mbuf_pool); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n", + ret, portid, i); + } + + /* Init one TX queue on each port */ + txq_conf = dev_info.default_txconf; + txq_conf.offloads = local_port_conf.txmode.offloads; + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, + rte_eth_dev_socket_id(portid), + &txq_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_tx_queue_setup:err=%d,port=%u\n", + ret, portid); + + /* Initialize TX buffers */ + tx_buffer[portid] = rte_zmalloc_socket("tx_buffer", + RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0, + rte_eth_dev_socket_id(portid)); + if (tx_buffer[portid] == NULL) + rte_exit(EXIT_FAILURE, + "Cannot allocate buffer for tx on port %u\n", + portid); + + rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST); + + ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid], + rte_eth_tx_buffer_count_callback, + &port_statistics.tx_dropped[portid]); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot set error callback for tx buffer on port %u\n", + portid); + + /* Start device */ + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_dev_start:err=%d, port=%u\n", + ret, portid); + + rte_eth_promiscuous_enable(portid); + + printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n", + portid, + ioat_ports_eth_addr[portid].addr_bytes[0],
doc/guides/sample_app_ug/ioat.rst line 275
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ ioat_ports_eth_addr[portid].addr_bytes[1], + ioat_ports_eth_addr[portid].addr_bytes[2], + ioat_ports_eth_addr[portid].addr_bytes[3], + ioat_ports_eth_addr[portid].addr_bytes[4], + ioat_ports_eth_addr[portid].addr_bytes[5]); + + cfg.ports[cfg.nb_ports].rxtx_port = portid; + cfg.ports[cfg.nb_ports++].nb_queues = nb_queues; + } +
This code is probably quite similar to that in other sample apps, so I don't think we need to include the full function here. It makes updating the code more difficult, so just refer to the function as doing the port init and leave it at that, I think. The snippets below give enough detail. [Marcin] Changed the description accordingly.
Quoted 60 lines
doc/guides/sample_app_ug/ioat.rst new file +60
-0+The Ethernet ports are configured with local settings using the +``rte_eth_dev_configure()`` function and the ``port_conf`` struct. +The RSS is enabled so that multiple Rx queues could be used for +packet receiving and copying by multiple CBDMA channels per port: + +.. code-block:: c + + /* configuring port to use RSS for multiple RX queues */ + static const struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = RTE_ETHER_MAX_LEN + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_PROTO_MASK, + } + } + }; + +For this example the ports are set up with the number of Rx queues +provided with -q option and 1 Tx queue using the +``rte_eth_rx_queue_setup()`` and ``rte_eth_tx_queue_setup()`` functions. + +The Ethernet port is then started: + +.. code-block:: c + + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n", + ret, portid); + + +Finally the Rx port is set in promiscuous mode: + +.. code-block:: c + + rte_eth_promiscuous_enable(portid); + + +After that each port application assigns resources needed. + +.. code-block:: c + + check_link_status(ioat_enabled_port_mask); + + if (!cfg.nb_ports) { + rte_exit(EXIT_FAILURE, + "All available ports are disabled. Please set portmask.\n"); + } + + /* Check if there is enough lcores for all ports. */ + cfg.nb_lcores = rte_lcore_count() - 1; + if (cfg.nb_lcores < 1) + rte_exit(EXIT_FAILURE, + "There should be at least one slave lcore.\n"); + + ret = 0;
doc/guides/sample_app_ug/ioat.rst line 345
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ + if (copy_mode == COPY_MODE_IOAT_NUM) { + assign_rawdevs(); + } else /* copy_mode == COPY_MODE_SW_NUM */ { + assign_rings(); + } + +A link status is checked of each port enabled by port mask using +``check_link_status()`` function. +
I don't think this block needs to be covered. No need to go into everything in detail, just focus on the key parts of the app that are unique to it, i.e. the copying and passing mbufs between threads parts. [Marcin] Changed the description accordingly.
Quoted 32 lines
doc/guides/sample_app_ug/ioat.rst new file +32
-0+.. code-block:: c + + /* check link status, return true if at least one port is up */ + static int + check_link_status(uint32_t port_mask) + { + uint16_t portid; + struct rte_eth_link link; + int retval = 0; + + printf("\nChecking link status\n"); + RTE_ETH_FOREACH_DEV(portid) { + if ((port_mask & (1 << portid)) == 0) + continue; + + memset(&link, 0, sizeof(link)); + rte_eth_link_get(portid, &link); + + /* Print link status */ + if (link.link_status) { + printf( + "Port %d Link Up. Speed %u Mbps - %s\n", + portid, link.link_speed, + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + ("full-duplex") : ("half-duplex\n")); + retval = 1; + } else + printf("Port %d Link Down\n", portid); + } + return retval; + } +
Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
Quoted 95 lines · MarcinX
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Friday, September 27, 2019 1:02 PM
To: Baran, MarcinX <[email protected]>
Cc: [email protected]
Subject: Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guideQuoted 87 lines · Marcin Baran
Added guide for IOAT sample app usage and code description.
Signed-off-by Marcin Baran <[email protected]>--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++++++++++ 3 files changed, 769 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rstdoc/guides/sample_app_ug/index.rst +1
-0@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtualdoc/guides/sample_app_ug/intro.rst +5
-0@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev +driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.doc/guides/sample_app_ug/ioat.rst new file +39
-0@@ -0,0 +1,764 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Intel Corporation. + +Sample Application of packet copying using Intel\ |reg| QuickData +Technology +===================================================================== +======= + +Overview +-------- + +This sample is intended as a demonstration of the basic components of +a DPDK forwarding application and example of how to use IOAT driver +API to make packets copies. + +Also while forwarding, the MAC addresses are affected as follows: + +* The source MAC address is replaced by the TX port MAC address + +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID + +This application can be used to compare performance of using software +packet copy with copy done using a DMA device for different sizes of packets. +The example will print out statistics each second. The stats shows +received/send packets and packets dropped or failed to copy. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``ioat`` sub-directory. + + +Running the Application +----------------------- + +In order to run the hardware copy application, the copying device +needs to be bound to user-space IO driver.
Quoted 9 lines
doc/guides/sample_app_ug/ioat.rst line 6
Quoted 4 lines
doc/guides/sample_app_ug/ioat.rst new file +4
-0+ +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData +Technology* guide for information on using the driver. +The document is not called that, as the IOAT guide is just part of the overall rawdev document. So I suggest you just reference the rawdev guide. [Marcin] I wanted to refer to the ioat guide in /doc/guides/rawdevs/ioat.rst which has that title. Is there another document or I referenced this one incorrectly?
It's one chapter in the overall "Rawdev Drivers" document:
https://doc.dpdk.org/guides/rawdevs/index.html
Since the (R) symbol doesn't seem to show up correctly in what you have above (though it looks correct to me), I suggest just referring to the "IOAT Rawdev Driver" chapter in the "Rawdev Drivers" document.
Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Friday, September 27, 2019 5:01 PM
To: Baran, MarcinX <[email protected]>
Cc: [email protected]
Subject: Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
Quoted 98 lines · MarcinX
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Friday, September 27, 2019 1:02 PM
To: Baran, MarcinX <[email protected]>
Cc: [email protected]
Subject: Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guideQuoted 89 lines · Marcin Baran
Added guide for IOAT sample app usage and code description.
Signed-off-by Marcin Baran <[email protected]>--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++++++++++ 3 files changed, 769 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rstdoc/guides/sample_app_ug/index.rst +1
-0@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtualdoc/guides/sample_app_ug/intro.rst +6
-0@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet +copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev +driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.doc/guides/sample_app_ug/ioat.rst new file +40
-0@@ -0,0 +1,764 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Intel Corporation. + +Sample Application of packet copying using Intel\ |reg| QuickData +Technology +=================================================================== +== +======= + +Overview +-------- + +This sample is intended as a demonstration of the basic components +of a DPDK forwarding application and example of how to use IOAT +driver API to make packets copies. + +Also while forwarding, the MAC addresses are affected as follows: + +* The source MAC address is replaced by the TX port MAC address + +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID + +This application can be used to compare performance of using +software packet copy with copy done using a DMA device for different sizes of packets. +The example will print out statistics each second. The stats shows +received/send packets and packets dropped or failed to copy. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``ioat`` sub-directory. + + +Running the Application +----------------------- + +In order to run the hardware copy application, the copying device +needs to be bound to user-space IO driver.
Quoted 9 lines
doc/guides/sample_app_ug/ioat.rst line 6
Quoted 4 lines
doc/guides/sample_app_ug/ioat.rst new file +4
-0+ +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData +Technology* guide for information on using the driver. +The document is not called that, as the IOAT guide is just part of the overall rawdev document. So I suggest you just reference the rawdev guide. [Marcin] I wanted to refer to the ioat guide in
/doc/guides/rawdevs/ioat.rst which has that title. Is there another document or I referenced this one incorrectly?
It's one chapter in the overall "Rawdev Drivers" document:
https://doc.dpdk.org/guides/rawdevs/index.html
Since the (R) symbol doesn't seem to show up correctly in what you have above (though it looks correct to me), I suggest just referring to the "IOAT Rawdev Driver" chapter in the "Rawdev Drivers" document. [Marcin] Ok, I will change it like that.
Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
In reply to Marcin BaranQuoted 11 lines · Marcin Baran
Added guide for IOAT sample app usage and
code description.Signed-off-by Marcin Baran <[email protected]>--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++++++++++ 3 files changed, 769 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst
<snip>
Quoted 57 lines
doc/guides/sample_app_ug/ioat.rst new file +57
-0+Depending on mode set (whether copy should be done by software or by hardware) +special structures are assigned to each port. If software copy was chosen, +application have to assign ring structures for packet exchanging between lcores +assigned to ports. + +.. code-block:: c + + static void + assign_rings(void) + { + uint32_t i; + + for (i = 0; i < cfg.nb_ports; i++) { + char ring_name[20]; + + snprintf(ring_name, 20, "rx_to_tx_ring_%u", i); + /* Create ring for inter core communication */ + cfg.ports[i].rx_to_tx_ring = rte_ring_create( + ring_name, ring_size, + rte_socket_id(), RING_F_SP_ENQ); + + if (cfg.ports[i].rx_to_tx_ring == NULL) + rte_exit(EXIT_FAILURE, "%s\n", + rte_strerror(rte_errno)); + } + } + + +When using hardware copy each Rx queue of the port is assigned an +IOAT device (``assign_rawdevs()``) using IOAT Rawdev Driver API +functions: + +.. code-block:: c + + static void + assign_rawdevs(void) + { + uint16_t nb_rawdev = 0, rdev_id = 0; + uint32_t i, j; + + for (i = 0; i < cfg.nb_ports; i++) { + for (j = 0; j < cfg.ports[i].nb_queues; j++) { + struct rte_rawdev_info rdev_info = { 0 }; + + do { + if (rdev_id == rte_rawdev_count()) + goto end; + rte_rawdev_info_get(rdev_id++, &rdev_info); + } while (strcmp(rdev_info.driver_name, + IOAT_PMD_RAWDEV_NAME_STR) != 0); + + cfg.ports[i].ioat_ids[j] = rdev_id - 1; + configure_rawdev_queue(cfg.ports[i].ioat_ids[j]); + ++nb_rawdev; + } + } + end:
doc/guides/sample_app_ug/ioat.rst
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues) + rte_exit(EXIT_FAILURE, + "Not enough IOAT rawdevs (%u) for all queues (%u).\n", + nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues); + RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev); + } + + +The initialization of hardware device is done by ``rte_rawdev_configure()`` +function and ``rte_rawdev_info`` struct.
... using ``rte_rawdev_info`` struct
doc/guides/sample_app_ug/ioat.rst line 446
Quoted 3 lines
After configuration the device is
doc/guides/sample_app_ug/ioat.rst new file +2
-0+started using ``rte_rawdev_start()`` function. Each of the above operations +is done in ``configure_rawdev_queue()``.
In the block below, there is no mention of where dev_config structure comes from. Presume it's a global variable, so maybe mention that in the text.
Quoted 19 lines
doc/guides/sample_app_ug/ioat.rst new file +19
-0+ +.. code-block:: c + + static void + configure_rawdev_queue(uint32_t dev_id) + { + struct rte_rawdev_info info = { .dev_private = &dev_config }; + + /* Configure hardware copy device */ + dev_config.ring_size = ring_size; + + if (rte_rawdev_configure(dev_id, &info) != 0) { + rte_exit(EXIT_FAILURE, + "Error with rte_rawdev_configure()\n"); + } + rte_rawdev_info_get(dev_id, &info); + if (dev_config.ring_size != ring_size) { + rte_exit(EXIT_FAILURE, + "Error, ring size is not %d (%d)\n",
doc/guides/sample_app_ug/ioat.rst line 475
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ ring_size, (int)dev_config.ring_size); + } + if (rte_rawdev_start(dev_id) != 0) { + rte_exit(EXIT_FAILURE, + "Error with rte_rawdev_start()\n"); + } + } + +If initialization is successful memory for hardware device +statistics is allocated.
Missing "," after successful.
Where is this memory allocated? It is done in main or elsewhere?
doc/guides/sample_app_ug/ioat.rst line 477
Quoted 2 lines
doc/guides/sample_app_ug/ioat.rst new file +2
-0+ +Finally ``main()`` functions starts all processing lcores and starts
s/functions/function/
s/processing lcores/packet handling lcores/
Quoted 1 line
+printing stats in a loop on master lcore. The application can be
s/master lcore/the master lcore/
doc/guides/sample_app_ug/ioat.rst line 487
Quoted 9 lines
doc/guides/sample_app_ug/ioat.rst new file +9
-0+interrupted and closed using ``Ctrl-C``. The master lcore waits for +all slave processes to finish, deallocates resources and exits. + +The processing lcores launching function are described below. + +The Lcores Launching Functions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As described above ``main()`` function invokes ``start_forwarding_cores()``
Missing "," after above.
Quoted 18 lines
doc/guides/sample_app_ug/ioat.rst new file +18
-0+function in order to start processing for each lcore: + +.. code-block:: c + + static void start_forwarding_cores(void) + { + uint32_t lcore_id = rte_lcore_id(); + + RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n", + __func__, rte_lcore_id()); + + if (cfg.nb_lcores == 1) { + lcore_id = rte_get_next_lcore(lcore_id, true, true); + rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop, + NULL, lcore_id); + } else if (cfg.nb_lcores > 1) { + lcore_id = rte_get_next_lcore(lcore_id, true, true); + rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
doc/guides/sample_app_ug/ioat.rst line 515
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ NULL, lcore_id); + + lcore_id = rte_get_next_lcore(lcore_id, true, true); + rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL, + lcore_id); + } + } + +The function launches Rx/Tx processing functions on configured lcores +for each port using ``rte_eal_remote_launch()``. The configured ports,
Remove "for each port"
Quoted 2 lines
+their number and number of assigned lcores are stored in user-defined +``rxtx_transmission_config`` struct that is initialized before launching
s/is/has been/
Did you describe how that structure was set up previously?
Quoted 4 lines
doc/guides/sample_app_ug/ioat.rst new file +4
-0+tasks: + +.. code-block:: c +
doc/guides/sample_app_ug/ioat.rst line 531
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ struct rxtx_transmission_config { + struct rxtx_port_config ports[RTE_MAX_ETHPORTS]; + uint16_t nb_ports; + uint16_t nb_lcores; + }; + +The Lcores Processing Functions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For receiving packets on each port an ``ioat_rx_port()`` function is used.
Missing "," after port.
s/an/the/
Quoted 1 line
+The function receives packets on each configured Rx queue. Depending on mode
s/mode/the mode/
doc/guides/sample_app_ug/ioat.rst line 535
Quoted 3 lines
doc/guides/sample_app_ug/ioat.rst new file +3
-0+the user chose, it will enqueue packets to IOAT rawdev channels and then invoke +copy process (hardware copy), or perform software copy of each packet using +``pktmbuf_sw_copy()`` function and enqueue them to 1 rte_ring:
s/1 rte_ring/an rte_ring/
Quoted 54 lines
doc/guides/sample_app_ug/ioat.rst new file +54
-0+ +.. code-block:: c + + /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ + static void + ioat_rx_port(struct rxtx_port_config *rx_config) + { + uint32_t nb_rx, nb_enq, i, j; + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; + for (i = 0; i < rx_config->nb_queues; i++) { + + nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i, + pkts_burst, MAX_PKT_BURST); + + if (nb_rx == 0) + continue; + + port_statistics.rx[rx_config->rxtx_port] += nb_rx; + + if (copy_mode == COPY_MODE_IOAT_NUM) { + /* Perform packet hardware copy */ + nb_enq = ioat_enqueue_packets(pkts_burst, + nb_rx, rx_config->ioat_ids[i]); + if (nb_enq > 0) + rte_ioat_do_copies(rx_config->ioat_ids[i]); + } else { + /* Perform packet software copy, free source packets */ + int ret; + struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST]; + + ret = rte_mempool_get_bulk(ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + + if (unlikely(ret < 0)) + rte_exit(EXIT_FAILURE, + "Unable to allocate memory.\n"); + + for (j = 0; j < nb_rx; j++) + pktmbuf_sw_copy(pkts_burst[j], + pkts_burst_copy[j]); + + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq = rte_ring_enqueue_burst( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx, NULL); + + /* Free any not enqueued packets. */ + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } +
doc/guides/sample_app_ug/ioat.rst line 599
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ port_statistics.copy_dropped[rx_config->rxtx_port] += + (nb_rx - nb_enq); + } + } + +The packets are received in burst mode using ``rte_eth_rx_burst()`` +function. When using hardware copy mode the packets are enqueued in +copying device's buffer using ``ioat_enqueue_packets()`` which calls +``rte_ioat_enqueue_copy()``. When all received packets are in the +buffer the copies are invoked by calling ``rte_ioat_do_copies()``.
s/copies are invoked/copy operations are started/
doc/guides/sample_app_ug/ioat.rst line 602
Quoted 3 lines
doc/guides/sample_app_ug/ioat.rst new file +3
-0+Function ``rte_ioat_enqueue_copy()`` operates on physical address of +the packet. Structure ``rte_mbuf`` contains only physical address to +start of the data buffer (``buf_iova``). Thus the address is shifted
s/shifted/adjusted/
Quoted 1 line
+by ``addr_offset`` value in order to get pointer to ``rearm_data``
s/pointer to/the address of/
doc/guides/sample_app_ug/ioat.rst line 605
Quoted 2 lines
doc/guides/sample_app_ug/ioat.rst new file +2
-0+member of ``rte_mbuf``. That way the packet is copied all at once +(with data and metadata).
"That way the both the packet data and metadata can be copied in a single operation".
Should also note that this shortcut can be used because the mbufs are "direct" mbufs allocated by the apps. If another app uses external buffers, or indirect mbufs, then multiple copy operations must be used.
Quoted 38 lines
doc/guides/sample_app_ug/ioat.rst new file +38
-0+ +.. code-block:: c + + static uint32_t + ioat_enqueue_packets(struct rte_mbuf **pkts, + uint32_t nb_rx, uint16_t dev_id) + { + int ret; + uint32_t i; + struct rte_mbuf *pkts_copy[MAX_PKT_BURST]; + + const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr, + &pkts[0]->rearm_data); + + ret = rte_mempool_get_bulk(ioat_pktmbuf_pool, + (void *)pkts_copy, nb_rx); + + if (unlikely(ret < 0)) + rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n"); + + for (i = 0; i < nb_rx; i++) { + /* Perform data copy */ + ret = rte_ioat_enqueue_copy(dev_id, + pkts[i]->buf_iova + - addr_offset, + pkts_copy[i]->buf_iova + - addr_offset, + rte_pktmbuf_data_len(pkts[i]) + + addr_offset, + (uintptr_t)pkts[i], + (uintptr_t)pkts_copy[i], + 0 /* nofence */); + + if (ret != 1) + break; + } + + ret = i;
doc/guides/sample_app_ug/ioat.rst line 653
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ /* Free any not enqueued packets. */ + rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i); + rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i], + nb_rx - i); + + return ret; + } + + +All done copies are processed by ``ioat_tx_port()`` function. When using
s/done/completed/
Quoted 102 lines
doc/guides/sample_app_ug/ioat.rst new file +102
-0+hardware copy mode the function invokes ``rte_ioat_completed_copies()`` +on each assigned IOAT channel to gather copied packets. If software copy +mode is used the function dequeues copied packets from the rte_ring. Then each +packet MAC address is changed if it was enabled. After that copies are sent +in burst mode using `` rte_eth_tx_burst()``. + + +.. code-block:: c + + /* Transmit packets from IOAT rawdev/rte_ring for one port. */ + static void + ioat_tx_port(struct rxtx_port_config *tx_config) + { + uint32_t i, j, nb_dq = 0; + struct rte_mbuf *mbufs_src[MAX_PKT_BURST]; + struct rte_mbuf *mbufs_dst[MAX_PKT_BURST]; + + if (copy_mode == COPY_MODE_IOAT_NUM) { + for (i = 0; i < tx_config->nb_queues; i++) { + /* Deque the mbufs from IOAT device. */ + nb_dq = rte_ioat_completed_copies( + tx_config->ioat_ids[i], MAX_PKT_BURST, + (void *)mbufs_src, (void *)mbufs_dst); + + if (nb_dq == 0) + break; + + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)mbufs_src, nb_dq); + + /* Update macs if enabled */ + if (mac_updating) { + for (j = 0; j < nb_dq; j++) + update_mac_addrs(mbufs_dst[j], + tx_config->rxtx_port); + } + + const uint16_t nb_tx = rte_eth_tx_burst( + tx_config->rxtx_port, 0, + (void *)mbufs_dst, nb_dq); + + port_statistics.tx[tx_config->rxtx_port] += nb_tx; + + /* Free any unsent packets. */ + if (unlikely(nb_tx < nb_dq)) + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } + } + else { + for (i = 0; i < tx_config->nb_queues; i++) { + /* Deque the mbufs from IOAT device. */ + nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring, + (void *)mbufs_dst, MAX_PKT_BURST, NULL); + + if (nb_dq == 0) + return; + + /* Update macs if enabled */ + if (mac_updating) { + for (j = 0; j < nb_dq; j++) + update_mac_addrs(mbufs_dst[j], + tx_config->rxtx_port); + } + + const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port, + 0, (void *)mbufs_dst, nb_dq); + + port_statistics.tx[tx_config->rxtx_port] += nb_tx; + + /* Free any unsent packets. */ + if (unlikely(nb_tx < nb_dq)) + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } + } + } + +The Packet Copying Functions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In order to perform packet copy there is a user-defined function +``pktmbuf_sw_copy()`` used. It copies a whole packet by copying +metadata from source packet to new mbuf, and then copying a data +chunk of source packet. Both memory copies are done using +``rte_memcpy()``: + +.. code-block:: c + + static inline void + pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) + { + /* Copy packet metadata */ + rte_memcpy(&dst->rearm_data, + &src->rearm_data, + offsetof(struct rte_mbuf, cacheline1) + - offsetof(struct rte_mbuf, rearm_data)); + + /* Copy packet data */ + rte_memcpy(rte_pktmbuf_mtod(dst, char *),
doc/guides/sample_app_ug/ioat.rst line 764
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +9
-0+ rte_pktmbuf_mtod(src, char *), src->data_len); + } + +The metadata in this example is copied from ``rearm_data`` member of +``rte_mbuf`` struct up to ``cacheline1``. + +In order to understand why software packet copying is done as shown +above please refer to the "Mbuf Library" section of the +*DPDK Programmer's Guide*.
Use a text editor that adds a newline automatically :-)
/Bruce
Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Friday, September 27, 2019 3:23 PM
To: Baran, MarcinX <[email protected]>
Cc: [email protected]; Mcnamara, John <[email protected]>; Kovacevic, Marko <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
Quoted 11 lines · Marcin Baran
Added guide for IOAT sample app usage and code description.
Signed-off-by Marcin Baran <[email protected]>--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 764 +++++++++++++++++++++++++++++ 3 files changed, 769 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst
<snip>
Quoted 57 lines
doc/guides/sample_app_ug/ioat.rst new file +57
-0+Depending on mode set (whether copy should be done by software or by +hardware) special structures are assigned to each port. If software +copy was chosen, application have to assign ring structures for +packet exchanging between lcores assigned to ports. + +.. code-block:: c + + static void + assign_rings(void) + { + uint32_t i; + + for (i = 0; i < cfg.nb_ports; i++) { + char ring_name[20]; + + snprintf(ring_name, 20, "rx_to_tx_ring_%u", i); + /* Create ring for inter core communication */ + cfg.ports[i].rx_to_tx_ring = rte_ring_create( + ring_name, ring_size, + rte_socket_id(), RING_F_SP_ENQ); + + if (cfg.ports[i].rx_to_tx_ring == NULL) + rte_exit(EXIT_FAILURE, "%s\n", + rte_strerror(rte_errno)); + } + } + + +When using hardware copy each Rx queue of the port is assigned an +IOAT device (``assign_rawdevs()``) using IOAT Rawdev Driver API +functions: + +.. code-block:: c + + static void + assign_rawdevs(void) + { + uint16_t nb_rawdev = 0, rdev_id = 0; + uint32_t i, j; + + for (i = 0; i < cfg.nb_ports; i++) { + for (j = 0; j < cfg.ports[i].nb_queues; j++) { + struct rte_rawdev_info rdev_info = { 0 }; + + do { + if (rdev_id == rte_rawdev_count()) + goto end; + rte_rawdev_info_get(rdev_id++, &rdev_info); + } while (strcmp(rdev_info.driver_name, + IOAT_PMD_RAWDEV_NAME_STR) != 0); + + cfg.ports[i].ioat_ids[j] = rdev_id - 1; + configure_rawdev_queue(cfg.ports[i].ioat_ids[j]); + ++nb_rawdev; + } + } + end:
doc/guides/sample_app_ug/ioat.rst
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues) + rte_exit(EXIT_FAILURE, + "Not enough IOAT rawdevs (%u) for all queues (%u).\n", + nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues); + RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev); + } + + +The initialization of hardware device is done by +``rte_rawdev_configure()`` function and ``rte_rawdev_info`` struct.
... using ``rte_rawdev_info`` struct
[Marcin] Changed the description accordingly.
Review
Quoted 3 lines
After configuration the device is
(quoted diff) +2
-0+started using ``rte_rawdev_start()`` function. Each of the above +operations is done in ``configure_rawdev_queue()``.
In the block below, there is no mention of where dev_config structure comes from. Presume it's a global variable, so maybe mention that in the text. [Marcin] Actually this code snipped was not updated, it now uses local dev_config variable of type struct rte_ioat_rawdev_config, so I updated the snippet.
Quoted 19 lines
doc/guides/sample_app_ug/ioat.rst new file +19
-0+ +.. code-block:: c + + static void + configure_rawdev_queue(uint32_t dev_id) + { + struct rte_rawdev_info info = { .dev_private = &dev_config }; + + /* Configure hardware copy device */ + dev_config.ring_size = ring_size; + + if (rte_rawdev_configure(dev_id, &info) != 0) { + rte_exit(EXIT_FAILURE, + "Error with rte_rawdev_configure()\n"); + } + rte_rawdev_info_get(dev_id, &info); + if (dev_config.ring_size != ring_size) { + rte_exit(EXIT_FAILURE, + "Error, ring size is not %d (%d)\n",
doc/guides/sample_app_ug/ioat.rst
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ ring_size, (int)dev_config.ring_size); + } + if (rte_rawdev_start(dev_id) != 0) { + rte_exit(EXIT_FAILURE, + "Error with rte_rawdev_start()\n"); + } + } + +If initialization is successful memory for hardware device statistics +is allocated.
Missing "," after successful.
Where is this memory allocated? It is done in main or elsewhere? [Marcin] This comment is invalid now as code has been updated, I removed it.
doc/guides/sample_app_ug/ioat.rst line 477
Quoted 2 lines
doc/guides/sample_app_ug/ioat.rst new file +2
-0+ +Finally ``main()`` functions starts all processing lcores and starts
s/functions/function/
s/processing lcores/packet handling lcores/
[Marcin] Changed the description accordingly
Quoted 1 line
+printing stats in a loop on master lcore. The application can be
s/master lcore/the master lcore/
[Marcin] Changed the description accordingly
doc/guides/sample_app_ug/ioat.rst
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+interrupted and closed using ``Ctrl-C``. The master lcore waits for +all slave processes to finish, deallocates resources and exits. + +The processing lcores launching function are described below. + +The Lcores Launching Functions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As described above ``main()`` function invokes +``start_forwarding_cores()``
Missing "," after above.
[Marcin] Changed the description accordingly
Quoted 19 lines
doc/guides/sample_app_ug/ioat.rst new file +19
-0+function in order to start processing for each lcore: + +.. code-block:: c + + static void start_forwarding_cores(void) + { + uint32_t lcore_id = rte_lcore_id(); + + RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n", + __func__, rte_lcore_id()); + + if (cfg.nb_lcores == 1) { + lcore_id = rte_get_next_lcore(lcore_id, true, true); + rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop, + NULL, lcore_id); + } else if (cfg.nb_lcores > 1) { + lcore_id = rte_get_next_lcore(lcore_id, true, true); + rte_eal_remote_launch((lcore_function_t *)rx_main_loop, + NULL, lcore_id);
doc/guides/sample_app_ug/ioat.rst
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ + lcore_id = rte_get_next_lcore(lcore_id, true, true); + rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL, + lcore_id); + } + } + +The function launches Rx/Tx processing functions on configured lcores +for each port using ``rte_eal_remote_launch()``. The configured +ports,
Remove "for each port"
[Marcin] Removed
Review
Quoted 3 lines
(quoted diff) +3
-0+their number and number of assigned lcores are stored in user-defined +``rxtx_transmission_config`` struct that is initialized before +launching
s/is/has been/
Did you describe how that structure was set up previously? [Marcin] Added description as to how and when it is set up.
Quoted 4 lines
doc/guides/sample_app_ug/ioat.rst new file +4
-0+tasks: + +.. code-block:: c +
doc/guides/sample_app_ug/ioat.rst line 531
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ struct rxtx_transmission_config { + struct rxtx_port_config ports[RTE_MAX_ETHPORTS]; + uint16_t nb_ports; + uint16_t nb_lcores; + }; + +The Lcores Processing Functions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For receiving packets on each port an ``ioat_rx_port()`` function is used.
Missing "," after port.
s/an/the/
[Marcin] Changed the description accordingly
Review
Quoted 2 lines
(quoted diff) +2
-0+The function receives packets on each configured Rx queue. Depending +on mode
s/mode/the mode/
[Marcin] Changed the description accordingly
Review
Quoted 3 lines
(quoted diff) +3
-0+the user chose, it will enqueue packets to IOAT rawdev channels and +then invoke copy process (hardware copy), or perform software copy of +each packet using ``pktmbuf_sw_copy()`` function and enqueue them to 1 rte_ring:
s/1 rte_ring/an rte_ring/
[Marcin] Changed the description accordingly
Quoted 54 lines
doc/guides/sample_app_ug/ioat.rst new file +54
-0+ +.. code-block:: c + + /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ + static void + ioat_rx_port(struct rxtx_port_config *rx_config) + { + uint32_t nb_rx, nb_enq, i, j; + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; + for (i = 0; i < rx_config->nb_queues; i++) { + + nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i, + pkts_burst, MAX_PKT_BURST); + + if (nb_rx == 0) + continue; + + port_statistics.rx[rx_config->rxtx_port] += nb_rx; + + if (copy_mode == COPY_MODE_IOAT_NUM) { + /* Perform packet hardware copy */ + nb_enq = ioat_enqueue_packets(pkts_burst, + nb_rx, rx_config->ioat_ids[i]); + if (nb_enq > 0) + rte_ioat_do_copies(rx_config->ioat_ids[i]); + } else { + /* Perform packet software copy, free source packets */ + int ret; + struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST]; + + ret = rte_mempool_get_bulk(ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + + if (unlikely(ret < 0)) + rte_exit(EXIT_FAILURE, + "Unable to allocate memory.\n"); + + for (j = 0; j < nb_rx; j++) + pktmbuf_sw_copy(pkts_burst[j], + pkts_burst_copy[j]); + + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq = rte_ring_enqueue_burst( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx, NULL); + + /* Free any not enqueued packets. */ + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } +
doc/guides/sample_app_ug/ioat.rst line 599
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ port_statistics.copy_dropped[rx_config->rxtx_port] += + (nb_rx - nb_enq); + } + } + +The packets are received in burst mode using ``rte_eth_rx_burst()`` +function. When using hardware copy mode the packets are enqueued in +copying device's buffer using ``ioat_enqueue_packets()`` which calls +``rte_ioat_enqueue_copy()``. When all received packets are in the +buffer the copies are invoked by calling ``rte_ioat_do_copies()``.
s/copies are invoked/copy operations are started/
[Marcin] Changed the description accordingly
doc/guides/sample_app_ug/ioat.rst line 602
Quoted 3 lines
doc/guides/sample_app_ug/ioat.rst new file +3
-0+Function ``rte_ioat_enqueue_copy()`` operates on physical address of +the packet. Structure ``rte_mbuf`` contains only physical address to +start of the data buffer (``buf_iova``). Thus the address is shifted
s/shifted/adjusted/
[Marcin] Changed the description accordingly
Quoted 1 line
+by ``addr_offset`` value in order to get pointer to ``rearm_data``
s/pointer to/the address of/
[Marcin] Changed the description accordingly
doc/guides/sample_app_ug/ioat.rst line 605
Quoted 2 lines
doc/guides/sample_app_ug/ioat.rst new file +2
-0+member of ``rte_mbuf``. That way the packet is copied all at once +(with data and metadata).
"That way the both the packet data and metadata can be copied in a single operation".
Should also note that this shortcut can be used because the mbufs are "direct" mbufs allocated by the apps. If another app uses external buffers, or indirect mbufs, then multiple copy operations must be used. [Marcin] Changed the description accordingly and added additional informations.
Quoted 39 lines
doc/guides/sample_app_ug/ioat.rst new file +39
-0+ +.. code-block:: c + + static uint32_t + ioat_enqueue_packets(struct rte_mbuf **pkts, + uint32_t nb_rx, uint16_t dev_id) + { + int ret; + uint32_t i; + struct rte_mbuf *pkts_copy[MAX_PKT_BURST]; + + const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr, + &pkts[0]->rearm_data); + + ret = rte_mempool_get_bulk(ioat_pktmbuf_pool, + (void *)pkts_copy, nb_rx); + + if (unlikely(ret < 0)) + rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n"); + + for (i = 0; i < nb_rx; i++) { + /* Perform data copy */ + ret = rte_ioat_enqueue_copy(dev_id, + pkts[i]->buf_iova + - addr_offset, + pkts_copy[i]->buf_iova + - addr_offset, + rte_pktmbuf_data_len(pkts[i]) + + addr_offset, + (uintptr_t)pkts[i], + (uintptr_t)pkts_copy[i], + 0 /* nofence */); + + if (ret != 1) + break; + } + + ret = i; + /* Free any not enqueued packets. */
doc/guides/sample_app_ug/ioat.rst
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +10
-0+ rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i); + rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i], + nb_rx - i); + + return ret; + } + + +All done copies are processed by ``ioat_tx_port()`` function. When +using
s/done/completed/
[Marcin] Changed the description accordingly
Quoted 102 lines
doc/guides/sample_app_ug/ioat.rst new file +102
-0+hardware copy mode the function invokes +``rte_ioat_completed_copies()`` on each assigned IOAT channel to +gather copied packets. If software copy mode is used the function +dequeues copied packets from the rte_ring. Then each packet MAC +address is changed if it was enabled. After that copies are sent in burst mode using `` rte_eth_tx_burst()``. + + +.. code-block:: c + + /* Transmit packets from IOAT rawdev/rte_ring for one port. */ + static void + ioat_tx_port(struct rxtx_port_config *tx_config) + { + uint32_t i, j, nb_dq = 0; + struct rte_mbuf *mbufs_src[MAX_PKT_BURST]; + struct rte_mbuf *mbufs_dst[MAX_PKT_BURST]; + + if (copy_mode == COPY_MODE_IOAT_NUM) { + for (i = 0; i < tx_config->nb_queues; i++) { + /* Deque the mbufs from IOAT device. */ + nb_dq = rte_ioat_completed_copies( + tx_config->ioat_ids[i], MAX_PKT_BURST, + (void *)mbufs_src, (void *)mbufs_dst); + + if (nb_dq == 0) + break; + + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)mbufs_src, nb_dq); + + /* Update macs if enabled */ + if (mac_updating) { + for (j = 0; j < nb_dq; j++) + update_mac_addrs(mbufs_dst[j], + tx_config->rxtx_port); + } + + const uint16_t nb_tx = rte_eth_tx_burst( + tx_config->rxtx_port, 0, + (void *)mbufs_dst, nb_dq); + + port_statistics.tx[tx_config->rxtx_port] += nb_tx; + + /* Free any unsent packets. */ + if (unlikely(nb_tx < nb_dq)) + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } + } + else { + for (i = 0; i < tx_config->nb_queues; i++) { + /* Deque the mbufs from IOAT device. */ + nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring, + (void *)mbufs_dst, MAX_PKT_BURST, NULL); + + if (nb_dq == 0) + return; + + /* Update macs if enabled */ + if (mac_updating) { + for (j = 0; j < nb_dq; j++) + update_mac_addrs(mbufs_dst[j], + tx_config->rxtx_port); + } + + const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port, + 0, (void *)mbufs_dst, nb_dq); + + port_statistics.tx[tx_config->rxtx_port] += nb_tx; + + /* Free any unsent packets. */ + if (unlikely(nb_tx < nb_dq)) + rte_mempool_put_bulk(ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } + } + } + +The Packet Copying Functions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In order to perform packet copy there is a user-defined function +``pktmbuf_sw_copy()`` used. It copies a whole packet by copying +metadata from source packet to new mbuf, and then copying a data +chunk of source packet. Both memory copies are done using +``rte_memcpy()``: + +.. code-block:: c + + static inline void + pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) + { + /* Copy packet metadata */ + rte_memcpy(&dst->rearm_data, + &src->rearm_data, + offsetof(struct rte_mbuf, cacheline1) + - offsetof(struct rte_mbuf, rearm_data)); + + /* Copy packet data */ + rte_memcpy(rte_pktmbuf_mtod(dst, char *),
doc/guides/sample_app_ug/ioat.rst
Quoted 10 lines
doc/guides/sample_app_ug/ioat.rst new file +9
-0+ rte_pktmbuf_mtod(src, char *), src->data_len); + } + +The metadata in this example is copied from ``rearm_data`` member of +``rte_mbuf`` struct up to ``cacheline1``. + +In order to understand why software packet copying is done as shown +above please refer to the "Mbuf Library" section of the *DPDK +Programmer's Guide*.
Use a text editor that adds a newline automatically :-) [Marcin] Added new line at end of file.
/Bruce
v6 cover examples/ioat: sample app on ioat driver usage
In reply to Marcin BaranA new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
v6:
- rearrange 'ioat_tx_port()' to remove unnecessary code
- improve documentation
- format patches for better readability
v5:
- change dependency name from 'pmd_ioat' to 'rawdev_ioat' fixing build errors (branch diverged from master)
v4:
- fix meson build support check
v3:
- add check for meson build whether IOAT is supported
v2:
- change printing total stats to be deltas
- update documentation
- fix 1 thread/sw copy/multiple Rx queues packet dropping
- divide patch into several presenting functional change
Marcin Baran (4):
examples/ioat: add software copy support examples/ioat: add rawdev copy mode support examples/ioat: add stats printing for each port doc/guides/: provide IOAT sample app guide
Pawel Modrak (2):
examples/ioat: create sample app on ioat driver usage examples/ioat: add two threads configuration
doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 581 ++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1003 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 15 + examples/meson.build | 1 + 8 files changed, 1662 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
Signature · 1 line
2.22.0.windows.1
PATCH v6 1/6 examples/ioat: create sample app on ioat driver usage
From: Pawel Modrak <[email protected]>
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The change covers ports initialization, closing connection and argument parsing.
--- examples/Makefile | 3 + examples/ioat/Makefile | 54 +++++ examples/ioat/ioatfwd.c | 439 ++++++++++++++++++++++++++++++++++++++ examples/ioat/meson.build | 15 ++ examples/meson.build | 1 + 5 files changed, 512 insertions(+) create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
examples/Makefile +3 -0
@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify DIRS-y += flow_filtering DIRS-y += helloworld +ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y) +DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat +endif DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline ifeq ($(CONFIG_RTE_LIBRTE_LPM),y) DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
examples/ioat/Makefile new file +54 -0
54 lines not on this page · raw message
examples/ioat/ioatfwd.c new file +439 -0
439 lines not on this page · raw message
examples/ioat/meson.build new file +15 -0
@@ -0,0 +1,15 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +build = dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV') + +deps += ['rawdev_ioat'] + +sources = files( + 'ioatfwd.c' +)
examples/meson.build +1 -0
@@ -16,6 +16,7 @@ all_examples = [ 'eventdev_pipeline', 'exception_path', 'fips_validation', 'flow_classify', 'flow_filtering', 'helloworld', + 'ioat', 'ip_fragmentation', 'ip_pipeline', 'ip_reassembly', 'ipsec-secgw', 'ipv4_multicast', 'kni',
Signature · 1 line
2.22.0.windows.1
PATCH v6 2/6 examples/ioat: add software copy support
In reply to Marcin BaranAdded support for copying packets
using software copy mode and MAC address
changing. The copies are processed using
one lcore.
--- examples/ioat/ioatfwd.c | 177 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 177 insertions(+)
examples/ioat/ioatfwd.c +177 -0
@@ -89,6 +89,154 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ];struct rte_mempool *ioat_pktmbuf_pool; +static void +update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) +{ +struct rte_ether_hdr *eth; +void *tmp; + + eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *); + +/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but +* it's acceptable cause it gets overwritten by rte_ether_addr_copy +*/ + tmp = ð->d_addr.addr_bytes[0 ]; + *((uint64_t *)tmp) =0x000000000002 + ((uint64_t )dest_portid <<40 ); + +/* src addr */ +rte_ether_addr_copy (&ioat_ports_eth_addr[dest_portid], ð->s_addr); +} + +static inline void +pktmbuf_sw_copy (struct rte_mbuf *src,struct rte_mbuf *dst) +{ +/* Copy packet metadata */ +rte_memcpy (&dst->rearm_data, + &src->rearm_data, + offsetof(struct rte_mbuf , cacheline1) + - offsetof(struct rte_mbuf , rearm_data)); + +/* Copy packet data */ + rte_memcpy(rte_pktmbuf_mtod(dst, char *), + rte_pktmbuf_mtod(src, char *), src->data_len); +} + +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ +static void +ioat_rx_port (struct rxtx_port_config *rx_config) +{ +uint32_t nb_rx, nb_enq, i, j; +struct rte_mbuf *pkts_burst[MAX_PKT_BURST ]; + +for (i =0 ; i < rx_config->nb_queues; i++) { + + nb_rx =rte_eth_rx_burst (rx_config->rxtx_port, i, + pkts_burst,MAX_PKT_BURST ); + +if (nb_rx ==0 ) +continue ; + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Perform packet hardware copy */ + }else { +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } + } +} + +/* Transmit packets from IOAT rawdev/rte_ring for one port. */ +static void +ioat_tx_port (struct rxtx_port_config *tx_config) +{ +uint32_t i, j, nb_dq =0 ; +struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ]; + +for (i =0 ; i < tx_config->nb_queues; i++) { +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Deque the mbufs from IOAT device. */ + }else { +/* Deque the mbufs from rx_to_tx_ring. */ + nb_dq =rte_ring_dequeue_burst ( + tx_config->rx_to_tx_ring, (void *)mbufs_dst, +MAX_PKT_BURST ,NULL ); + } + +if (nb_dq ==0 ) +return ; + +/* Update macs if enabled */ +if (mac_updating) { +for (j =0 ; j < nb_dq; j++) +update_mac_addrs (mbufs_dst[j], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst ( + tx_config->rxtx_port,0 , + (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } +} + +/* Main rx and tx loop if only one slave lcore available */ +static void +rxtx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx and tx loop for copy on" +" lcore %u\n" ,rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) { +ioat_rx_port (&cfg.ports[i]); +ioat_tx_port (&cfg.ports[i]); + } +} + +static void start_forwarding_cores (void ) +{ +uint32_t lcore_id =rte_lcore_id (); + +RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , + __func__,rte_lcore_id ()); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); +} +/* Display usage */ static void ioat_usage (const char *prgname)
@@ -253,6 +401,26 @@ check_link_status(uint32_t port_mask)return retval; } +static void +assign_rings (void ) +{ +uint32_t i; + +for (i =0 ; i < cfg.nb_ports; i++) { +char ring_name[RTE_RING_NAMESIZE ]; + +snprintf (ring_name,sizeof (ring_name),"rx_to_tx_ring_%u" , i); +/* Create ring for inter core communication */ + cfg.ports[i].rx_to_tx_ring =rte_ring_create ( + ring_name, ring_size, +rte_socket_id (),RING_F_SP_ENQ |RING_F_SC_DEQ ); + +if (cfg.ports[i].rx_to_tx_ring ==NULL ) +rte_exit (EXIT_FAILURE ,"Ring create failed: %s\n" , +rte_strerror (rte_errno)); + } +} +/* * Initializes a given port using global settings and with the RX buffers * coming from the mbuf_pool passed as a parameter.
@@ -428,10 +596,19 @@ main(int argc, char **argv)if (cfg.nb_lcores <1 )rte_exit (EXIT_FAILURE ,"There should be at least one slave lcore.\n" ); + +assign_rings (); + +start_forwarding_cores (); + +/* force_quit is true when we get here */ +rte_eal_mp_wait_lcore (); +for (i =0 ; i < cfg.nb_ports; i++) {printf ("Closing port %d\n" , cfg.ports[i].rxtx_port);rte_eth_dev_stop (cfg.ports[i].rxtx_port);rte_eth_dev_close (cfg.ports[i].rxtx_port); +rte_ring_free (cfg.ports[i].rx_to_tx_ring); }printf ("Bye...\n" );
Signature · 1 line
2.22.0.windows.1
PATCH v6 3/6 examples/ioat: add rawdev copy mode support
In reply to Marcin BaranAdded support for copying packets using
rawdev device. Each port's Rx queue is
assigned DMA channel for copy.
--- examples/ioat/ioatfwd.c | 117 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 115 insertions(+), 2 deletions(-)
examples/ioat/ioatfwd.c +115 -2
@@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) rte_pktmbuf_mtod(src, char *), src->data_len); } +static uint32_t +ioat_enqueue_packets (struct rte_mbuf **pkts, +uint32_t nb_rx,uint16_t dev_id) +{ +int ret; +uint32_t i; +struct rte_mbuf *pkts_copy[MAX_PKT_BURST ]; + +const uint64_t addr_offset =RTE_PTR_DIFF (pkts[0 ]->buf_addr, + &pkts[0 ]->rearm_data); + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE ,"Unable to allocate memory.\n" ); + +for (i =0 ; i < nb_rx; i++) { +/* Perform data copy */ + ret =rte_ioat_enqueue_copy (dev_id, + pkts[i]->buf_iova + - addr_offset, + pkts_copy[i]->buf_iova + - addr_offset, +rte_pktmbuf_data_len (pkts[i]) + + addr_offset, + (uintptr_t )pkts[i], + (uintptr_t )pkts_copy[i], +0 /* nofence */ ); + +if (ret !=1 ) +break ; + } + + ret = i; +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i); +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], + nb_rx - i); + + +return ret; +} +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ static void ioat_rx_port (struct rxtx_port_config *rx_config)
@@ -138,6 +182,10 @@ ioat_rx_port(struct rxtx_port_config *rx_config)if (copy_mode ==COPY_MODE_IOAT_NUM ) {/* Perform packet hardware copy */ + nb_enq =ioat_enqueue_packets (pkts_burst, + nb_rx, rx_config->ioat_ids[i]); +if (nb_enq >0 ) +rte_ioat_do_copies (rx_config->ioat_ids[i]); }else {/* Perform packet software copy, free source packets */ int ret;
@@ -174,11 +222,15 @@ static void ioat_tx_port(struct rxtx_port_config *tx_config) {uint32_t i, j, nb_dq =0 ; +struct rte_mbuf *mbufs_src[MAX_PKT_BURST ];struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ];for (i =0 ; i < tx_config->nb_queues; i++) {if (copy_mode ==COPY_MODE_IOAT_NUM ) {/* Deque the mbufs from IOAT device. */ + nb_dq =rte_ioat_completed_copies ( + tx_config->ioat_ids[i],MAX_PKT_BURST , + (void *)mbufs_src, (void *)mbufs_dst); }else {/* Deque the mbufs from rx_to_tx_ring. */ nb_dq = rte_ring_dequeue_burst(
@@ -189,6 +241,10 @@ ioat_tx_port(struct rxtx_port_config *tx_config)if (nb_dq ==0 )return ; +if (copy_mode ==COPY_MODE_IOAT_NUM ) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)mbufs_src, nb_dq); +/* Update macs if enabled */ if (mac_updating) {for (j =0 ; j < nb_dq; j++)
@@ -401,6 +457,52 @@ check_link_status(uint32_t port_mask)return retval; } +static void +configure_rawdev_queue (uint32_t dev_id) +{ +struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size }; +struct rte_rawdev_info info = { .dev_private = &dev_config }; + +if (rte_rawdev_configure (dev_id, &info) !=0 ) { +rte_exit (EXIT_FAILURE , +"Error with rte_rawdev_configure()\n" ); + } +if (rte_rawdev_start (dev_id) !=0 ) { +rte_exit (EXIT_FAILURE , +"Error with rte_rawdev_start()\n" ); + } +} + +static void +assign_rawdevs (void ) +{ +uint16_t nb_rawdev =0 , rdev_id =0 ; +uint32_t i, j; + +for (i =0 ; i < cfg.nb_ports; i++) { +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { +struct rte_rawdev_info rdev_info = {0 }; + +do { +if (rdev_id ==rte_rawdev_count ()) + goto end; +rte_rawdev_info_get (rdev_id++, &rdev_info); + }while (strcmp (rdev_info.driver_name, +IOAT_PMD_RAWDEV_NAME_STR ) !=0 ); + + cfg.ports[i].ioat_ids[j] = rdev_id -1 ; +configure_rawdev_queue (cfg.ports[i].ioat_ids[j]); + ++nb_rawdev; + } + } +end: +if (nb_rawdev < cfg.nb_ports * cfg.ports[0 ].nb_queues) +rte_exit (EXIT_FAILURE , +"Not enough IOAT rawdevs (%u) for all queues (%u).\n" , + nb_rawdev, cfg.nb_ports * cfg.ports[0 ].nb_queues); +RTE_LOG (INFO ,IOAT ,"Number of used rawdevs: %u.\n" , nb_rawdev); +} +static void assign_rings (void ) {
@@ -597,18 +699,29 @@ main(int argc, char **argv)rte_exit (EXIT_FAILURE ,"There should be at least one slave lcore.\n" ); -assign_rings (); +if (copy_mode ==COPY_MODE_IOAT_NUM ) +assign_rawdevs (); +else /* copy_mode == COPY_MODE_SW_NUM */ +assign_rings ();start_forwarding_cores ();/* force_quit is true when we get here */ rte_eal_mp_wait_lcore (); +uint32_t j;for (i =0 ; i < cfg.nb_ports; i++) {printf ("Closing port %d\n" , cfg.ports[i].rxtx_port);rte_eth_dev_stop (cfg.ports[i].rxtx_port);rte_eth_dev_close (cfg.ports[i].rxtx_port); -rte_ring_free (cfg.ports[i].rx_to_tx_ring); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { +printf ("Stopping rawdev %d\n" , + cfg.ports[i].ioat_ids[j]); +rte_rawdev_stop (cfg.ports[i].ioat_ids[j]); + } + }else /* copy_mode == COPY_MODE_SW_NUM */ +rte_ring_free (cfg.ports[i].rx_to_tx_ring); }printf ("Bye...\n" );
Signature · 1 line
2.22.0.windows.1
PATCH v6 4/6 examples/ioat: add two threads configuration
In reply to Marcin BaranFrom: Pawel Modrak <[email protected]>
Added possibility to use two lcores: first for
packet receiving and copying, second for packets
sending.
--- examples/ioat/ioatfwd.c | 47 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 43 insertions(+), 4 deletions(-)
examples/ioat/ioatfwd.c +43 -4
@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], nb_rx - i); -return ret; }
@@ -264,6 +263,36 @@ ioat_tx_port(struct rxtx_port_config *tx_config) } } +/* Main rx processing loop for IOAT rawdev. */ +static void +rx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx loop for copy on lcore %u\n" , +rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) +ioat_rx_port (&cfg.ports[i]); +} + +/* Main tx processing loop for hardware copy. */ +static void +tx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main tx loop for copy on lcore %u\n" , +rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) +ioat_tx_port (&cfg.ports[i]); +} +/* Main rx and tx loop if only one slave lcore available */ static void rxtx_main_loop (void )
@@ -288,9 +317,19 @@ static void start_forwarding_cores(void)RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , __func__,rte_lcore_id ()); - lcore_id =rte_get_next_lcore (lcore_id, true, true); -rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, -NULL , lcore_id); +if (cfg.nb_lcores ==1 ) { + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); + }else if (cfg.nb_lcores >1 ) { + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rx_main_loop, +NULL , lcore_id); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)tx_main_loop,NULL , + lcore_id); + } }/* Display usage */
Signature · 1 line
2.22.0.windows.1
PATCH v6 5/6 examples/ioat: add stats printing for each port
In reply to Marcin BaranAdded printing stats of ports each second. The stats printing is done using master core. The information provided informs about packets received, dropped and send as well as statistics of rawdev devices.
--- examples/ioat/ioatfwd.c | 235 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 235 insertions(+)
examples/ioat/ioatfwd.c +235 -0
@@ -48,6 +48,23 @@ struct rxtx_transmission_config {uint16_t nb_lcores; }; +/* per-port statistics struct */ +struct ioat_port_statistics { +uint64_t rx[RTE_MAX_ETHPORTS ]; +uint64_t tx[RTE_MAX_ETHPORTS ]; +uint64_t tx_dropped[RTE_MAX_ETHPORTS ]; +uint64_t copy_dropped[RTE_MAX_ETHPORTS ]; +}; +struct ioat_port_statistics port_statistics; + +struct total_statistics { +uint64_t total_packets_dropped; +uint64_t total_packets_tx; +uint64_t total_packets_rx; +uint64_t total_successful_enqueues; +uint64_t total_failed_enqueues; +}; + typedef enum copy_mode_t { #define COPY_MODE_SW "sw" COPY_MODE_SW_NUM,
@@ -89,6 +106,204 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ];struct rte_mempool *ioat_pktmbuf_pool; +/* Print out statistics for one port. */ +static void +print_port_stats (uint16_t port_id) +{ +printf ("\nStatistics for port %u ------------------------------" +"\nPackets sent: %34" PRIu64 +"\nPackets received: %30" PRIu64 +"\nPackets dropped on tx: %25" PRIu64 +"\nPackets dropped on copy: %23" PRIu64, + port_id, + port_statistics.tx[port_id], + port_statistics.rx[port_id], + port_statistics.tx_dropped[port_id], + port_statistics.copy_dropped[port_id]); +} + +/* Print out statistics for one IOAT rawdev device. */ +static void +print_rawdev_stats (uint32_t dev_id,uint64_t *xstats, +unsigned int *ids_xstats,uint16_t nb_xstats, +struct rte_rawdev_xstats_name *names_xstats) +{ +uint16_t i; + +printf ("\nIOAT channel %u" , dev_id); +for (i =0 ; i < nb_xstats; i++) +printf ("\n\t %s: %*" PRIu64, + names_xstats[ids_xstats[i]].name, + (int )(37 -strlen (names_xstats[ids_xstats[i]].name)), + xstats[i]); +} + +static void +print_total_stats (struct total_statistics *ts) +{ +printf ("\nAggregate statistics ===============================" +"\nTotal packets Tx: %24" PRIu64" [pps]" +"\nTotal packets Rx: %24" PRIu64" [pps]" +"\nTotal packets dropped: %19" PRIu64" [pps]" , + ts->total_packets_tx, + ts->total_packets_rx, + ts->total_packets_dropped); + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +printf ("\nTotal IOAT successful enqueues: %8" PRIu64" [enq/s]" +"\nTotal IOAT failed enqueues: %12" PRIu64" [enq/s]" , + ts->total_successful_enqueues, + ts->total_failed_enqueues); + } + +printf ("\n====================================================\n" ); +} + +/* Print out statistics on packets dropped. */ +static void +print_stats (char *prgname) +{ +struct total_statistics ts, delta_ts; +uint32_t i, port_id, dev_id; +struct rte_rawdev_xstats_name *names_xstats; +uint64_t *xstats; +unsigned int *ids_xstats, nb_xstats; +char status_string[120 ];/* to print at the top of the output */ +int status_strlen; + + +const char clr[] = {27 ,'[' ,'2' ,'J' ,'\0' }; +const char topLeft[] = {27 ,'[' ,'1' ,';' ,'1' ,'H' ,'\0' }; + + status_strlen =snprintf (status_string,sizeof (status_string), +"%s, " , prgname); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Worker Threads = %d, " , +rte_lcore_count () >2 ?2 :1 ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Copy Mode = %s,\n" , copy_mode ==COPY_MODE_SW_NUM ? +COPY_MODE_SW :COPY_MODE_IOAT ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Updating MAC = %s, " , mac_updating ? +"enabled" :"disabled" ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Rx Queues = %d, " , nb_queues); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Ring Size = %d\n" , ring_size); + +/* Allocate memory for xstats names and values */ + nb_xstats =rte_rawdev_xstats_names_get ( + cfg.ports[0 ].ioat_ids[0 ],NULL ,0 ); + + names_xstats =malloc (sizeof (*names_xstats) * nb_xstats); +if (names_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat names memory\n" ); + } +rte_rawdev_xstats_names_get (cfg.ports[0 ].ioat_ids[0 ], + names_xstats, nb_xstats); + + ids_xstats =malloc (sizeof (*ids_xstats) *2 ); +if (ids_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat ids_xstats memory\n" ); + } + + xstats =malloc (sizeof (*xstats) *2 ); +if (xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat memory\n" ); + } + +/* Get failed/successful enqueues stats index */ + ids_xstats[0 ] = ids_xstats[1 ] = nb_xstats; +for (i =0 ; i < nb_xstats; i++) { +if (!strcmp (names_xstats[i].name,"failed_enqueues" )) + ids_xstats[0 ] = i; +else if (!strcmp (names_xstats[i].name,"successful_enqueues" )) + ids_xstats[1 ] = i; +if (ids_xstats[0 ] < nb_xstats && ids_xstats[1 ] < nb_xstats) +break ; + } +if (ids_xstats[0 ] == nb_xstats || ids_xstats[1 ] == nb_xstats) { +rte_exit (EXIT_FAILURE , +"Error getting failed/successful enqueues stats index\n" ); + } + +memset (&ts,0 ,sizeof (struct total_statistics )); + +while (!force_quit) { +/* Sleep for 1 second each round - init sleep allows reading +* messages from app startup. +*/ +sleep (1 ); + +/* Clear screen and move to top left */ +printf ("%s%s" , clr, topLeft); + +memset (&delta_ts,0 ,sizeof (struct total_statistics )); + +printf ("%s" , status_string); + +for (i =0 ; i < cfg.nb_ports; i++) { + port_id = cfg.ports[i].rxtx_port; +print_port_stats (port_id); + + delta_ts.total_packets_dropped += + port_statistics.tx_dropped[port_id] + + port_statistics.copy_dropped[port_id]; + delta_ts.total_packets_tx += + port_statistics.tx[port_id]; + delta_ts.total_packets_rx += + port_statistics.rx[port_id]; + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +uint32_t j; + +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { + dev_id = cfg.ports[i].ioat_ids[j]; +rte_rawdev_xstats_get (dev_id, + ids_xstats, xstats,2 ); + +print_rawdev_stats (dev_id, xstats, + ids_xstats,2 , names_xstats); + + delta_ts.total_failed_enqueues += + xstats[ids_xstats[0 ]]; + delta_ts.total_successful_enqueues += + xstats[ids_xstats[1 ]]; + } + } + } + + delta_ts.total_packets_tx -= ts.total_packets_tx; + delta_ts.total_packets_rx -= ts.total_packets_rx; + delta_ts.total_packets_dropped -= ts.total_packets_dropped; + delta_ts.total_failed_enqueues -= ts.total_failed_enqueues; + delta_ts.total_successful_enqueues -= + ts.total_successful_enqueues; + +printf ("\n" ); +print_total_stats (&delta_ts); + + ts.total_packets_tx += delta_ts.total_packets_tx; + ts.total_packets_rx += delta_ts.total_packets_rx; + ts.total_packets_dropped += delta_ts.total_packets_dropped; + ts.total_failed_enqueues += delta_ts.total_failed_enqueues; + ts.total_successful_enqueues += + delta_ts.total_successful_enqueues; + } + +free (names_xstats); +free (xstats); +free (ids_xstats); +} +static void update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) {
@@ -179,6 +394,8 @@ ioat_rx_port(struct rxtx_port_config *rx_config)if (nb_rx ==0 )continue ; + port_statistics.rx[rx_config->rxtx_port] += nb_rx; +if (copy_mode ==COPY_MODE_IOAT_NUM ) {/* Perform packet hardware copy */ nb_enq = ioat_enqueue_packets(pkts_burst,
@@ -213,6 +430,9 @@ ioat_rx_port(struct rxtx_port_config *rx_config) (void *)&pkts_burst_copy[nb_enq], nb_rx - nb_enq); } + + port_statistics.copy_dropped[rx_config->rxtx_port] += + (nb_rx - nb_enq); } }
@@ -255,6 +475,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config) tx_config->rxtx_port,0 , (void *)mbufs_dst, nb_dq); + port_statistics.tx[tx_config->rxtx_port] += nb_tx; +/* Free any unsent packets. */ if (unlikely (nb_tx < nb_dq)) rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -653,6 +875,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)rte_eth_tx_buffer_init (tx_buffer[portid],MAX_PKT_BURST ); + ret =rte_eth_tx_buffer_set_err_callback (tx_buffer[portid], + rte_eth_tx_buffer_count_callback, + &port_statistics.tx_dropped[portid]); +if (ret <0 ) +rte_exit (EXIT_FAILURE , +"Cannot set error callback for tx buffer on port %u\n" , + portid); +/* Start device */ ret =rte_eth_dev_start (portid);if (ret <0 )
@@ -729,6 +959,9 @@ main(int argc, char **argv)RTE_ETH_FOREACH_DEV (portid)port_init (portid, ioat_pktmbuf_pool, nb_queues); +/* Initialize port xstats */ +memset (&port_statistics,0 ,sizeof (port_statistics)); +while (!check_link_status (ioat_enabled_port_mask) && !force_quit)sleep (1 );
@@ -744,6 +977,8 @@ main(int argc, char **argv)assign_rings ();start_forwarding_cores (); +/* master core prints stats while other cores forward */ +print_stats (argv[0 ]);/* force_quit is true when we get here */ rte_eal_mp_wait_lcore ();
Signature · 1 line
2.22.0.windows.1
PATCH v6 6/6 doc/guides/: provide IOAT sample app guide
In reply to Marcin BaranAdded guide for IOAT sample app usage and
code description.
--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 581 +++++++++++++++++++++++++++++ 3 files changed, 586 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst
doc/guides/sample_app_ug/index.rst +1 -0
@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtual
doc/guides/sample_app_ug/intro.rst +4 -0
@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.
doc/guides/sample_app_ug/ioat.rst new file +581 -0
581 lines not on this page · raw message
Signature · 1 line
2.22.0.windows.1
Re: [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage
In reply to Marcin BaranQuoted 34 lines · Marcin Baran
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
v6:
- rearrange 'ioat_tx_port()' to remove unnecessary code
- improve documentation
- format patches for better readabilityv5:
- change dependency name from 'pmd_ioat' to 'rawdev_ioat' fixing build errors (branch diverged from master)v4:
- fix meson build support checkv3:
- add check for meson build whether IOAT is supportedv2:
- change printing total stats to be deltas
- update documentation
- fix 1 thread/sw copy/multiple Rx queues packet dropping
- divide patch into several presenting functional changeMarcin Baran (4):
examples/ioat: add software copy support examples/ioat: add rawdev copy mode support examples/ioat: add stats printing for each port doc/guides/: provide IOAT sample app guide
Quoted 4 lines
Pawel Modrak (2):
examples/ioat: create sample app on ioat driver usage examples/ioat: add two threads configuration
Re: [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage
In reply to Marcin BaranQuoted 38 lines · Marcin Baran
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
v6:
- rearrange 'ioat_tx_port()' to remove unnecessary code
- improve documentation
- format patches for better readabilityv5:
- change dependency name from 'pmd_ioat' to 'rawdev_ioat' fixing build errors (branch diverged from master)v4:
- fix meson build support checkv3:
- add check for meson build whether IOAT is supportedv2:
- change printing total stats to be deltas
- update documentation
- fix 1 thread/sw copy/multiple Rx queues packet dropping
- divide patch into several presenting functional changeMarcin Baran (4):
examples/ioat: add software copy support examples/ioat: add rawdev copy mode support examples/ioat: add stats printing for each port doc/guides/: provide IOAT sample app guidePawel Modrak (2):
examples/ioat: create sample app on ioat driver usage examples/ioat: add two threads configuration
Quoted 14 lines
doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 581 ++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1003 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 15 + examples/meson.build | 1 + 8 files changed, 1662 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
Just a nit, but I think you need to add something to the MAINTAINERS file for this example app, maybe it can be a follow up patch if it's the only rework
Re: [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage
Quoted 54 lines · Kevin Traynor
Quoted 38 lines · Marcin Baran
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
v6:
- rearrange 'ioat_tx_port()' to remove unnecessary code
- improve documentation
- format patches for better readabilityv5:
- change dependency name from 'pmd_ioat' to 'rawdev_ioat' fixing build errors (branch diverged from master)v4:
- fix meson build support checkv3:
- add check for meson build whether IOAT is supportedv2:
- change printing total stats to be deltas
- update documentation
- fix 1 thread/sw copy/multiple Rx queues packet dropping
- divide patch into several presenting functional changeMarcin Baran (4):
examples/ioat: add software copy support examples/ioat: add rawdev copy mode support examples/ioat: add stats printing for each port doc/guides/: provide IOAT sample app guidePawel Modrak (2):
examples/ioat: create sample app on ioat driver usage examples/ioat: add two threads configurationQuoted 14 lines
doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 581 ++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1003 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 15 + examples/meson.build | 1 + 8 files changed, 1662 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
Quoted 3 lines
Just a nit, but I think you need to add something to the
MAINTAINERSfile for this example app, maybe it can be a follow up patch if it's the only rework
Good catch. For completeness I've sent up a V7 of this set, adding this example under the existing ioat section in MAINTAINERS.
/Bruce
v7 cover examples/ioat: sample app for ioat driver
In reply to Marcin BaranA new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The patch includes sample application as well as it's guide.
v7:
- MAINTAINERS file entry added
v6:
- rearrange 'ioat_tx_port()' to remove unnecessary code
- improve documentation
- format patches for better readability
v5:
- change dependency name from 'pmd_ioat' to 'rawdev_ioat' fixing build errors (branch diverged from master)
v4:
- fix meson build support check
v3:
- add check for meson build whether IOAT is supported
v2:
- change printing total stats to be deltas
- update documentation
- fix 1 thread/sw copy/multiple Rx queues packet dropping
- divide patch into several presenting functional change
Marcin Baran (4):
examples/ioat: add software copy support
examples/ioat: add rawdev copy mode support
examples/ioat: add stats printing for each port
doc/guides/: provide IOAT sample app guide
Pawel Modrak (2):
examples/ioat: create sample app on ioat driver usage examples/ioat: add two threads configuration
MAINTAINERS | 1 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 581 ++++++++++++++++ examples/Makefile | 3 + examples/ioat/Makefile | 54 ++ examples/ioat/ioatfwd.c | 1003 ++++++++++++++++++++++++++++ examples/ioat/meson.build | 15 + examples/meson.build | 1 + 9 files changed, 1663 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
Signature · 1 line
2.21.0
PATCH v7 1/6 examples/ioat: new sample app for ioat driver
From: Pawel Modrak <[email protected]>
A new sample app demonstrating use of driver for CBDMA. The app receives packets, performs software or hardware copy, changes packets' MAC addresses (if enabled) and forwards them. The change covers ports initialization, closing connection and argument parsing.
---
V6: add maintainers file entry for this new app
--- MAINTAINERS | 1 + examples/Makefile | 3 + examples/ioat/Makefile | 54 +++++ examples/ioat/ioatfwd.c | 439 ++++++++++++++++++++++++++++++++++++++ examples/ioat/meson.build | 15 ++ examples/meson.build | 1 + 6 files changed, 513 insertions(+) create mode 100644 examples/ioat/Makefile create mode 100644 examples/ioat/ioatfwd.c create mode 100644 examples/ioat/meson.build
MAINTAINERS +1 -0
@@ -1107,6 +1107,7 @@ IOAT Rawdev M: Bruce Richardson <[email protected]> F: drivers/raw/ioat/ F: doc/guides/rawdevs/ioat.rst +F: examples/ioat/ NXP DPAA2 QDMA M: Nipun Gupta <[email protected]>
examples/Makefile +3 -0
@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify DIRS-y += flow_filtering DIRS-y += helloworld +ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y) +DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat +endif DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline ifeq ($(CONFIG_RTE_LIBRTE_LPM),y) DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
examples/ioat/Makefile new file +54 -0
54 lines not on this page · raw message
examples/ioat/ioatfwd.c new file +439 -0
439 lines not on this page · raw message
examples/ioat/meson.build new file +15 -0
@@ -0,0 +1,15 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +build = dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV') + +deps += ['rawdev_ioat'] + +sources = files( + 'ioatfwd.c' +)
examples/meson.build +1 -0
@@ -16,6 +16,7 @@ all_examples = [ 'eventdev_pipeline', 'exception_path', 'fips_validation', 'flow_classify', 'flow_filtering', 'helloworld', + 'ioat', 'ip_fragmentation', 'ip_pipeline', 'ip_reassembly', 'ipsec-secgw', 'ipv4_multicast', 'kni',
Signature · 1 line
2.21.0
PATCH v7 2/6 examples/ioat: add software copy support
In reply to Bruce RichardsonFrom: Marcin Baran <[email protected]>
Added support for copying packets using software copy mode and MAC address changing. The copies are processed using one lcore.
--- examples/ioat/ioatfwd.c | 177 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 177 insertions(+)
examples/ioat/ioatfwd.c +177 -0
@@ -89,6 +89,154 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ];struct rte_mempool *ioat_pktmbuf_pool; +static void +update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) +{ +struct rte_ether_hdr *eth; +void *tmp; + + eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *); + +/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but +* it's acceptable cause it gets overwritten by rte_ether_addr_copy +*/ + tmp = ð->d_addr.addr_bytes[0 ]; + *((uint64_t *)tmp) =0x000000000002 + ((uint64_t )dest_portid <<40 ); + +/* src addr */ +rte_ether_addr_copy (&ioat_ports_eth_addr[dest_portid], ð->s_addr); +} + +static inline void +pktmbuf_sw_copy (struct rte_mbuf *src,struct rte_mbuf *dst) +{ +/* Copy packet metadata */ +rte_memcpy (&dst->rearm_data, + &src->rearm_data, + offsetof(struct rte_mbuf , cacheline1) + - offsetof(struct rte_mbuf , rearm_data)); + +/* Copy packet data */ + rte_memcpy(rte_pktmbuf_mtod(dst, char *), + rte_pktmbuf_mtod(src, char *), src->data_len); +} + +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ +static void +ioat_rx_port (struct rxtx_port_config *rx_config) +{ +uint32_t nb_rx, nb_enq, i, j; +struct rte_mbuf *pkts_burst[MAX_PKT_BURST ]; + +for (i =0 ; i < rx_config->nb_queues; i++) { + + nb_rx =rte_eth_rx_burst (rx_config->rxtx_port, i, + pkts_burst,MAX_PKT_BURST ); + +if (nb_rx ==0 ) +continue ; + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Perform packet hardware copy */ + }else { +/* Perform packet software copy, free source packets */ +int ret; +struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST ]; + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE , +"Unable to allocate memory.\n" ); + +for (j =0 ; j < nb_rx; j++) +pktmbuf_sw_copy (pkts_burst[j], + pkts_burst_copy[j]); + +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)pkts_burst, nb_rx); + + nb_enq =rte_ring_enqueue_burst ( + rx_config->rx_to_tx_ring, + (void *)pkts_burst_copy, nb_rx,NULL ); + +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&pkts_burst_copy[nb_enq], + nb_rx - nb_enq); + } + } +} + +/* Transmit packets from IOAT rawdev/rte_ring for one port. */ +static void +ioat_tx_port (struct rxtx_port_config *tx_config) +{ +uint32_t i, j, nb_dq =0 ; +struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ]; + +for (i =0 ; i < tx_config->nb_queues; i++) { +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +/* Deque the mbufs from IOAT device. */ + }else { +/* Deque the mbufs from rx_to_tx_ring. */ + nb_dq =rte_ring_dequeue_burst ( + tx_config->rx_to_tx_ring, (void *)mbufs_dst, +MAX_PKT_BURST ,NULL ); + } + +if (nb_dq ==0 ) +return ; + +/* Update macs if enabled */ +if (mac_updating) { +for (j =0 ; j < nb_dq; j++) +update_mac_addrs (mbufs_dst[j], + tx_config->rxtx_port); + } + +const uint16_t nb_tx =rte_eth_tx_burst ( + tx_config->rxtx_port,0 , + (void *)mbufs_dst, nb_dq); + +/* Free any unsent packets. */ +if (unlikely (nb_tx < nb_dq)) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)&mbufs_dst[nb_tx], + nb_dq - nb_tx); + } +} + +/* Main rx and tx loop if only one slave lcore available */ +static void +rxtx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx and tx loop for copy on" +" lcore %u\n" ,rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) { +ioat_rx_port (&cfg.ports[i]); +ioat_tx_port (&cfg.ports[i]); + } +} + +static void start_forwarding_cores (void ) +{ +uint32_t lcore_id =rte_lcore_id (); + +RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , + __func__,rte_lcore_id ()); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); +} +/* Display usage */ static void ioat_usage (const char *prgname)
@@ -253,6 +401,26 @@ check_link_status(uint32_t port_mask)return retval; } +static void +assign_rings (void ) +{ +uint32_t i; + +for (i =0 ; i < cfg.nb_ports; i++) { +char ring_name[RTE_RING_NAMESIZE ]; + +snprintf (ring_name,sizeof (ring_name),"rx_to_tx_ring_%u" , i); +/* Create ring for inter core communication */ + cfg.ports[i].rx_to_tx_ring =rte_ring_create ( + ring_name, ring_size, +rte_socket_id (),RING_F_SP_ENQ |RING_F_SC_DEQ ); + +if (cfg.ports[i].rx_to_tx_ring ==NULL ) +rte_exit (EXIT_FAILURE ,"Ring create failed: %s\n" , +rte_strerror (rte_errno)); + } +} +/* * Initializes a given port using global settings and with the RX buffers * coming from the mbuf_pool passed as a parameter.
@@ -428,10 +596,19 @@ main(int argc, char **argv)if (cfg.nb_lcores <1 )rte_exit (EXIT_FAILURE ,"There should be at least one slave lcore.\n" ); + +assign_rings (); + +start_forwarding_cores (); + +/* force_quit is true when we get here */ +rte_eal_mp_wait_lcore (); +for (i =0 ; i < cfg.nb_ports; i++) {printf ("Closing port %d\n" , cfg.ports[i].rxtx_port);rte_eth_dev_stop (cfg.ports[i].rxtx_port);rte_eth_dev_close (cfg.ports[i].rxtx_port); +rte_ring_free (cfg.ports[i].rx_to_tx_ring); }printf ("Bye...\n" );
Signature · 1 line
2.21.0
PATCH v7 3/6 examples/ioat: add rawdev copy mode support
In reply to Bruce RichardsonFrom: Marcin Baran <[email protected]>
Added support for copying packets using rawdev device. Each port's Rx queue is assigned DMA channel for copy.
--- examples/ioat/ioatfwd.c | 117 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 115 insertions(+), 2 deletions(-)
examples/ioat/ioatfwd.c +115 -2
@@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) rte_pktmbuf_mtod(src, char *), src->data_len); } +static uint32_t +ioat_enqueue_packets (struct rte_mbuf **pkts, +uint32_t nb_rx,uint16_t dev_id) +{ +int ret; +uint32_t i; +struct rte_mbuf *pkts_copy[MAX_PKT_BURST ]; + +const uint64_t addr_offset =RTE_PTR_DIFF (pkts[0 ]->buf_addr, + &pkts[0 ]->rearm_data); + + ret =rte_mempool_get_bulk (ioat_pktmbuf_pool, + (void *)pkts_copy, nb_rx); + +if (unlikely (ret <0 )) +rte_exit (EXIT_FAILURE ,"Unable to allocate memory.\n" ); + +for (i =0 ; i < nb_rx; i++) { +/* Perform data copy */ + ret =rte_ioat_enqueue_copy (dev_id, + pkts[i]->buf_iova + - addr_offset, + pkts_copy[i]->buf_iova + - addr_offset, +rte_pktmbuf_data_len (pkts[i]) + + addr_offset, + (uintptr_t )pkts[i], + (uintptr_t )pkts_copy[i], +0 /* nofence */ ); + +if (ret !=1 ) +break ; + } + + ret = i; +/* Free any not enqueued packets. */ +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i); +rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], + nb_rx - i); + + +return ret; +} +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */ static void ioat_rx_port (struct rxtx_port_config *rx_config)
@@ -138,6 +182,10 @@ ioat_rx_port(struct rxtx_port_config *rx_config)if (copy_mode ==COPY_MODE_IOAT_NUM ) {/* Perform packet hardware copy */ + nb_enq =ioat_enqueue_packets (pkts_burst, + nb_rx, rx_config->ioat_ids[i]); +if (nb_enq >0 ) +rte_ioat_do_copies (rx_config->ioat_ids[i]); }else {/* Perform packet software copy, free source packets */ int ret;
@@ -174,11 +222,15 @@ static void ioat_tx_port(struct rxtx_port_config *tx_config) {uint32_t i, j, nb_dq =0 ; +struct rte_mbuf *mbufs_src[MAX_PKT_BURST ];struct rte_mbuf *mbufs_dst[MAX_PKT_BURST ];for (i =0 ; i < tx_config->nb_queues; i++) {if (copy_mode ==COPY_MODE_IOAT_NUM ) {/* Deque the mbufs from IOAT device. */ + nb_dq =rte_ioat_completed_copies ( + tx_config->ioat_ids[i],MAX_PKT_BURST , + (void *)mbufs_src, (void *)mbufs_dst); }else {/* Deque the mbufs from rx_to_tx_ring. */ nb_dq = rte_ring_dequeue_burst(
@@ -189,6 +241,10 @@ ioat_tx_port(struct rxtx_port_config *tx_config)if (nb_dq ==0 )return ; +if (copy_mode ==COPY_MODE_IOAT_NUM ) +rte_mempool_put_bulk (ioat_pktmbuf_pool, + (void *)mbufs_src, nb_dq); +/* Update macs if enabled */ if (mac_updating) {for (j =0 ; j < nb_dq; j++)
@@ -401,6 +457,52 @@ check_link_status(uint32_t port_mask)return retval; } +static void +configure_rawdev_queue (uint32_t dev_id) +{ +struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size }; +struct rte_rawdev_info info = { .dev_private = &dev_config }; + +if (rte_rawdev_configure (dev_id, &info) !=0 ) { +rte_exit (EXIT_FAILURE , +"Error with rte_rawdev_configure()\n" ); + } +if (rte_rawdev_start (dev_id) !=0 ) { +rte_exit (EXIT_FAILURE , +"Error with rte_rawdev_start()\n" ); + } +} + +static void +assign_rawdevs (void ) +{ +uint16_t nb_rawdev =0 , rdev_id =0 ; +uint32_t i, j; + +for (i =0 ; i < cfg.nb_ports; i++) { +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { +struct rte_rawdev_info rdev_info = {0 }; + +do { +if (rdev_id ==rte_rawdev_count ()) + goto end; +rte_rawdev_info_get (rdev_id++, &rdev_info); + }while (strcmp (rdev_info.driver_name, +IOAT_PMD_RAWDEV_NAME_STR ) !=0 ); + + cfg.ports[i].ioat_ids[j] = rdev_id -1 ; +configure_rawdev_queue (cfg.ports[i].ioat_ids[j]); + ++nb_rawdev; + } + } +end: +if (nb_rawdev < cfg.nb_ports * cfg.ports[0 ].nb_queues) +rte_exit (EXIT_FAILURE , +"Not enough IOAT rawdevs (%u) for all queues (%u).\n" , + nb_rawdev, cfg.nb_ports * cfg.ports[0 ].nb_queues); +RTE_LOG (INFO ,IOAT ,"Number of used rawdevs: %u.\n" , nb_rawdev); +} +static void assign_rings (void ) {
@@ -597,18 +699,29 @@ main(int argc, char **argv)rte_exit (EXIT_FAILURE ,"There should be at least one slave lcore.\n" ); -assign_rings (); +if (copy_mode ==COPY_MODE_IOAT_NUM ) +assign_rawdevs (); +else /* copy_mode == COPY_MODE_SW_NUM */ +assign_rings ();start_forwarding_cores ();/* force_quit is true when we get here */ rte_eal_mp_wait_lcore (); +uint32_t j;for (i =0 ; i < cfg.nb_ports; i++) {printf ("Closing port %d\n" , cfg.ports[i].rxtx_port);rte_eth_dev_stop (cfg.ports[i].rxtx_port);rte_eth_dev_close (cfg.ports[i].rxtx_port); -rte_ring_free (cfg.ports[i].rx_to_tx_ring); +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { +printf ("Stopping rawdev %d\n" , + cfg.ports[i].ioat_ids[j]); +rte_rawdev_stop (cfg.ports[i].ioat_ids[j]); + } + }else /* copy_mode == COPY_MODE_SW_NUM */ +rte_ring_free (cfg.ports[i].rx_to_tx_ring); }printf ("Bye...\n" );
Signature · 1 line
2.21.0
PATCH v7 4/6 examples/ioat: add two threads configuration
In reply to Bruce RichardsonFrom: Pawel Modrak <[email protected]>
Added possibility to use two lcores: first for packet receiving and copying, second for packets sending.
--- examples/ioat/ioatfwd.c | 47 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 43 insertions(+), 4 deletions(-)
examples/ioat/ioatfwd.c +43 -4
@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], nb_rx - i); -return ret; }
@@ -264,6 +263,36 @@ ioat_tx_port(struct rxtx_port_config *tx_config) } } +/* Main rx processing loop for IOAT rawdev. */ +static void +rx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main rx loop for copy on lcore %u\n" , +rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) +ioat_rx_port (&cfg.ports[i]); +} + +/* Main tx processing loop for hardware copy. */ +static void +tx_main_loop (void ) +{ +uint16_t i; +uint16_t nb_ports = cfg.nb_ports; + +RTE_LOG (INFO ,IOAT ,"Entering main tx loop for copy on lcore %u\n" , +rte_lcore_id ()); + +while (!force_quit) +for (i =0 ; i < nb_ports; i++) +ioat_tx_port (&cfg.ports[i]); +} +/* Main rx and tx loop if only one slave lcore available */ static void rxtx_main_loop (void )
@@ -288,9 +317,19 @@ static void start_forwarding_cores(void)RTE_LOG (INFO ,IOAT ,"Entering %s on lcore %u\n" , __func__,rte_lcore_id ()); - lcore_id =rte_get_next_lcore (lcore_id, true, true); -rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, -NULL , lcore_id); +if (cfg.nb_lcores ==1 ) { + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rxtx_main_loop, +NULL , lcore_id); + }else if (cfg.nb_lcores >1 ) { + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)rx_main_loop, +NULL , lcore_id); + + lcore_id =rte_get_next_lcore (lcore_id, true, true); +rte_eal_remote_launch ((lcore_function_t *)tx_main_loop,NULL , + lcore_id); + } }/* Display usage */
Signature · 1 line
2.21.0
Re: [dpdk-dev] [PATCH v7 4/6] examples/ioat: add two threads configuration
07/10/2019 13:08, Bruce Richardson:
examples/ioat/ioatfwd.c ioat_enqueue_packets(struct rte_mbuf **pkts, · line 165
Quoted 10 lines
From: Pawel Modrak <[email protected]>
examples/ioat/ioatfwd.c +0
-1@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,rte_mempool_put_bulk (ioat_pktmbuf_pool, (void *)&pkts_copy[i], nb_rx - i); -return ret; }
This line can be removed in previous patch :)
(will do while merging)
PATCH v7 5/6 examples/ioat: add stats printing for each port
In reply to Bruce RichardsonFrom: Marcin Baran <[email protected]>
Added printing stats of ports each second. The stats printing is done using master core. The information provided informs about packets received, dropped and send as well as statistics of rawdev devices.
--- examples/ioat/ioatfwd.c | 235 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 235 insertions(+)
examples/ioat/ioatfwd.c +235 -0
@@ -48,6 +48,23 @@ struct rxtx_transmission_config {uint16_t nb_lcores; }; +/* per-port statistics struct */ +struct ioat_port_statistics { +uint64_t rx[RTE_MAX_ETHPORTS ]; +uint64_t tx[RTE_MAX_ETHPORTS ]; +uint64_t tx_dropped[RTE_MAX_ETHPORTS ]; +uint64_t copy_dropped[RTE_MAX_ETHPORTS ]; +}; +struct ioat_port_statistics port_statistics; + +struct total_statistics { +uint64_t total_packets_dropped; +uint64_t total_packets_tx; +uint64_t total_packets_rx; +uint64_t total_successful_enqueues; +uint64_t total_failed_enqueues; +}; + typedef enum copy_mode_t { #define COPY_MODE_SW "sw" COPY_MODE_SW_NUM,
@@ -89,6 +106,204 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS ];struct rte_mempool *ioat_pktmbuf_pool; +/* Print out statistics for one port. */ +static void +print_port_stats (uint16_t port_id) +{ +printf ("\nStatistics for port %u ------------------------------" +"\nPackets sent: %34" PRIu64 +"\nPackets received: %30" PRIu64 +"\nPackets dropped on tx: %25" PRIu64 +"\nPackets dropped on copy: %23" PRIu64, + port_id, + port_statistics.tx[port_id], + port_statistics.rx[port_id], + port_statistics.tx_dropped[port_id], + port_statistics.copy_dropped[port_id]); +} + +/* Print out statistics for one IOAT rawdev device. */ +static void +print_rawdev_stats (uint32_t dev_id,uint64_t *xstats, +unsigned int *ids_xstats,uint16_t nb_xstats, +struct rte_rawdev_xstats_name *names_xstats) +{ +uint16_t i; + +printf ("\nIOAT channel %u" , dev_id); +for (i =0 ; i < nb_xstats; i++) +printf ("\n\t %s: %*" PRIu64, + names_xstats[ids_xstats[i]].name, + (int )(37 -strlen (names_xstats[ids_xstats[i]].name)), + xstats[i]); +} + +static void +print_total_stats (struct total_statistics *ts) +{ +printf ("\nAggregate statistics ===============================" +"\nTotal packets Tx: %24" PRIu64" [pps]" +"\nTotal packets Rx: %24" PRIu64" [pps]" +"\nTotal packets dropped: %19" PRIu64" [pps]" , + ts->total_packets_tx, + ts->total_packets_rx, + ts->total_packets_dropped); + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +printf ("\nTotal IOAT successful enqueues: %8" PRIu64" [enq/s]" +"\nTotal IOAT failed enqueues: %12" PRIu64" [enq/s]" , + ts->total_successful_enqueues, + ts->total_failed_enqueues); + } + +printf ("\n====================================================\n" ); +} + +/* Print out statistics on packets dropped. */ +static void +print_stats (char *prgname) +{ +struct total_statistics ts, delta_ts; +uint32_t i, port_id, dev_id; +struct rte_rawdev_xstats_name *names_xstats; +uint64_t *xstats; +unsigned int *ids_xstats, nb_xstats; +char status_string[120 ];/* to print at the top of the output */ +int status_strlen; + + +const char clr[] = {27 ,'[' ,'2' ,'J' ,'\0' }; +const char topLeft[] = {27 ,'[' ,'1' ,';' ,'1' ,'H' ,'\0' }; + + status_strlen =snprintf (status_string,sizeof (status_string), +"%s, " , prgname); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Worker Threads = %d, " , +rte_lcore_count () >2 ?2 :1 ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Copy Mode = %s,\n" , copy_mode ==COPY_MODE_SW_NUM ? +COPY_MODE_SW :COPY_MODE_IOAT ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Updating MAC = %s, " , mac_updating ? +"enabled" :"disabled" ); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Rx Queues = %d, " , nb_queues); + status_strlen +=snprintf (status_string + status_strlen, +sizeof (status_string) - status_strlen, +"Ring Size = %d\n" , ring_size); + +/* Allocate memory for xstats names and values */ + nb_xstats =rte_rawdev_xstats_names_get ( + cfg.ports[0 ].ioat_ids[0 ],NULL ,0 ); + + names_xstats =malloc (sizeof (*names_xstats) * nb_xstats); +if (names_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat names memory\n" ); + } +rte_rawdev_xstats_names_get (cfg.ports[0 ].ioat_ids[0 ], + names_xstats, nb_xstats); + + ids_xstats =malloc (sizeof (*ids_xstats) *2 ); +if (ids_xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat ids_xstats memory\n" ); + } + + xstats =malloc (sizeof (*xstats) *2 ); +if (xstats ==NULL ) { +rte_exit (EXIT_FAILURE , +"Error allocating xstat memory\n" ); + } + +/* Get failed/successful enqueues stats index */ + ids_xstats[0 ] = ids_xstats[1 ] = nb_xstats; +for (i =0 ; i < nb_xstats; i++) { +if (!strcmp (names_xstats[i].name,"failed_enqueues" )) + ids_xstats[0 ] = i; +else if (!strcmp (names_xstats[i].name,"successful_enqueues" )) + ids_xstats[1 ] = i; +if (ids_xstats[0 ] < nb_xstats && ids_xstats[1 ] < nb_xstats) +break ; + } +if (ids_xstats[0 ] == nb_xstats || ids_xstats[1 ] == nb_xstats) { +rte_exit (EXIT_FAILURE , +"Error getting failed/successful enqueues stats index\n" ); + } + +memset (&ts,0 ,sizeof (struct total_statistics )); + +while (!force_quit) { +/* Sleep for 1 second each round - init sleep allows reading +* messages from app startup. +*/ +sleep (1 ); + +/* Clear screen and move to top left */ +printf ("%s%s" , clr, topLeft); + +memset (&delta_ts,0 ,sizeof (struct total_statistics )); + +printf ("%s" , status_string); + +for (i =0 ; i < cfg.nb_ports; i++) { + port_id = cfg.ports[i].rxtx_port; +print_port_stats (port_id); + + delta_ts.total_packets_dropped += + port_statistics.tx_dropped[port_id] + + port_statistics.copy_dropped[port_id]; + delta_ts.total_packets_tx += + port_statistics.tx[port_id]; + delta_ts.total_packets_rx += + port_statistics.rx[port_id]; + +if (copy_mode ==COPY_MODE_IOAT_NUM ) { +uint32_t j; + +for (j =0 ; j < cfg.ports[i].nb_queues; j++) { + dev_id = cfg.ports[i].ioat_ids[j]; +rte_rawdev_xstats_get (dev_id, + ids_xstats, xstats,2 ); + +print_rawdev_stats (dev_id, xstats, + ids_xstats,2 , names_xstats); + + delta_ts.total_failed_enqueues += + xstats[ids_xstats[0 ]]; + delta_ts.total_successful_enqueues += + xstats[ids_xstats[1 ]]; + } + } + } + + delta_ts.total_packets_tx -= ts.total_packets_tx; + delta_ts.total_packets_rx -= ts.total_packets_rx; + delta_ts.total_packets_dropped -= ts.total_packets_dropped; + delta_ts.total_failed_enqueues -= ts.total_failed_enqueues; + delta_ts.total_successful_enqueues -= + ts.total_successful_enqueues; + +printf ("\n" ); +print_total_stats (&delta_ts); + + ts.total_packets_tx += delta_ts.total_packets_tx; + ts.total_packets_rx += delta_ts.total_packets_rx; + ts.total_packets_dropped += delta_ts.total_packets_dropped; + ts.total_failed_enqueues += delta_ts.total_failed_enqueues; + ts.total_successful_enqueues += + delta_ts.total_successful_enqueues; + } + +free (names_xstats); +free (xstats); +free (ids_xstats); +} +static void update_mac_addrs (struct rte_mbuf *m,uint32_t dest_portid) {
@@ -179,6 +394,8 @@ ioat_rx_port(struct rxtx_port_config *rx_config)if (nb_rx ==0 )continue ; + port_statistics.rx[rx_config->rxtx_port] += nb_rx; +if (copy_mode ==COPY_MODE_IOAT_NUM ) {/* Perform packet hardware copy */ nb_enq = ioat_enqueue_packets(pkts_burst,
@@ -213,6 +430,9 @@ ioat_rx_port(struct rxtx_port_config *rx_config) (void *)&pkts_burst_copy[nb_enq], nb_rx - nb_enq); } + + port_statistics.copy_dropped[rx_config->rxtx_port] += + (nb_rx - nb_enq); } }
@@ -255,6 +475,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config) tx_config->rxtx_port,0 , (void *)mbufs_dst, nb_dq); + port_statistics.tx[tx_config->rxtx_port] += nb_tx; +/* Free any unsent packets. */ if (unlikely (nb_tx < nb_dq)) rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -653,6 +875,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)rte_eth_tx_buffer_init (tx_buffer[portid],MAX_PKT_BURST ); + ret =rte_eth_tx_buffer_set_err_callback (tx_buffer[portid], + rte_eth_tx_buffer_count_callback, + &port_statistics.tx_dropped[portid]); +if (ret <0 ) +rte_exit (EXIT_FAILURE , +"Cannot set error callback for tx buffer on port %u\n" , + portid); +/* Start device */ ret =rte_eth_dev_start (portid);if (ret <0 )
@@ -729,6 +959,9 @@ main(int argc, char **argv)RTE_ETH_FOREACH_DEV (portid)port_init (portid, ioat_pktmbuf_pool, nb_queues); +/* Initialize port xstats */ +memset (&port_statistics,0 ,sizeof (port_statistics)); +while (!check_link_status (ioat_enabled_port_mask) && !force_quit)sleep (1 );
@@ -744,6 +977,8 @@ main(int argc, char **argv)assign_rings ();start_forwarding_cores (); +/* master core prints stats while other cores forward */ +print_stats (argv[0 ]);/* force_quit is true when we get here */ rte_eal_mp_wait_lcore ();
Signature · 1 line
2.21.0
PATCH v7 6/6 doc/guides/: provide IOAT sample app guide
In reply to Bruce RichardsonFrom: Marcin Baran <[email protected]>
Added guide for IOAT sample app usage and code description.
--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 581 +++++++++++++++++++++++++++++ 3 files changed, 586 insertions(+) create mode 100644 doc/guides/sample_app_ug/ioat.rst
doc/guides/sample_app_ug/index.rst +1 -0
@@ -23,6 +23,7 @@ Sample Applications User Guides ip_reassembly kernel_nic_interface keep_alive + ioat l2_forward_crypto l2_forward_job_stats l2_forward_real_virtual
doc/guides/sample_app_ug/intro.rst +4 -0
@@ -91,6 +91,10 @@ examples are highlighted below. forwarding, or ``l3fwd`` application does forwarding based on Internet Protocol, IPv4 or IPv6 like a simple router. +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying, + or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for + copying packets between two threads. + * :doc:`Packet Distributor<dist_app>`: The Packet Distributor demonstrates how to distribute packets arriving on an Rx port to different cores for processing and transmission.
doc/guides/sample_app_ug/ioat.rst new file +581 -0
581 lines not on this page · raw message
Signature · 1 line
2.21.0
Re: [dpdk-dev] [PATCH v7 6/6] doc/guides/: provide IOAT sample app guide
07/10/2019 13:08, Bruce Richardson:
Quoted 10 lines
From: Marcin Baran <[email protected]>
Added guide for IOAT sample app usage and code description.
Signed-off-by Marcin Baran <[email protected]>Acked-by Bruce Richardson <[email protected]>--- doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/intro.rst | 4 + doc/guides/sample_app_ug/ioat.rst | 581 +++++++++++++++++++++++++++++
For completeness, this file should be added in MAINTAINERS (will do).
Re: [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver
In reply to Bruce Richardson07/10/2019 13:08, Bruce Richardson:
Quoted 9 lines
Marcin Baran (4):
examples/ioat: add software copy support
examples/ioat: add rawdev copy mode support
examples/ioat: add stats printing for each port
doc/guides/: provide IOAT sample app guidePawel Modrak (2):
examples/ioat: create sample app on ioat driver usage examples/ioat: add two threads configuration
Applied (with minor updates), thanks