Skip to contentSkip to conversation

Miscellaneous fixes

PATCH Superseded Thierry Herbelot 6 participants 46 messages Aug 7, 2019 – Oct 10, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Miscellaneous fixes

This series of patches includes fixes for issues seen with 6WIND fast path, built on DPDK.
The patches are in 6WIND version of DPDK, and should have been upstreamed a long time ago.

Guo Fengtian (1):
net/ixgbevf: fix stats update after a PF reset

Laurent Hardy (1):
net/i40e: set speed to undefined for default case in link update

Olivier Matz (5):
ethdev: fix description of tx descriptor status
net/e1000: fix Tx descriptor status api (igb)
net/e1000: fix Tx descriptor status api (em)
net/ixgbe: fix Tx descriptor status api
net/i40e: fix Tx descriptor status api

Thibaut Collet (2):
virtio: fix rx stats with vectorized functions
virtio: get all pending rx packets with vectorized functions

Thierry Herbelot (3):
drivers/crypto/openssl: use a local copy for the session contexts drivers/crypto/dpaa_sec: update DPAA iova table in dpaa_mem_vtop drivers/crypto/octeontx: enable unbinding for the OcteonTx crypto
engines

 drivers/crypto/dpaa_sec/dpaa_sec.c           |  5 +++-
 drivers/crypto/octeontx/otx_cryptodev.c      |  1 +
 drivers/crypto/openssl/rte_openssl_pmd.c     | 34 +++++++++++++++------
 drivers/net/e1000/em_rxtx.c                  | 33 +++++++++++++-------
 drivers/net/e1000/igb_rxtx.c                 |  9 +++---
 drivers/net/i40e/i40e_ethdev.c               |  4 +--
 drivers/net/i40e/i40e_ethdev_vf.c            |  8 +++--
 drivers/net/i40e/i40e_rxtx.c                 | 37 ++++++++++++++++-------
 drivers/net/ixgbe/ixgbe_ethdev.c             |  6 ++--
 drivers/net/ixgbe/ixgbe_rxtx.c               | 45 ++++++++++++++++++++--------
 drivers/net/ixgbe/ixgbe_rxtx.h               |  1 +
 drivers/net/virtio/virtio_rxtx.c             |  2 +-
 drivers/net/virtio/virtio_rxtx.h             |  2 ++
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 11 +++++--
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 11 +++++--
 lib/librte_ethdev/rte_ethdev.h               |  4 +--
 16 files changed, 153 insertions(+), 60 deletions(-)
Signature · 1 line
2.11.0
#1 lore raw

PATCH 1/12 net/ixgbevf: fix stats update after a PF reset

From: Guo Fengtian <[email protected]>

When PF is set down, in VF, the value of stats register is zero. So only increase stats when it's non zero.

Fixes af75078fece3 ('first public release')
Cc stable at dpdk.org
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/net/ixgbe/ixgbe_ethdev.c +4 -2
index 03fc1f71799c..57f5bfa219c1 100644
@@ -385,7 +385,8 @@ static void ixgbe_l2_tunnel_conf(struct rte_eth_dev *dev);
 #define UPDATE_VF_STAT(reg, last, cur)                          \
 {                                                               \
 	uint32_t latest = IXGBE_READ_REG(hw, reg);              \
-	cur += (latest - last) & UINT_MAX;                      \
+	if (latest)                                             \
+		cur += (latest - last) & UINT_MAX;              \
 	last = latest;                                          \
 }
 
@@ -394,7 +395,8 @@ static void ixgbe_l2_tunnel_conf(struct rte_eth_dev *dev);
 	u64 new_lsb = IXGBE_READ_REG(hw, lsb);                   \
 	u64 new_msb = IXGBE_READ_REG(hw, msb);                   \
 	u64 latest = ((new_msb << 32) | new_lsb);                \
-	cur += (0x1000000000LL + latest - last) & 0xFFFFFFFFFLL; \
+	if (latest)                                              \
+		cur += (0x1000000000LL + latest - last) & 0xFFFFFFFFFLL;\
 	last = latest;                                           \
 }
 
Signature · 1 line
2.11.0
#2 lore raw

PATCH 2/12 ethdev: fix description of tx descriptor status

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The API comment of rte_eth_tx_descriptor_status() was incorrect. The reference descriptor (when offset = 0) is not where the next packet will be sent, but where the latest packet has been enqueued.

Fixes 52f5cdd2e897 ("ethdev: add descriptor status API")
Cc stable at dpdk.org
---
 lib/librte_ethdev/rte_ethdev.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_ethdev/rte_ethdev.h +2 -2
index dc6596bc93b4..b423e71050e9 100644
@@ -4245,8 +4245,8 @@ rte_eth_rx_descriptor_status(uint16_t port_id, uint16_t queue_id,
  * @param queue_id
  *  A valid Tx queue identifier on this port.
  * @param offset
- *  The offset of the descriptor starting from tail (0 is the place where
- *  the next packet will be send).
+ *  The offset of the descriptor starting from tail (0 is the last written
+ *  descriptor).
  *
  * @return
  *  - (RTE_ETH_TX_DESC_FULL) Descriptor is being processed by the hw, i.e.
Signature · 1 line
2.11.0
#3 lore raw

PATCH 3/12 net/e1000: fix Tx descriptor status api (igb)

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The Tx descriptor status api was not behaving as expected. This API is used to inspect the content of the descriptors in the Tx ring to determine the length of the Tx queue.

Since the software advances the tail pointer and the hardware advances the head pointer, the Tx queue is located before txq->tx_tail in the ring. Therefore, a call to rte_eth_tx_descriptor_status(..., offset=20) should inspect the 20th descriptor before the tail, not after.

Fixes 978f8eea1719 ("net/e1000: implement descriptor status API (igb)")
Cc stable at dpdk.org
---
 drivers/net/e1000/igb_rxtx.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/net/e1000/igb_rxtx.c +5 -4
index c5606de5d7a0..c22118e59a21 100644
@@ -1835,14 +1835,15 @@ eth_igb_tx_descriptor_status(void *tx_queue, uint16_t offset)
 {
 	struct igb_tx_queue *txq = tx_queue;
 	volatile uint32_t *status;
-	uint32_t desc;
+	int32_t desc;
 
 	if (unlikely(offset >= txq->nb_tx_desc))
 		return -EINVAL;
 
-	desc = txq->tx_tail + offset;
-	if (desc >= txq->nb_tx_desc)
-		desc -= txq->nb_tx_desc;
+	desc = txq->tx_tail - offset - 1;
+	if (desc < 0)
+		desc += txq->nb_tx_desc;
+	desc = txq->sw_ring[desc].last_id;
 
 	status = &txq->tx_ring[desc].wb.status;
 	if (*status & rte_cpu_to_le_32(E1000_TXD_STAT_DD))
Signature · 1 line
2.11.0
#4 lore raw

PATCH 4/12 net/e1000: fix Tx descriptor status api (em)

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The Tx descriptor status api was not behaving as expected. This API is used to inspect the content of the descriptors in the Tx ring to determine the length of the Tx queue.

Since the software advances the tail pointer and the hardware advances the head pointer, the Tx queue is located before txq->tx_tail in the ring. Therefore, a call to rte_eth_tx_descriptor_status(..., offset=20) should inspect the 20th descriptor before the tail, not after.

As before, we still need to take care about only checking descriptors that have the RS bit.

Additionally, we can avoid an access to the ring if offset is greater or equal to nb_tx_desc - nb_tx_free.

Fixes b9082317cab3 ("net/e1000: implement descriptor status API (em)")
Cc stable at dpdk.org
---
 drivers/net/e1000/em_rxtx.c | 33 +++++++++++++++++++++++----------
 1 file changed, 23 insertions(+), 10 deletions(-)
drivers/net/e1000/em_rxtx.c +23 -10
index 5925e490641b..3061998c7768 100644
@@ -152,6 +152,7 @@ struct em_tx_queue {
 	uint64_t               tx_ring_phys_addr; /**< TX ring DMA address. */
 	struct em_tx_entry    *sw_ring; /**< virtual address of SW ring. */
 	volatile uint32_t      *tdt_reg_addr; /**< Address of TDT register. */
+	volatile uint32_t      *tdh_reg_addr; /**< Address of TDH register. */
 	uint16_t               nb_tx_desc;    /**< number of TX descriptors. */
 	uint16_t               tx_tail;  /**< Current value of TDT register. */
 	/**< Start freeing TX buffers if there are less free descriptors than
@@ -1304,6 +1305,7 @@ eth_em_tx_queue_setup(struct rte_eth_dev *dev,
 	txq->port_id = dev->data->port_id;
 
 	txq->tdt_reg_addr = E1000_PCI_REG_ADDR(hw, E1000_TDT(queue_idx));
+	txq->tdh_reg_addr = E1000_PCI_REG_ADDR(hw, E1000_TDH(queue_idx));
 	txq->tx_ring_phys_addr = tz->iova;
 	txq->tx_ring = (struct e1000_data_desc *) tz->addr;
 
@@ -1557,22 +1559,33 @@ eth_em_tx_descriptor_status(void *tx_queue, uint16_t offset)
 {
 	struct em_tx_queue *txq = tx_queue;
 	volatile uint8_t *status;
-	uint32_t desc;
+	int32_t desc, dd;
 
 	if (unlikely(offset >= txq->nb_tx_desc))
 		return -EINVAL;
+	if (offset >= txq->nb_tx_desc - txq->nb_tx_free)
+		return RTE_ETH_TX_DESC_DONE;
 
-	desc = txq->tx_tail + offset;
-	/* go to next desc that has the RS bit */
-	desc = ((desc + txq->tx_rs_thresh - 1) / txq->tx_rs_thresh) *
-		txq->tx_rs_thresh;
-	if (desc >= txq->nb_tx_desc) {
-		desc -= txq->nb_tx_desc;
-		if (desc >= txq->nb_tx_desc)
-			desc -= txq->nb_tx_desc;
+	desc = txq->tx_tail - offset - 1;
+	if (desc < 0)
+		desc += txq->nb_tx_desc;
+
+	/* offset is too small, no other way than reading PCI reg */
+	if (unlikely(offset < txq->tx_rs_thresh)) {
+		int16_t tx_head, queue_size;
+		tx_head = e1000_read_addr(txq->tdh_reg_addr);
+		queue_size = txq->tx_tail - tx_head;
+		if (queue_size < 0)
+			queue_size += txq->nb_tx_desc;
+		return queue_size > offset ? RTE_ETH_TX_DESC_FULL :
+			RTE_ETH_TX_DESC_DONE;
 	}
 
-	status = &txq->tx_ring[desc].upper.fields.status;
+	/* index of the dd bit to look at */
+	dd = (desc / txq->tx_rs_thresh + 1) * txq->tx_rs_thresh - 1;
+	dd = txq->sw_ring[dd].last_id;
+
+	status = &txq->tx_ring[dd].upper.fields.status;
 	if (*status & E1000_TXD_STAT_DD)
 		return RTE_ETH_TX_DESC_DONE;
 
Signature · 1 line
2.11.0
#5 lore raw

PATCH 5/12 net/ixgbe: fix Tx descriptor status api

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The Tx descriptor status api was not behaving as expected. This API is used to inspect the content of the descriptors in the Tx ring to determine the length of the Tx queue.

Since the software advances the tail pointer and the hardware advances the head pointer, the Tx queue is located before txq->tx_tail in the ring. Therefore, a call to rte_eth_tx_descriptor_status(..., offset=20) should inspect the 20th descriptor before the tail, not after.

As before, we still need to take care about only checking descriptors that have the RS bit.

Additionally, we can avoid an access to the ring if offset is greater or equal to nb_tx_desc - nb_tx_free.

Fixes 5da8b8814178 ("net/ixgbe: implement descriptor status API")
Cc stable at dpdk.org
---
 drivers/net/ixgbe/ixgbe_rxtx.c | 45 +++++++++++++++++++++++++++++++-----------
 drivers/net/ixgbe/ixgbe_rxtx.h |  1 +
 2 files changed, 34 insertions(+), 12 deletions(-)
drivers/net/ixgbe/ixgbe_rxtx.c +33 -12
index edcfa60cec98..68e3aea5ed46 100644
@@ -2627,10 +2627,15 @@ ixgbe_dev_tx_queue_setup(struct rte_eth_dev *dev,
 	    hw->mac.type == ixgbe_mac_X540_vf ||
 	    hw->mac.type == ixgbe_mac_X550_vf ||
 	    hw->mac.type == ixgbe_mac_X550EM_x_vf ||
-	    hw->mac.type == ixgbe_mac_X550EM_a_vf)
+	    hw->mac.type == ixgbe_mac_X550EM_a_vf) {
 		txq->tdt_reg_addr = IXGBE_PCI_REG_ADDR(hw, IXGBE_VFTDT(queue_idx));
-	else
+		txq->tdh_reg_addr = IXGBE_PCI_REG_ADDR(hw,
+				IXGBE_VFTDH(queue_idx));
+	} else {
 		txq->tdt_reg_addr = IXGBE_PCI_REG_ADDR(hw, IXGBE_TDT(txq->reg_idx));
+		txq->tdh_reg_addr = IXGBE_PCI_REG_ADDR(hw,
+				IXGBE_TDH(txq->reg_idx));
+	}
 
 	txq->tx_ring_phys_addr = tz->iova;
 	txq->tx_ring = (union ixgbe_adv_tx_desc *) tz->addr;
@@ -3163,22 +3168,38 @@ ixgbe_dev_tx_descriptor_status(void *tx_queue, uint16_t offset)
 {
 	struct ixgbe_tx_queue *txq = tx_queue;
 	volatile uint32_t *status;
-	uint32_t desc;
+	int32_t desc, dd;
 
 	if (unlikely(offset >= txq->nb_tx_desc))
 		return -EINVAL;
+	if (offset >= txq->nb_tx_desc - txq->nb_tx_free)
+		return RTE_ETH_TX_DESC_DONE;
+
+	desc = txq->tx_tail - offset - 1;
+	if (desc < 0)
+		desc += txq->nb_tx_desc;
 
-	desc = txq->tx_tail + offset;
-	/* go to next desc that has the RS bit */
-	desc = ((desc + txq->tx_rs_thresh - 1) / txq->tx_rs_thresh) *
-		txq->tx_rs_thresh;
-	if (desc >= txq->nb_tx_desc) {
-		desc -= txq->nb_tx_desc;
-		if (desc >= txq->nb_tx_desc)
-			desc -= txq->nb_tx_desc;
+	/* offset is too small, no other way than reading PCI reg */
+	if (unlikely(offset < txq->tx_rs_thresh)) {
+		int16_t tx_head, queue_size;
+		tx_head = ixgbe_read_addr(txq->tdh_reg_addr);
+		queue_size = txq->tx_tail - tx_head;
+		if (queue_size < 0)
+			queue_size += txq->nb_tx_desc;
+		return queue_size > offset ? RTE_ETH_TX_DESC_FULL :
+			RTE_ETH_TX_DESC_DONE;
 	}
 
-	status = &txq->tx_ring[desc].wb.status;
+	/* index of the dd bit to look at */
+	dd = (desc / txq->tx_rs_thresh + 1) * txq->tx_rs_thresh - 1;
+
+	/* In full featured mode, RS bit is only set in the last descriptor */
+	/* of a multisegments packet */
+	if (!((txq->offloads == 0) &&
+	      (txq->tx_rs_thresh >= RTE_PMD_IXGBE_TX_MAX_BURST)))
+		dd = txq->sw_ring[dd].last_id;
+
+	status = &txq->tx_ring[dd].wb.status;
 	if (*status & rte_cpu_to_le_32(IXGBE_ADVTXD_STAT_DD))
 		return RTE_ETH_TX_DESC_DONE;
 
drivers/net/ixgbe/ixgbe_rxtx.h +1 -0
index 505d344b9cee..05fd4167576c 100644
@@ -201,6 +201,7 @@ struct ixgbe_tx_queue {
 		struct ixgbe_tx_entry_v *sw_ring_v; /**< address of SW ring for vector PMD */
 	};
 	volatile uint32_t   *tdt_reg_addr; /**< Address of TDT register. */
+	volatile uint32_t   *tdh_reg_addr; /**< Address of TDH register. */
 	uint16_t            nb_tx_desc;    /**< number of TX descriptors. */
 	uint16_t            tx_tail;       /**< current value of TDT reg. */
 	/**< Start freeing TX buffers if there are less free descriptors than
Signature · 1 line
2.11.0
#6 lore raw

PATCH 6/12 net/i40e: fix Tx descriptor status api

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The Tx descriptor status api was not behaving as expected. This API is used to inspect the content of the descriptors in the Tx ring to determine the length of the Tx queue.

Since the software advances the tail pointer and the hardware advances the head pointer, the Tx queue is located before txq->tx_tail in the ring. Therefore, a call to rte_eth_tx_descriptor_status(..., offset=20) should inspect the 20th descriptor before the tail, not after.

As before, we still need to take care about only checking descriptors that have the RS bit.

Additionally, we can avoid an access to the ring if offset is greater or equal to nb_tx_desc - nb_tx_free.

Fixes a9dd9af6f38e ("net/i40e: implement descriptor status API")
Cc stable at dpdk.org
---
 drivers/net/i40e/i40e_rxtx.c | 37 +++++++++++++++++++++++++++----------
 1 file changed, 27 insertions(+), 10 deletions(-)
drivers/net/i40e/i40e_rxtx.c +27 -10
index 692c3bab4b5f..4fbbc097ed4f 100644
@@ -2031,22 +2031,39 @@ i40e_dev_tx_descriptor_status(void *tx_queue, uint16_t offset)
 	struct i40e_tx_queue *txq = tx_queue;
 	volatile uint64_t *status;
 	uint64_t mask, expect;
-	uint32_t desc;
+	int32_t desc, dd;
 
 	if (unlikely(offset >= txq->nb_tx_desc))
 		return -EINVAL;
+	if (offset >= txq->nb_tx_desc - txq->nb_tx_free)
+		return RTE_ETH_TX_DESC_DONE;
+
+	desc = txq->tx_tail - offset - 1;
+	if (desc < 0)
+		desc += txq->nb_tx_desc;
 
-	desc = txq->tx_tail + offset;
-	/* go to next desc that has the RS bit */
-	desc = ((desc + txq->tx_rs_thresh - 1) / txq->tx_rs_thresh) *
-		txq->tx_rs_thresh;
-	if (desc >= txq->nb_tx_desc) {
-		desc -= txq->nb_tx_desc;
-		if (desc >= txq->nb_tx_desc)
-			desc -= txq->nb_tx_desc;
+	/* offset is too small, no other way than reading PCI reg */
+	if (unlikely(offset < txq->tx_rs_thresh)) {
+		int16_t tx_head, queue_size;
+		tx_head = I40E_READ_REG(I40E_VSI_TO_HW(txq->vsi),
+					I40E_QTX_HEAD(txq->reg_idx));
+		queue_size = txq->tx_tail - tx_head;
+		if (queue_size < 0)
+			queue_size += txq->nb_tx_desc;
+		return queue_size > offset ? RTE_ETH_TX_DESC_FULL :
+			RTE_ETH_TX_DESC_DONE;
 	}
 
-	status = &txq->tx_ring[desc].cmd_type_offset_bsz;
+	/* index of the dd bit to look at */
+	dd = (desc / txq->tx_rs_thresh + 1) * txq->tx_rs_thresh - 1;
+
+	/* In full featured mode, RS bit is only set in the last descriptor */
+	/* of a multisegments packet */
+	if (!((txq->offloads == 0) &&
+	      (txq->tx_rs_thresh >= RTE_PMD_I40E_TX_MAX_BURST)))
+		dd = txq->sw_ring[dd].last_id;
+
+	status = &txq->tx_ring[dd].cmd_type_offset_bsz;
 	mask = rte_le_to_cpu_64(I40E_TXD_QW1_DTYPE_MASK);
 	expect = rte_cpu_to_le_64(
 		I40E_TX_DESC_DTYPE_DESC_DONE << I40E_TXD_QW1_DTYPE_SHIFT);
Signature · 1 line
2.11.0
#7 lore raw

PATCH 7/12 net/i40e: set speed to undefined for default case in link update

In reply to Thierry Herbelot

From: Laurent Hardy <[email protected]>

During PF/VF link update, a default speed value of 100M will be set if get_link_info has failed or speed is unknown.

Consequently if PF is put in no-carrier state, VFs will switch to "in carrier" state due to a link up + a link speed set to 100M (default value if no speed detected).

To be consistent with linux drivers on which PF and VFs are in same carrier state, sets default speed to undefined (instead of 100M) and updates a link status of VF only if link is up and speed is different from undefined.

Fixes 4861cde46116 ('i40e: new poll mode driver')
Cc stable at dpdk.org
---
 drivers/net/i40e/i40e_ethdev.c    | 4 ++--
 drivers/net/i40e/i40e_ethdev_vf.c | 8 +++++---
 2 files changed, 7 insertions(+), 5 deletions(-)
drivers/net/i40e/i40e_ethdev.c +2 -2
index 4e40b7ab5250..76abe8209a10 100644
@@ -2743,7 +2743,7 @@ update_link_aq(struct i40e_hw *hw, struct rte_eth_link *link,
 		status = i40e_aq_get_link_info(hw, enable_lse,
 						&link_status, NULL);
 		if (unlikely(status != I40E_SUCCESS)) {
-			link->link_speed = ETH_SPEED_NUM_100M;
+			link->link_speed = ETH_SPEED_NUM_NONE;
 			link->link_duplex = ETH_LINK_FULL_DUPLEX;
 			PMD_DRV_LOG(ERR, "Failed to get link info");
 			return;
@@ -2777,7 +2777,7 @@ update_link_aq(struct i40e_hw *hw, struct rte_eth_link *link,
 		link->link_speed = ETH_SPEED_NUM_40G;
 		break;
 	default:
-		link->link_speed = ETH_SPEED_NUM_100M;
+		link->link_speed = ETH_SPEED_NUM_NONE;
 		break;
 	}
 }
drivers/net/i40e/i40e_ethdev_vf.c +5 -3
index 308fb9835ab1..9ba351710b7a 100644
@@ -2143,13 +2143,15 @@ i40evf_dev_link_update(struct rte_eth_dev *dev,
 		new_link.link_speed = ETH_SPEED_NUM_40G;
 		break;
 	default:
-		new_link.link_speed = ETH_SPEED_NUM_100M;
+		new_link.link_speed = ETH_SPEED_NUM_NONE;
 		break;
 	}
 	/* full duplex only */
 	new_link.link_duplex = ETH_LINK_FULL_DUPLEX;
-	new_link.link_status = vf->link_up ? ETH_LINK_UP :
-					     ETH_LINK_DOWN;
+	new_link.link_status = vf->link_up
+				&& new_link.link_speed != ETH_SPEED_NUM_NONE
+				? ETH_LINK_UP
+				: ETH_LINK_DOWN;
 	new_link.link_autoneg =
 		!(dev->data->dev_conf.link_speeds & ETH_LINK_SPEED_FIXED);
 
Signature · 1 line
2.11.0
#8 lore raw

PATCH 8/12 virtio: fix rx stats with vectorized functions

In reply to Thierry Herbelot

From: Thibaut Collet <[email protected]>

With vectorized functions, only the rx stats for number of packets is incremented.
Update also the other statistics.
Performance impact is about 2%

Fixes fc3d66212fed ("virtio: add vector Rx")
Cc stable at dpdk.org
---
 drivers/net/virtio/virtio_rxtx.c             | 2 +-
 drivers/net/virtio/virtio_rxtx.h             | 2 ++
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 6 ++++++
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 6 ++++++
 4 files changed, 15 insertions(+), 1 deletion(-)
drivers/net/virtio/virtio_rxtx.c +1 -1
index 27ead19fbe81..6dd62bf51863 100644
@@ -1083,7 +1083,7 @@ virtio_discard_rxbuf_inorder(struct virtqueue *vq, struct rte_mbuf *m)
 	}
 }
 
-static inline void
+void
 virtio_update_packet_stats(struct virtnet_stats *stats, struct rte_mbuf *mbuf)
 {
 	uint32_t s = mbuf->pkt_len;
drivers/net/virtio/virtio_rxtx.h +2 -0
index 685cc4f8104c..1eb8dae227ee 100644
@@ -59,5 +59,7 @@ struct virtnet_ctl {
 };
 
 int virtio_rxq_vec_setup(struct virtnet_rx *rxvq);
+void virtio_update_packet_stats(struct virtnet_stats *stats,
+				struct rte_mbuf *mbuf);
 
 #endif /* _VIRTIO_RXTX_H_ */
drivers/net/virtio/virtio_rxtx_simple_neon.c +6 -0
index cdc2a4d28ed5..e4b18cba0ce5 100644
@@ -47,6 +47,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
 	struct rte_mbuf **sw_ring_end;
+	struct rte_mbuf **ref_rx_pkts;
 	uint16_t nb_pkts_received = 0;
 
 	uint8x16_t shuf_msk1 = {
@@ -105,6 +106,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
 		nb_pkts_received < nb_used;) {
 		uint64x2_t desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
@@ -204,5 +206,9 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	vq->vq_used_cons_idx += nb_pkts_received;
 	vq->vq_free_cnt += nb_pkts_received;
 	rxvq->stats.packets += nb_pkts_received;
+	for (nb_used = 0; nb_used < nb_pkts_received; nb_used++) {
+		rxvq->stats.bytes += ref_rx_pkts[nb_used]->pkt_len;
+		virtio_update_packet_stats(&rxvq->stats, ref_rx_pkts[nb_used]);
+	}
 	return nb_pkts_received;
 }
drivers/net/virtio/virtio_rxtx_simple_sse.c +6 -0
index af76708d66ae..c757e8c9d601 100644
@@ -48,6 +48,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
 	struct rte_mbuf **sw_ring_end;
+	struct rte_mbuf **ref_rx_pkts;
 	uint16_t nb_pkts_received = 0;
 	__m128i shuf_msk1, shuf_msk2, len_adjust;
 
@@ -107,6 +108,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
 		nb_pkts_received < nb_used;) {
 		__m128i desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
@@ -190,5 +192,9 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	vq->vq_used_cons_idx += nb_pkts_received;
 	vq->vq_free_cnt += nb_pkts_received;
 	rxvq->stats.packets += nb_pkts_received;
+	for (nb_used = 0; nb_used < nb_pkts_received; nb_used++) {
+		rxvq->stats.bytes += ref_rx_pkts[nb_used]->pkt_len;
+		virtio_update_packet_stats(&rxvq->stats, ref_rx_pkts[nb_used]);
+	}
 	return nb_pkts_received;
 }
Signature · 1 line
2.11.0
#9 lore raw

PATCH 9/12 virtio: get all pending rx packets with vectorized functions

In reply to Thierry Herbelot

From: Thibaut Collet <[email protected]>

The loop to read packets does not take all packets as the number of available packets (nb_used) is decremented in the loop. Take all available packets provides a performance improvement of 3%.

Fixes fc3d66212fed ("virtio: add vector Rx")
Cc stable at dpdk.org
---
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 5 +++--
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 5 +++--
 2 files changed, 6 insertions(+), 4 deletions(-)
drivers/net/virtio/virtio_rxtx_simple_neon.c +3 -2
index e4b18cba0ce5..66bdaa00e01f 100644
@@ -42,7 +42,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct virtnet_rx *rxvq = rx_queue;
 	struct virtqueue *vq = rxvq->vq;
 	struct virtio_hw *hw = vq->hw;
-	uint16_t nb_used;
+	uint16_t nb_used, nb_total;
 	uint16_t desc_idx;
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
@@ -106,9 +106,10 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	nb_total = nb_used;
 	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
-		nb_pkts_received < nb_used;) {
+		nb_pkts_received < nb_total;) {
 		uint64x2_t desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		uint64x2_t mbp[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		uint64x2_t pkt_mb[RTE_VIRTIO_DESC_PER_LOOP];
drivers/net/virtio/virtio_rxtx_simple_sse.c +3 -2
index c757e8c9d601..811b416755e7 100644
@@ -43,7 +43,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct virtnet_rx *rxvq = rx_queue;
 	struct virtqueue *vq = rxvq->vq;
 	struct virtio_hw *hw = vq->hw;
-	uint16_t nb_used;
+	uint16_t nb_used, nb_total;
 	uint16_t desc_idx;
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
@@ -108,9 +108,10 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	nb_total = nb_used;
 	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
-		nb_pkts_received < nb_used;) {
+		nb_pkts_received < nb_total;) {
 		__m128i desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		__m128i mbp[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		__m128i pkt_mb[RTE_VIRTIO_DESC_PER_LOOP];
Signature · 1 line
2.11.0
#10 lore raw

PATCH 10/12 drivers/crypto/openssl: use a local copy for the session contexts

In reply to Thierry Herbelot

Session contexts are used for temporary storage when processing a packet.
If packets for the same session are to be processed simultaneously on multiple cores, separate contexts must be used.

Note: with openssl 1.1.1 EVP_CIPHER_CTX can no longer be defined as a variable on the stack: it must be allocated. This in turn reduces the performance.

Fixes d61f70b4c918 ('crypto/libcrypto: add driver for OpenSSL library')
Cc stable at dpdk.org
---
 drivers/crypto/openssl/rte_openssl_pmd.c | 34 +++++++++++++++++++++++---------
 1 file changed, 25 insertions(+), 9 deletions(-)
drivers/crypto/openssl/rte_openssl_pmd.c +25 -9
index 2f5552840741..ce2d12347737 100644
@@ -1290,6 +1290,7 @@ process_openssl_combined_op
 	int srclen, aadlen, status = -1;
 	uint32_t offset;
 	uint8_t taglen;
+	EVP_CIPHER_CTX *ctx_copy;
 
 	/*
 	 * Segmented destination buffer is not supported for
@@ -1326,6 +1327,8 @@ process_openssl_combined_op
 	}
 
 	taglen = sess->auth.digest_length;
+	ctx_copy = EVP_CIPHER_CTX_new();
+	EVP_CIPHER_CTX_copy(ctx_copy, sess->cipher.ctx);
 
 	if (sess->cipher.direction == RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
 		if (sess->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC ||
@@ -1333,12 +1336,12 @@ process_openssl_combined_op
 			status = process_openssl_auth_encryption_gcm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, sess->cipher.ctx);
+					dst, tag, ctx_copy);
 		else
 			status = process_openssl_auth_encryption_ccm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, taglen, sess->cipher.ctx);
+					dst, tag, taglen, ctx_copy);
 
 	} else {
 		if (sess->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC ||
@@ -1346,14 +1349,15 @@ process_openssl_combined_op
 			status = process_openssl_auth_decryption_gcm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, sess->cipher.ctx);
+					dst, tag, ctx_copy);
 		else
 			status = process_openssl_auth_decryption_ccm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, taglen, sess->cipher.ctx);
+					dst, tag, taglen, ctx_copy);
 	}
 
+	EVP_CIPHER_CTX_free(ctx_copy);
 	if (status != 0) {
 		if (status == (-EFAULT) &&
 				sess->auth.operation ==
@@ -1372,6 +1376,7 @@ process_openssl_cipher_op
 {
 	uint8_t *dst, *iv;
 	int srclen, status;
+	EVP_CIPHER_CTX *ctx_copy;
 
 	/*
 	 * Segmented destination buffer is not supported for
@@ -1388,22 +1393,25 @@ process_openssl_cipher_op
 
 	iv = rte_crypto_op_ctod_offset(op, uint8_t *,
 			sess->iv.offset);
+	ctx_copy = EVP_CIPHER_CTX_new();
+	EVP_CIPHER_CTX_copy(ctx_copy, sess->cipher.ctx);
 
 	if (sess->cipher.mode == OPENSSL_CIPHER_LIB)
 		if (sess->cipher.direction == RTE_CRYPTO_CIPHER_OP_ENCRYPT)
 			status = process_openssl_cipher_encrypt(mbuf_src, dst,
 					op->sym->cipher.data.offset, iv,
-					srclen, sess->cipher.ctx);
+					srclen, ctx_copy);
 		else
 			status = process_openssl_cipher_decrypt(mbuf_src, dst,
 					op->sym->cipher.data.offset, iv,
-					srclen, sess->cipher.ctx);
+					srclen, ctx_copy);
 	else
 		status = process_openssl_cipher_des3ctr(mbuf_src, dst,
 				op->sym->cipher.data.offset, iv,
 				sess->cipher.key.data, srclen,
-				sess->cipher.ctx);
+				ctx_copy);
 
+	EVP_CIPHER_CTX_free(ctx_copy);
 	if (status != 0)
 		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
 }
@@ -1507,6 +1515,8 @@ process_openssl_auth_op(struct openssl_qp *qp, struct rte_crypto_op *op,
 {
 	uint8_t *dst;
 	int srclen, status;
+	EVP_MD_CTX *ctx_a;
+	HMAC_CTX *ctx_h;
 
 	srclen = op->sym->auth.data.length;
 
@@ -1514,14 +1524,20 @@ process_openssl_auth_op(struct openssl_qp *qp, struct rte_crypto_op *op,
 
 	switch (sess->auth.mode) {
 	case OPENSSL_AUTH_AS_AUTH:
+		ctx_a = EVP_MD_CTX_create();
+		EVP_MD_CTX_copy_ex(ctx_a, sess->auth.auth.ctx);
 		status = process_openssl_auth(mbuf_src, dst,
 				op->sym->auth.data.offset, NULL, NULL, srclen,
-				sess->auth.auth.ctx, sess->auth.auth.evp_algo);
+				ctx_a, sess->auth.auth.evp_algo);
+		EVP_MD_CTX_destroy(ctx_a);
 		break;
 	case OPENSSL_AUTH_AS_HMAC:
+		ctx_h = HMAC_CTX_new();
+		HMAC_CTX_copy(ctx_h, sess->auth.hmac.ctx);
 		status = process_openssl_auth_hmac(mbuf_src, dst,
 				op->sym->auth.data.offset, srclen,
-				sess->auth.hmac.ctx);
+				ctx_h);
+		HMAC_CTX_free(ctx_h);
 		break;
 	default:
 		status = -1;
Signature · 1 line
2.11.0
#11 lore raw

PATCH 11/12 drivers/crypto/dpaa_sec: update DPAA iova table in dpaa_mem_vtop

In reply to Thierry Herbelot

dpaa_sec needs translations between physical and virtual addresses. V to P translation is relatively fast, as memory is managed in contiguous segments.

The result of each V to P translation is used to update the DPAA iova table, which should be updated by a Mem event callback, but is not. Then the DPAA iova table has entries for all needed memory ranges.

With this patch, dpaa_mem_ptov will always use dpaax_iova_table_get_va, which ensures optimal performance.

Fixes 5a7dbb934d75 ('dpaa: enable dpaax library')
Cc stable at dpdk.org
---
 drivers/crypto/dpaa_sec/dpaa_sec.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/crypto/dpaa_sec/dpaa_sec.c +4 -1
index 122c80a072ff..22b8b1d63ce0 100644
@@ -38,6 +38,7 @@
 #include <rte_dpaa_bus.h>
 #include <dpaa_sec.h>
 #include <dpaa_sec_log.h>
+#include <dpaax_iova_table.h>
 
 enum rta_sec_era rta_sec_era;
 
@@ -100,8 +101,10 @@ dpaa_mem_vtop(void *vaddr)
 	const struct rte_memseg *ms;
 
 	ms = rte_mem_virt2memseg(vaddr, NULL);
-	if (ms)
+	if (ms) {
+		dpaax_iova_table_update(ms->iova, (void *)ms->addr_64, ms->len);
 		return ms->iova + RTE_PTR_DIFF(vaddr, ms->addr);
+	}
 	return (size_t)NULL;
 }
 
Signature · 1 line
2.11.0
#12 lore raw

PATCH 12/12 drivers/crypto/octeontx: enable unbinding for the OcteonTx crypto engines

In reply to Thierry Herbelot

Like for Ethernet ports, the OcteonTx crypto engines must first be unbound from their kernel module, then rebound to vfio-pci, before being usable in DPDK.

As this capability is detected at runtime by dpdk-pmdinfo, add the info in the PMD registering directives.

Then an external script can be used for bind and unbind.

Fixes bfe2ae495ee268 ('crypto/octeontx: add PMD skeleton')
Cc stable at dpdk.org
---
 drivers/crypto/octeontx/otx_cryptodev.c | 1 +
 1 file changed, 1 insertion(+)
drivers/crypto/octeontx/otx_cryptodev.c +1 -0
index fc64a5f3041f..16f1909966d0 100644
@@ -118,6 +118,7 @@ static struct cryptodev_driver otx_cryptodev_drv;
 
 RTE_PMD_REGISTER_PCI(CRYPTODEV_NAME_OCTEONTX_PMD, otx_cryptodev_pmd);
 RTE_PMD_REGISTER_PCI_TABLE(CRYPTODEV_NAME_OCTEONTX_PMD, pci_id_cpt_table);
+RTE_PMD_REGISTER_KMOD_DEP(CRYPTODEV_NAME_OCTEONTX_PMD, "* igb_uio | uio_pci_generic | vfio-pci");
 RTE_PMD_REGISTER_CRYPTO_DRIVER(otx_cryptodev_drv, otx_cryptodev_pmd.driver,
 		otx_cryptodev_driver_id);
 
Signature · 1 line
2.11.0
#13 lore raw

v2 cover Miscellaneous fixes

In reply to Thierry Herbelot

This series of patches includes fixes for issues seen with 6WIND fast path, built on DPDK.
The patches are in 6WIND version of DPDK, and should have been upstreamed a long time ago.

V2 changes:
fix checkpatch issues

Guo Fengtian (1):
net/ixgbevf: fix stats update after a PF reset

Laurent Hardy (1):
net/i40e: set speed to undefined for default case in link update

Olivier Matz (5):
ethdev: fix description of tx descriptor status
net/e1000: fix Tx descriptor status api (igb)
net/e1000: fix Tx descriptor status api (em)
net/ixgbe: fix Tx descriptor status api
net/i40e: fix Tx descriptor status api

Thibaut Collet (2):
virtio: fix rx stats with vectorized functions
virtio: get all pending rx packets with vectorized functions

Thierry Herbelot (3):
drivers/crypto/openssl: use a local copy for the session contexts drivers/crypto/dpaa_sec: update DPAA iova table in dpaa_mem_vtop drivers/crypto/octeontx: enable unbinding for the OcteonTx crypto
engines

 drivers/crypto/dpaa_sec/dpaa_sec.c           |  5 +++-
 drivers/crypto/octeontx/otx_cryptodev.c      |  1 +
 drivers/crypto/openssl/rte_openssl_pmd.c     | 34 +++++++++++++++------
 drivers/net/e1000/em_rxtx.c                  | 33 +++++++++++++-------
 drivers/net/e1000/igb_rxtx.c                 |  9 +++---
 drivers/net/i40e/i40e_ethdev.c               |  4 +--
 drivers/net/i40e/i40e_ethdev_vf.c            |  8 +++--
 drivers/net/i40e/i40e_rxtx.c                 | 37 ++++++++++++++++-------
 drivers/net/ixgbe/ixgbe_ethdev.c             |  6 ++--
 drivers/net/ixgbe/ixgbe_rxtx.c               | 45 ++++++++++++++++++++--------
 drivers/net/ixgbe/ixgbe_rxtx.h               |  1 +
 drivers/net/virtio/virtio_rxtx.c             |  2 +-
 drivers/net/virtio/virtio_rxtx.h             |  2 ++
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 11 +++++--
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 11 +++++--
 lib/librte_ethdev/rte_ethdev.h               |  4 +--
 16 files changed, 153 insertions(+), 60 deletions(-)
Signature · 1 line
2.11.0
#14 lore raw

PATCH v2 1/12 net/ixgbevf: fix stats update after a PF reset

From: Guo Fengtian <[email protected]>

When PF is set down, in VF, the value of stats register is zero. So only increase stats when it's non zero.

Fixes af75078fece3 ('first public release')
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/net/ixgbe/ixgbe_ethdev.c +4 -2
index 03fc1f71799c..57f5bfa219c1 100644
@@ -385,7 +385,8 @@ static void ixgbe_l2_tunnel_conf(struct rte_eth_dev *dev);
 #define UPDATE_VF_STAT(reg, last, cur)                          \
 {                                                               \
 	uint32_t latest = IXGBE_READ_REG(hw, reg);              \
-	cur += (latest - last) & UINT_MAX;                      \
+	if (latest)                                             \
+		cur += (latest - last) & UINT_MAX;              \
 	last = latest;                                          \
 }
 
@@ -394,7 +395,8 @@ static void ixgbe_l2_tunnel_conf(struct rte_eth_dev *dev);
 	u64 new_lsb = IXGBE_READ_REG(hw, lsb);                   \
 	u64 new_msb = IXGBE_READ_REG(hw, msb);                   \
 	u64 latest = ((new_msb << 32) | new_lsb);                \
-	cur += (0x1000000000LL + latest - last) & 0xFFFFFFFFFLL; \
+	if (latest)                                              \
+		cur += (0x1000000000LL + latest - last) & 0xFFFFFFFFFLL;\
 	last = latest;                                           \
 }
 
Signature · 1 line
2.11.0
#15 lore raw

PATCH v2 2/12 ethdev: fix description of tx descriptor status

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The API comment of rte_eth_tx_descriptor_status() was incorrect. The reference descriptor (when offset = 0) is not where the next packet will be sent, but where the latest packet has been enqueued.

Fixes 52f5cdd2e897 ("ethdev: add descriptor status API")
---
 lib/librte_ethdev/rte_ethdev.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_ethdev/rte_ethdev.h +2 -2
index dc6596bc93b4..b423e71050e9 100644
@@ -4245,8 +4245,8 @@ rte_eth_rx_descriptor_status(uint16_t port_id, uint16_t queue_id,
  * @param queue_id
  *  A valid Tx queue identifier on this port.
  * @param offset
- *  The offset of the descriptor starting from tail (0 is the place where
- *  the next packet will be send).
+ *  The offset of the descriptor starting from tail (0 is the last written
+ *  descriptor).
  *
  * @return
  *  - (RTE_ETH_TX_DESC_FULL) Descriptor is being processed by the hw, i.e.
Signature · 1 line
2.11.0
#16 lore raw

PATCH v2 3/12 net/e1000: fix Tx descriptor status api (igb)

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The Tx descriptor status api was not behaving as expected. This API is used to inspect the content of the descriptors in the Tx ring to determine the length of the Tx queue.

Since the software advances the tail pointer and the hardware advances the head pointer, the Tx queue is located before txq->tx_tail in the ring. Therefore, a call to rte_eth_tx_descriptor_status(..., offset=20) should inspect the 20th descriptor before the tail, not after.

Fixes 978f8eea1719 ("net/e1000: implement descriptor status API (igb)")
---
 drivers/net/e1000/igb_rxtx.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/net/e1000/igb_rxtx.c +5 -4
index c5606de5d7a0..c22118e59a21 100644
@@ -1835,14 +1835,15 @@ eth_igb_tx_descriptor_status(void *tx_queue, uint16_t offset)
 {
 	struct igb_tx_queue *txq = tx_queue;
 	volatile uint32_t *status;
-	uint32_t desc;
+	int32_t desc;
 
 	if (unlikely(offset >= txq->nb_tx_desc))
 		return -EINVAL;
 
-	desc = txq->tx_tail + offset;
-	if (desc >= txq->nb_tx_desc)
-		desc -= txq->nb_tx_desc;
+	desc = txq->tx_tail - offset - 1;
+	if (desc < 0)
+		desc += txq->nb_tx_desc;
+	desc = txq->sw_ring[desc].last_id;
 
 	status = &txq->tx_ring[desc].wb.status;
 	if (*status & rte_cpu_to_le_32(E1000_TXD_STAT_DD))
Signature · 1 line
2.11.0
#17 lore raw

PATCH v2 4/12 net/e1000: fix Tx descriptor status api (em)

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The Tx descriptor status api was not behaving as expected. This API is used to inspect the content of the descriptors in the Tx ring to determine the length of the Tx queue.

Since the software advances the tail pointer and the hardware advances the head pointer, the Tx queue is located before txq->tx_tail in the ring. Therefore, a call to rte_eth_tx_descriptor_status(..., offset=20) should inspect the 20th descriptor before the tail, not after.

As before, we still need to take care about only checking descriptors that have the RS bit.

Additionally, we can avoid an access to the ring if offset is greater or equal to nb_tx_desc - nb_tx_free.

Fixes b9082317cab3 ("net/e1000: implement descriptor status API (em)")
---
 drivers/net/e1000/em_rxtx.c | 33 +++++++++++++++++++++++----------
 1 file changed, 23 insertions(+), 10 deletions(-)
drivers/net/e1000/em_rxtx.c +23 -10
index 5925e490641b..3061998c7768 100644
@@ -152,6 +152,7 @@ struct em_tx_queue {
 	uint64_t               tx_ring_phys_addr; /**< TX ring DMA address. */
 	struct em_tx_entry    *sw_ring; /**< virtual address of SW ring. */
 	volatile uint32_t      *tdt_reg_addr; /**< Address of TDT register. */
+	volatile uint32_t      *tdh_reg_addr; /**< Address of TDH register. */
 	uint16_t               nb_tx_desc;    /**< number of TX descriptors. */
 	uint16_t               tx_tail;  /**< Current value of TDT register. */
 	/**< Start freeing TX buffers if there are less free descriptors than
@@ -1304,6 +1305,7 @@ eth_em_tx_queue_setup(struct rte_eth_dev *dev,
 	txq->port_id = dev->data->port_id;
 
 	txq->tdt_reg_addr = E1000_PCI_REG_ADDR(hw, E1000_TDT(queue_idx));
+	txq->tdh_reg_addr = E1000_PCI_REG_ADDR(hw, E1000_TDH(queue_idx));
 	txq->tx_ring_phys_addr = tz->iova;
 	txq->tx_ring = (struct e1000_data_desc *) tz->addr;
 
@@ -1557,22 +1559,33 @@ eth_em_tx_descriptor_status(void *tx_queue, uint16_t offset)
 {
 	struct em_tx_queue *txq = tx_queue;
 	volatile uint8_t *status;
-	uint32_t desc;
+	int32_t desc, dd;
 
 	if (unlikely(offset >= txq->nb_tx_desc))
 		return -EINVAL;
+	if (offset >= txq->nb_tx_desc - txq->nb_tx_free)
+		return RTE_ETH_TX_DESC_DONE;
 
-	desc = txq->tx_tail + offset;
-	/* go to next desc that has the RS bit */
-	desc = ((desc + txq->tx_rs_thresh - 1) / txq->tx_rs_thresh) *
-		txq->tx_rs_thresh;
-	if (desc >= txq->nb_tx_desc) {
-		desc -= txq->nb_tx_desc;
-		if (desc >= txq->nb_tx_desc)
-			desc -= txq->nb_tx_desc;
+	desc = txq->tx_tail - offset - 1;
+	if (desc < 0)
+		desc += txq->nb_tx_desc;
+
+	/* offset is too small, no other way than reading PCI reg */
+	if (unlikely(offset < txq->tx_rs_thresh)) {
+		int16_t tx_head, queue_size;
+		tx_head = e1000_read_addr(txq->tdh_reg_addr);
+		queue_size = txq->tx_tail - tx_head;
+		if (queue_size < 0)
+			queue_size += txq->nb_tx_desc;
+		return queue_size > offset ? RTE_ETH_TX_DESC_FULL :
+			RTE_ETH_TX_DESC_DONE;
 	}
 
-	status = &txq->tx_ring[desc].upper.fields.status;
+	/* index of the dd bit to look at */
+	dd = (desc / txq->tx_rs_thresh + 1) * txq->tx_rs_thresh - 1;
+	dd = txq->sw_ring[dd].last_id;
+
+	status = &txq->tx_ring[dd].upper.fields.status;
 	if (*status & E1000_TXD_STAT_DD)
 		return RTE_ETH_TX_DESC_DONE;
 
Signature · 1 line
2.11.0
#18 lore raw

PATCH v2 5/12 net/ixgbe: fix Tx descriptor status api

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The Tx descriptor status api was not behaving as expected. This API is used to inspect the content of the descriptors in the Tx ring to determine the length of the Tx queue.

Since the software advances the tail pointer and the hardware advances the head pointer, the Tx queue is located before txq->tx_tail in the ring. Therefore, a call to rte_eth_tx_descriptor_status(..., offset=20) should inspect the 20th descriptor before the tail, not after.

As before, we still need to take care about only checking descriptors that have the RS bit.

Additionally, we can avoid an access to the ring if offset is greater or equal to nb_tx_desc - nb_tx_free.

Fixes 5da8b8814178 ("net/ixgbe: implement descriptor status API")
---
 drivers/net/ixgbe/ixgbe_rxtx.c | 45 +++++++++++++++++++++++++++++++-----------
 drivers/net/ixgbe/ixgbe_rxtx.h |  1 +
 2 files changed, 34 insertions(+), 12 deletions(-)
drivers/net/ixgbe/ixgbe_rxtx.c +33 -12
index edcfa60cec98..4abc2fe37488 100644
@@ -2627,10 +2627,15 @@ ixgbe_dev_tx_queue_setup(struct rte_eth_dev *dev,
 	    hw->mac.type == ixgbe_mac_X540_vf ||
 	    hw->mac.type == ixgbe_mac_X550_vf ||
 	    hw->mac.type == ixgbe_mac_X550EM_x_vf ||
-	    hw->mac.type == ixgbe_mac_X550EM_a_vf)
+	    hw->mac.type == ixgbe_mac_X550EM_a_vf) {
 		txq->tdt_reg_addr = IXGBE_PCI_REG_ADDR(hw, IXGBE_VFTDT(queue_idx));
-	else
+		txq->tdh_reg_addr = IXGBE_PCI_REG_ADDR(hw,
+						       IXGBE_VFTDH(queue_idx));
+	} else {
 		txq->tdt_reg_addr = IXGBE_PCI_REG_ADDR(hw, IXGBE_TDT(txq->reg_idx));
+		txq->tdh_reg_addr = IXGBE_PCI_REG_ADDR(hw,
+						       IXGBE_TDH(txq->reg_idx));
+	}
 
 	txq->tx_ring_phys_addr = tz->iova;
 	txq->tx_ring = (union ixgbe_adv_tx_desc *) tz->addr;
@@ -3163,22 +3168,38 @@ ixgbe_dev_tx_descriptor_status(void *tx_queue, uint16_t offset)
 {
 	struct ixgbe_tx_queue *txq = tx_queue;
 	volatile uint32_t *status;
-	uint32_t desc;
+	int32_t desc, dd;
 
 	if (unlikely(offset >= txq->nb_tx_desc))
 		return -EINVAL;
+	if (offset >= txq->nb_tx_desc - txq->nb_tx_free)
+		return RTE_ETH_TX_DESC_DONE;
+
+	desc = txq->tx_tail - offset - 1;
+	if (desc < 0)
+		desc += txq->nb_tx_desc;
 
-	desc = txq->tx_tail + offset;
-	/* go to next desc that has the RS bit */
-	desc = ((desc + txq->tx_rs_thresh - 1) / txq->tx_rs_thresh) *
-		txq->tx_rs_thresh;
-	if (desc >= txq->nb_tx_desc) {
-		desc -= txq->nb_tx_desc;
-		if (desc >= txq->nb_tx_desc)
-			desc -= txq->nb_tx_desc;
+	/* offset is too small, no other way than reading PCI reg */
+	if (unlikely(offset < txq->tx_rs_thresh)) {
+		int16_t tx_head, queue_size;
+		tx_head = ixgbe_read_addr(txq->tdh_reg_addr);
+		queue_size = txq->tx_tail - tx_head;
+		if (queue_size < 0)
+			queue_size += txq->nb_tx_desc;
+		return queue_size > offset ? RTE_ETH_TX_DESC_FULL :
+			RTE_ETH_TX_DESC_DONE;
 	}
 
-	status = &txq->tx_ring[desc].wb.status;
+	/* index of the dd bit to look at */
+	dd = (desc / txq->tx_rs_thresh + 1) * txq->tx_rs_thresh - 1;
+
+	/* In full featured mode, RS bit is only set in the last descriptor */
+	/* of a multisegments packet */
+	if (!(txq->offloads == 0 &&
+	      txq->tx_rs_thresh >= RTE_PMD_IXGBE_TX_MAX_BURST))
+		dd = txq->sw_ring[dd].last_id;
+
+	status = &txq->tx_ring[dd].wb.status;
 	if (*status & rte_cpu_to_le_32(IXGBE_ADVTXD_STAT_DD))
 		return RTE_ETH_TX_DESC_DONE;
 
drivers/net/ixgbe/ixgbe_rxtx.h +1 -0
index 505d344b9cee..05fd4167576c 100644
@@ -201,6 +201,7 @@ struct ixgbe_tx_queue {
 		struct ixgbe_tx_entry_v *sw_ring_v; /**< address of SW ring for vector PMD */
 	};
 	volatile uint32_t   *tdt_reg_addr; /**< Address of TDT register. */
+	volatile uint32_t   *tdh_reg_addr; /**< Address of TDH register. */
 	uint16_t            nb_tx_desc;    /**< number of TX descriptors. */
 	uint16_t            tx_tail;       /**< current value of TDT reg. */
 	/**< Start freeing TX buffers if there are less free descriptors than
Signature · 1 line
2.11.0
#19 lore raw

PATCH v2 6/12 net/i40e: fix Tx descriptor status api

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The Tx descriptor status api was not behaving as expected. This API is used to inspect the content of the descriptors in the Tx ring to determine the length of the Tx queue.

Since the software advances the tail pointer and the hardware advances the head pointer, the Tx queue is located before txq->tx_tail in the ring. Therefore, a call to rte_eth_tx_descriptor_status(..., offset=20) should inspect the 20th descriptor before the tail, not after.

As before, we still need to take care about only checking descriptors that have the RS bit.

Additionally, we can avoid an access to the ring if offset is greater or equal to nb_tx_desc - nb_tx_free.

Fixes a9dd9af6f38e ("net/i40e: implement descriptor status API")
---
 drivers/net/i40e/i40e_rxtx.c | 37 +++++++++++++++++++++++++++----------
 1 file changed, 27 insertions(+), 10 deletions(-)
drivers/net/i40e/i40e_rxtx.c +27 -10
index 692c3bab4b5f..d84a97732f1e 100644
@@ -2031,22 +2031,39 @@ i40e_dev_tx_descriptor_status(void *tx_queue, uint16_t offset)
 	struct i40e_tx_queue *txq = tx_queue;
 	volatile uint64_t *status;
 	uint64_t mask, expect;
-	uint32_t desc;
+	int32_t desc, dd;
 
 	if (unlikely(offset >= txq->nb_tx_desc))
 		return -EINVAL;
+	if (offset >= txq->nb_tx_desc - txq->nb_tx_free)
+		return RTE_ETH_TX_DESC_DONE;
+
+	desc = txq->tx_tail - offset - 1;
+	if (desc < 0)
+		desc += txq->nb_tx_desc;
 
-	desc = txq->tx_tail + offset;
-	/* go to next desc that has the RS bit */
-	desc = ((desc + txq->tx_rs_thresh - 1) / txq->tx_rs_thresh) *
-		txq->tx_rs_thresh;
-	if (desc >= txq->nb_tx_desc) {
-		desc -= txq->nb_tx_desc;
-		if (desc >= txq->nb_tx_desc)
-			desc -= txq->nb_tx_desc;
+	/* offset is too small, no other way than reading PCI reg */
+	if (unlikely(offset < txq->tx_rs_thresh)) {
+		int16_t tx_head, queue_size;
+		tx_head = I40E_READ_REG(I40E_VSI_TO_HW(txq->vsi),
+					I40E_QTX_HEAD(txq->reg_idx));
+		queue_size = txq->tx_tail - tx_head;
+		if (queue_size < 0)
+			queue_size += txq->nb_tx_desc;
+		return queue_size > offset ? RTE_ETH_TX_DESC_FULL :
+			RTE_ETH_TX_DESC_DONE;
 	}
 
-	status = &txq->tx_ring[desc].cmd_type_offset_bsz;
+	/* index of the dd bit to look at */
+	dd = (desc / txq->tx_rs_thresh + 1) * txq->tx_rs_thresh - 1;
+
+	/* In full featured mode, RS bit is only set in the last descriptor */
+	/* of a multisegments packet */
+	if (!(txq->offloads == 0 &&
+	      txq->tx_rs_thresh >= RTE_PMD_I40E_TX_MAX_BURST))
+		dd = txq->sw_ring[dd].last_id;
+
+	status = &txq->tx_ring[dd].cmd_type_offset_bsz;
 	mask = rte_le_to_cpu_64(I40E_TXD_QW1_DTYPE_MASK);
 	expect = rte_cpu_to_le_64(
 		I40E_TX_DESC_DTYPE_DESC_DONE << I40E_TXD_QW1_DTYPE_SHIFT);
Signature · 1 line
2.11.0
#20 lore raw

PATCH v2 7/12 net/i40e: set speed to undefined for default case in link update

In reply to Thierry Herbelot

From: Laurent Hardy <[email protected]>

During PF/VF link update, a default speed value of 100M will be set if get_link_info has failed or speed is unknown.

Consequently if PF is put in no-carrier state, VFs will switch to "in carrier" state due to a link up + a link speed set to 100M (default value if no speed detected).

To be consistent with linux drivers on which PF and VFs are in same carrier state, sets default speed to undefined (instead of 100M) and updates a link status of VF only if link is up and speed is different from undefined.

Fixes 4861cde46116 ('i40e: new poll mode driver')
---
 drivers/net/i40e/i40e_ethdev.c    | 4 ++--
 drivers/net/i40e/i40e_ethdev_vf.c | 8 +++++---
 2 files changed, 7 insertions(+), 5 deletions(-)
drivers/net/i40e/i40e_ethdev.c +2 -2
index 4e40b7ab5250..76abe8209a10 100644
@@ -2743,7 +2743,7 @@ update_link_aq(struct i40e_hw *hw, struct rte_eth_link *link,
 		status = i40e_aq_get_link_info(hw, enable_lse,
 						&link_status, NULL);
 		if (unlikely(status != I40E_SUCCESS)) {
-			link->link_speed = ETH_SPEED_NUM_100M;
+			link->link_speed = ETH_SPEED_NUM_NONE;
 			link->link_duplex = ETH_LINK_FULL_DUPLEX;
 			PMD_DRV_LOG(ERR, "Failed to get link info");
 			return;
@@ -2777,7 +2777,7 @@ update_link_aq(struct i40e_hw *hw, struct rte_eth_link *link,
 		link->link_speed = ETH_SPEED_NUM_40G;
 		break;
 	default:
-		link->link_speed = ETH_SPEED_NUM_100M;
+		link->link_speed = ETH_SPEED_NUM_NONE;
 		break;
 	}
 }
drivers/net/i40e/i40e_ethdev_vf.c +5 -3
index 308fb9835ab1..bf707e57b29b 100644
@@ -2143,13 +2143,15 @@ i40evf_dev_link_update(struct rte_eth_dev *dev,
 		new_link.link_speed = ETH_SPEED_NUM_40G;
 		break;
 	default:
-		new_link.link_speed = ETH_SPEED_NUM_100M;
+		new_link.link_speed = ETH_SPEED_NUM_NONE;
 		break;
 	}
 	/* full duplex only */
 	new_link.link_duplex = ETH_LINK_FULL_DUPLEX;
-	new_link.link_status = vf->link_up ? ETH_LINK_UP :
-					     ETH_LINK_DOWN;
+	new_link.link_status = vf->link_up &&
+				new_link.link_speed != ETH_SPEED_NUM_NONE
+				? ETH_LINK_UP
+				: ETH_LINK_DOWN;
 	new_link.link_autoneg =
 		!(dev->data->dev_conf.link_speeds & ETH_LINK_SPEED_FIXED);
 
Signature · 1 line
2.11.0
#21 lore raw

PATCH v2 8/12 virtio: fix rx stats with vectorized functions

In reply to Thierry Herbelot

From: Thibaut Collet <[email protected]>

With vectorized functions, only the rx stats for number of packets is incremented.
Update also the other statistics.
Performance impact is about 2%

Fixes fc3d66212fed ("virtio: add vector Rx")
---
 drivers/net/virtio/virtio_rxtx.c             | 2 +-
 drivers/net/virtio/virtio_rxtx.h             | 2 ++
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 6 ++++++
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 6 ++++++
 4 files changed, 15 insertions(+), 1 deletion(-)
drivers/net/virtio/virtio_rxtx.c +1 -1
index 27ead19fbe81..6dd62bf51863 100644
@@ -1083,7 +1083,7 @@ virtio_discard_rxbuf_inorder(struct virtqueue *vq, struct rte_mbuf *m)
 	}
 }
 
-static inline void
+void
 virtio_update_packet_stats(struct virtnet_stats *stats, struct rte_mbuf *mbuf)
 {
 	uint32_t s = mbuf->pkt_len;
drivers/net/virtio/virtio_rxtx.h +2 -0
index 685cc4f8104c..1eb8dae227ee 100644
@@ -59,5 +59,7 @@ struct virtnet_ctl {
 };
 
 int virtio_rxq_vec_setup(struct virtnet_rx *rxvq);
+void virtio_update_packet_stats(struct virtnet_stats *stats,
+				struct rte_mbuf *mbuf);
 
 #endif /* _VIRTIO_RXTX_H_ */
drivers/net/virtio/virtio_rxtx_simple_neon.c +6 -0
index cdc2a4d28ed5..e4b18cba0ce5 100644
@@ -47,6 +47,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
 	struct rte_mbuf **sw_ring_end;
+	struct rte_mbuf **ref_rx_pkts;
 	uint16_t nb_pkts_received = 0;
 
 	uint8x16_t shuf_msk1 = {
@@ -105,6 +106,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
 		nb_pkts_received < nb_used;) {
 		uint64x2_t desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
@@ -204,5 +206,9 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	vq->vq_used_cons_idx += nb_pkts_received;
 	vq->vq_free_cnt += nb_pkts_received;
 	rxvq->stats.packets += nb_pkts_received;
+	for (nb_used = 0; nb_used < nb_pkts_received; nb_used++) {
+		rxvq->stats.bytes += ref_rx_pkts[nb_used]->pkt_len;
+		virtio_update_packet_stats(&rxvq->stats, ref_rx_pkts[nb_used]);
+	}
 	return nb_pkts_received;
 }
drivers/net/virtio/virtio_rxtx_simple_sse.c +6 -0
index af76708d66ae..c757e8c9d601 100644
@@ -48,6 +48,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
 	struct rte_mbuf **sw_ring_end;
+	struct rte_mbuf **ref_rx_pkts;
 	uint16_t nb_pkts_received = 0;
 	__m128i shuf_msk1, shuf_msk2, len_adjust;
 
@@ -107,6 +108,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
 		nb_pkts_received < nb_used;) {
 		__m128i desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
@@ -190,5 +192,9 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	vq->vq_used_cons_idx += nb_pkts_received;
 	vq->vq_free_cnt += nb_pkts_received;
 	rxvq->stats.packets += nb_pkts_received;
+	for (nb_used = 0; nb_used < nb_pkts_received; nb_used++) {
+		rxvq->stats.bytes += ref_rx_pkts[nb_used]->pkt_len;
+		virtio_update_packet_stats(&rxvq->stats, ref_rx_pkts[nb_used]);
+	}
 	return nb_pkts_received;
 }
Signature · 1 line
2.11.0
#22 lore raw

Re: [dpdk-dev] [dpdk-stable] [PATCH 19.11 V2 08/12] virtio: fix rx stats with vectorized functions

Quoted 6 lines · Thierry Herbelot

From: Thibaut Collet <[email protected]>

With vectorized functions, only the rx stats for number of packets is incremented.
Update also the other statistics.
Performance impact is about 2%

Could you share some details about your performance test?

Quoted 21 lines
drivers/net/virtio/virtio_rxtx_simple_sse.c +2 -0
index af76708d66ae..c757e8c9d601 100644
@@ -48,6 +48,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
 	struct rte_mbuf **sw_ring_end;
+	struct rte_mbuf **ref_rx_pkts;
 	uint16_t nb_pkts_received = 0;
 	__m128i shuf_msk1, shuf_msk2, len_adjust;
 
@@ -107,6 +108,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
 		nb_pkts_received < nb_used;) {
 		__m128i desc[RTE_VIRTIO_DESC_PER_LOOP / 2];

drivers/net/virtio/virtio_rxtx_simple_sse.c virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts, · line 197

Quoted 6 lines
drivers/net/virtio/virtio_rxtx_simple_sse.c +3 -0
@@ -190,5 +192,9 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	vq->vq_used_cons_idx += nb_pkts_received;
 	vq->vq_free_cnt += nb_pkts_received;
 	rxvq->stats.packets += nb_pkts_received;
+	for (nb_used = 0; nb_used < nb_pkts_received; nb_used++) {
+		rxvq->stats.bytes += ref_rx_pkts[nb_used]->pkt_len;
+		virtio_update_packet_stats(&rxvq->stats, ref_rx_pkts[nb_used]);

The stats.bytes was updated twice by above code.

Quoted 6 lines
drivers/net/virtio/virtio_rxtx_simple_sse.c +1 -0
+	}
 	return nb_pkts_received;
 }
Signature · 1 line
2.11.0
#23 lore raw

Re: [dpdk-dev] [dpdk-stable] [PATCH 19.11 V2 08/12] virtio: fix rx stats with vectorized functions

Quoted 10 lines · Tiwei Bie
Quoted 6 lines · Thierry Herbelot

From: Thibaut Collet <[email protected]>

With vectorized functions, only the rx stats for number of packets is incremented.
Update also the other statistics.
Performance impact is about 2%

Could you share some details about your performance test?

The test has been done with a 6wind application based on dpdk and I have not keep details. With test pmd impact is maybe a little bit higher.

Quoted 18 lines
Quoted 18 lines
drivers/net/virtio/virtio_rxtx_simple_sse.c +1 -0
index af76708d66ae..c757e8c9d601 100644
@@ -48,6 +48,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
      struct vring_used_elem *rused;
      struct rte_mbuf **sw_ring;
      struct rte_mbuf **sw_ring_end;
+     struct rte_mbuf **ref_rx_pkts;
      uint16_t nb_pkts_received = 0;
      __m128i shuf_msk1, shuf_msk2, len_adjust;
 
@@ -107,6 +108,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
                      virtqueue_notify(vq);
      }
 
Quoted 13 lines

drivers/net/virtio/virtio_rxtx_simple_sse.c virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts, · line 197

Quoted 11 lines
drivers/net/virtio/virtio_rxtx_simple_sse.c +4 -0
+     ref_rx_pkts = rx_pkts;
      for (nb_pkts_received = 0;
              nb_pkts_received < nb_used;) {
              __m128i desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
@@ -190,5 +192,9 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
      vq->vq_used_cons_idx += nb_pkts_received;
      vq->vq_free_cnt += nb_pkts_received;
      rxvq->stats.packets += nb_pkts_received;
+     for (nb_used = 0; nb_used < nb_pkts_received; nb_used++) {
+             rxvq->stats.bytes += ref_rx_pkts[nb_used]->pkt_len;
+             virtio_update_packet_stats(&rxvq->stats, ref_rx_pkts[nb_used]);

The stats.bytes was updated twice by above code.

Agree. My correction is an old one, done on the old 18.11 dpdk, and so done before patch
https://git.dpdk.org/dpdk/commit/drivers/net/virtio/virtio_rxtx.c?id=81e5cdf19e583c742040d3be83b8cc79b451e243.
So the line
+ rxvq->stats.bytes += ref_rx_pkts[nb_used]->pkt_len;
must be removed to be compliant with Ilya Maximets patch.

Quoted 7 lines
Quoted 7 lines
drivers/net/virtio/virtio_rxtx_simple_sse.c +1 -1
+     }
      return nb_pkts_received;
 }
--

2.11.0

#24 lore raw

PATCH v2 9/12 virtio: get all pending rx packets with vectorized functions

In reply to Thierry Herbelot

From: Thibaut Collet <[email protected]>

The loop to read packets does not take all packets as the number of available packets (nb_used) is decremented in the loop. Take all available packets provides a performance improvement of 3%.

Fixes fc3d66212fed ("virtio: add vector Rx")
---
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 5 +++--
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 5 +++--
 2 files changed, 6 insertions(+), 4 deletions(-)
drivers/net/virtio/virtio_rxtx_simple_neon.c +3 -2
index e4b18cba0ce5..66bdaa00e01f 100644
@@ -42,7 +42,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct virtnet_rx *rxvq = rx_queue;
 	struct virtqueue *vq = rxvq->vq;
 	struct virtio_hw *hw = vq->hw;
-	uint16_t nb_used;
+	uint16_t nb_used, nb_total;
 	uint16_t desc_idx;
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
@@ -106,9 +106,10 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	nb_total = nb_used;
 	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
-		nb_pkts_received < nb_used;) {
+		nb_pkts_received < nb_total;) {
 		uint64x2_t desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		uint64x2_t mbp[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		uint64x2_t pkt_mb[RTE_VIRTIO_DESC_PER_LOOP];
drivers/net/virtio/virtio_rxtx_simple_sse.c +3 -2
index c757e8c9d601..811b416755e7 100644
@@ -43,7 +43,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct virtnet_rx *rxvq = rx_queue;
 	struct virtqueue *vq = rxvq->vq;
 	struct virtio_hw *hw = vq->hw;
-	uint16_t nb_used;
+	uint16_t nb_used, nb_total;
 	uint16_t desc_idx;
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
@@ -108,9 +108,10 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	nb_total = nb_used;
 	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
-		nb_pkts_received < nb_used;) {
+		nb_pkts_received < nb_total;) {
 		__m128i desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		__m128i mbp[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		__m128i pkt_mb[RTE_VIRTIO_DESC_PER_LOOP];
Signature · 1 line
2.11.0
#25 lore raw

PATCH v2 10/12 drivers/crypto/openssl: use a local copy for the session contexts

In reply to Thierry Herbelot

Session contexts are used for temporary storage when processing a packet.
If packets for the same session are to be processed simultaneously on multiple cores, separate contexts must be used.

Note: with openssl 1.1.1 EVP_CIPHER_CTX can no longer be defined as a variable on the stack: it must be allocated. This in turn reduces the performance.

Fixes d61f70b4c918 ('crypto/libcrypto: add driver for OpenSSL library')
---
 drivers/crypto/openssl/rte_openssl_pmd.c | 34 +++++++++++++++++++++++---------
 1 file changed, 25 insertions(+), 9 deletions(-)
drivers/crypto/openssl/rte_openssl_pmd.c +25 -9
index 2f5552840741..ce2d12347737 100644
@@ -1290,6 +1290,7 @@ process_openssl_combined_op
 	int srclen, aadlen, status = -1;
 	uint32_t offset;
 	uint8_t taglen;
+	EVP_CIPHER_CTX *ctx_copy;
 
 	/*
 	 * Segmented destination buffer is not supported for
@@ -1326,6 +1327,8 @@ process_openssl_combined_op
 	}
 
 	taglen = sess->auth.digest_length;
+	ctx_copy = EVP_CIPHER_CTX_new();
+	EVP_CIPHER_CTX_copy(ctx_copy, sess->cipher.ctx);
 
 	if (sess->cipher.direction == RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
 		if (sess->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC ||
@@ -1333,12 +1336,12 @@ process_openssl_combined_op
 			status = process_openssl_auth_encryption_gcm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, sess->cipher.ctx);
+					dst, tag, ctx_copy);
 		else
 			status = process_openssl_auth_encryption_ccm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, taglen, sess->cipher.ctx);
+					dst, tag, taglen, ctx_copy);
 
 	} else {
 		if (sess->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC ||
@@ -1346,14 +1349,15 @@ process_openssl_combined_op
 			status = process_openssl_auth_decryption_gcm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, sess->cipher.ctx);
+					dst, tag, ctx_copy);
 		else
 			status = process_openssl_auth_decryption_ccm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, taglen, sess->cipher.ctx);
+					dst, tag, taglen, ctx_copy);
 	}
 
+	EVP_CIPHER_CTX_free(ctx_copy);
 	if (status != 0) {
 		if (status == (-EFAULT) &&
 				sess->auth.operation ==
@@ -1372,6 +1376,7 @@ process_openssl_cipher_op
 {
 	uint8_t *dst, *iv;
 	int srclen, status;
+	EVP_CIPHER_CTX *ctx_copy;
 
 	/*
 	 * Segmented destination buffer is not supported for
@@ -1388,22 +1393,25 @@ process_openssl_cipher_op
 
 	iv = rte_crypto_op_ctod_offset(op, uint8_t *,
 			sess->iv.offset);
+	ctx_copy = EVP_CIPHER_CTX_new();
+	EVP_CIPHER_CTX_copy(ctx_copy, sess->cipher.ctx);
 
 	if (sess->cipher.mode == OPENSSL_CIPHER_LIB)
 		if (sess->cipher.direction == RTE_CRYPTO_CIPHER_OP_ENCRYPT)
 			status = process_openssl_cipher_encrypt(mbuf_src, dst,
 					op->sym->cipher.data.offset, iv,
-					srclen, sess->cipher.ctx);
+					srclen, ctx_copy);
 		else
 			status = process_openssl_cipher_decrypt(mbuf_src, dst,
 					op->sym->cipher.data.offset, iv,
-					srclen, sess->cipher.ctx);
+					srclen, ctx_copy);
 	else
 		status = process_openssl_cipher_des3ctr(mbuf_src, dst,
 				op->sym->cipher.data.offset, iv,
 				sess->cipher.key.data, srclen,
-				sess->cipher.ctx);
+				ctx_copy);
 
+	EVP_CIPHER_CTX_free(ctx_copy);
 	if (status != 0)
 		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
 }
@@ -1507,6 +1515,8 @@ process_openssl_auth_op(struct openssl_qp *qp, struct rte_crypto_op *op,
 {
 	uint8_t *dst;
 	int srclen, status;
+	EVP_MD_CTX *ctx_a;
+	HMAC_CTX *ctx_h;
 
 	srclen = op->sym->auth.data.length;
 
@@ -1514,14 +1524,20 @@ process_openssl_auth_op(struct openssl_qp *qp, struct rte_crypto_op *op,
 
 	switch (sess->auth.mode) {
 	case OPENSSL_AUTH_AS_AUTH:
+		ctx_a = EVP_MD_CTX_create();
+		EVP_MD_CTX_copy_ex(ctx_a, sess->auth.auth.ctx);
 		status = process_openssl_auth(mbuf_src, dst,
 				op->sym->auth.data.offset, NULL, NULL, srclen,
-				sess->auth.auth.ctx, sess->auth.auth.evp_algo);
+				ctx_a, sess->auth.auth.evp_algo);
+		EVP_MD_CTX_destroy(ctx_a);
 		break;
 	case OPENSSL_AUTH_AS_HMAC:
+		ctx_h = HMAC_CTX_new();
+		HMAC_CTX_copy(ctx_h, sess->auth.hmac.ctx);
 		status = process_openssl_auth_hmac(mbuf_src, dst,
 				op->sym->auth.data.offset, srclen,
-				sess->auth.hmac.ctx);
+				ctx_h);
+		HMAC_CTX_free(ctx_h);
 		break;
 	default:
 		status = -1;
Signature · 1 line
2.11.0
#26 lore raw

PATCH v2 11/12 drivers/crypto/dpaa_sec: update DPAA iova table in dpaa_mem_vtop

In reply to Thierry Herbelot

dpaa_sec needs translations between physical and virtual addresses. V to P translation is relatively fast, as memory is managed in contiguous segments.

The result of each V to P translation is used to update the DPAA iova table, which should be updated by a Mem event callback, but is not. Then the DPAA iova table has entries for all needed memory ranges.

With this patch, dpaa_mem_ptov will always use dpaax_iova_table_get_va, which ensures optimal performance.

Fixes 5a7dbb934d75 ('dpaa: enable dpaax library')
---
 drivers/crypto/dpaa_sec/dpaa_sec.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/crypto/dpaa_sec/dpaa_sec.c +4 -1
index 122c80a072ff..22b8b1d63ce0 100644
@@ -38,6 +38,7 @@
 #include <rte_dpaa_bus.h>
 #include <dpaa_sec.h>
 #include <dpaa_sec_log.h>
+#include <dpaax_iova_table.h>
 
 enum rta_sec_era rta_sec_era;
 
@@ -100,8 +101,10 @@ dpaa_mem_vtop(void *vaddr)
 	const struct rte_memseg *ms;
 
 	ms = rte_mem_virt2memseg(vaddr, NULL);
-	if (ms)
+	if (ms) {
+		dpaax_iova_table_update(ms->iova, (void *)ms->addr_64, ms->len);
 		return ms->iova + RTE_PTR_DIFF(vaddr, ms->addr);
+	}
 	return (size_t)NULL;
 }
 
Signature · 1 line
2.11.0
#27 lore raw

PATCH v2 12/12 drivers/crypto/octeontx: enable unbinding for the OcteonTx crypto engines

In reply to Thierry Herbelot

Like for Ethernet ports, the OcteonTx crypto engines must first be unbound from their kernel module, then rebound to vfio-pci, before being usable in DPDK.

As this capability is detected at runtime by dpdk-pmdinfo, add the info in the PMD registering directives.

Then an external script can be used for bind and unbind.

Fixes bfe2ae495ee268 ('crypto/octeontx: add PMD skeleton')
---
 drivers/crypto/octeontx/otx_cryptodev.c | 1 +
 1 file changed, 1 insertion(+)
drivers/crypto/octeontx/otx_cryptodev.c +1 -0
index fc64a5f3041f..16f1909966d0 100644
@@ -118,6 +118,7 @@ static struct cryptodev_driver otx_cryptodev_drv;
 
 RTE_PMD_REGISTER_PCI(CRYPTODEV_NAME_OCTEONTX_PMD, otx_cryptodev_pmd);
 RTE_PMD_REGISTER_PCI_TABLE(CRYPTODEV_NAME_OCTEONTX_PMD, pci_id_cpt_table);
+RTE_PMD_REGISTER_KMOD_DEP(CRYPTODEV_NAME_OCTEONTX_PMD, "* igb_uio | uio_pci_generic | vfio-pci");
 RTE_PMD_REGISTER_CRYPTO_DRIVER(otx_cryptodev_drv, otx_cryptodev_pmd.driver,
 		otx_cryptodev_driver_id);
 
Signature · 1 line
2.11.0
#28 lore raw

v3 cover Miscellaneous fixes

In reply to Thierry Herbelot

This series of patches includes fixes for issues seen with 6WIND fast path, built on DPDK.
The patches are in 6WIND version of DPDK, and should have been upstreamed a long time ago.

V3 changes:
fix double increment of stats in virtio vectorized functions

V2 changes:
fix checkpatch issues

Guo Fengtian (1):
net/ixgbevf: fix stats update after a PF reset

Laurent Hardy (1):
net/i40e: set speed to undefined for default case in link update

Olivier Matz (5):
ethdev: fix description of tx descriptor status
net/e1000: fix Tx descriptor status api (igb)
net/e1000: fix Tx descriptor status api (em)
net/ixgbe: fix Tx descriptor status api
net/i40e: fix Tx descriptor status api

Thibaut Collet (2):
virtio: fix rx stats with vectorized functions
virtio: get all pending rx packets with vectorized functions

Thierry Herbelot (3):
drivers/crypto/openssl: use a local copy for the session contexts drivers/crypto/dpaa_sec: update DPAA iova table in dpaa_mem_vtop drivers/crypto/octeontx: enable unbinding for the OcteonTx crypto
engines

 drivers/crypto/dpaa_sec/dpaa_sec.c           |  5 +++-
 drivers/crypto/octeontx/otx_cryptodev.c      |  1 +
 drivers/crypto/openssl/rte_openssl_pmd.c     | 34 +++++++++++++++------
 drivers/net/e1000/em_rxtx.c                  | 33 +++++++++++++-------
 drivers/net/e1000/igb_rxtx.c                 |  9 +++---
 drivers/net/i40e/i40e_ethdev.c               |  4 +--
 drivers/net/i40e/i40e_ethdev_vf.c            |  8 +++--
 drivers/net/i40e/i40e_rxtx.c                 | 37 ++++++++++++++++-------
 drivers/net/ixgbe/ixgbe_ethdev.c             |  6 ++--
 drivers/net/ixgbe/ixgbe_rxtx.c               | 45 ++++++++++++++++++++--------
 drivers/net/ixgbe/ixgbe_rxtx.h               |  1 +
 drivers/net/virtio/virtio_rxtx.c             |  2 +-
 drivers/net/virtio/virtio_rxtx.h             |  2 ++
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 10 +++++--
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 10 +++++--
 lib/librte_ethdev/rte_ethdev.h               |  4 +--
 16 files changed, 151 insertions(+), 60 deletions(-)
Signature · 1 line
2.11.0
#29 lore raw

PATCH v3 1/12 net/ixgbevf: fix stats update after a PF reset

From: Guo Fengtian <[email protected]>

When PF is set down, in VF, the value of stats register is zero. So only increase stats when it's non zero.

Fixes af75078fece3 ('first public release')
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/net/ixgbe/ixgbe_ethdev.c +4 -2
index 03fc1f71799c..57f5bfa219c1 100644
@@ -385,7 +385,8 @@ static void ixgbe_l2_tunnel_conf(struct rte_eth_dev *dev);
 #define UPDATE_VF_STAT(reg, last, cur)                          \
 {                                                               \
 	uint32_t latest = IXGBE_READ_REG(hw, reg);              \
-	cur += (latest - last) & UINT_MAX;                      \
+	if (latest)                                             \
+		cur += (latest - last) & UINT_MAX;              \
 	last = latest;                                          \
 }
 
@@ -394,7 +395,8 @@ static void ixgbe_l2_tunnel_conf(struct rte_eth_dev *dev);
 	u64 new_lsb = IXGBE_READ_REG(hw, lsb);                   \
 	u64 new_msb = IXGBE_READ_REG(hw, msb);                   \
 	u64 latest = ((new_msb << 32) | new_lsb);                \
-	cur += (0x1000000000LL + latest - last) & 0xFFFFFFFFFLL; \
+	if (latest)                                              \
+		cur += (0x1000000000LL + latest - last) & 0xFFFFFFFFFLL;\
 	last = latest;                                           \
 }
 
Signature · 1 line
2.11.0
#30 lore raw

PATCH v3 2/12 ethdev: fix description of tx descriptor status

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The API comment of rte_eth_tx_descriptor_status() was incorrect. The reference descriptor (when offset = 0) is not where the next packet will be sent, but where the latest packet has been enqueued.

Fixes 52f5cdd2e897 ("ethdev: add descriptor status API")
---
 lib/librte_ethdev/rte_ethdev.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_ethdev/rte_ethdev.h +2 -2
index dc6596bc93b4..b423e71050e9 100644
@@ -4245,8 +4245,8 @@ rte_eth_rx_descriptor_status(uint16_t port_id, uint16_t queue_id,
  * @param queue_id
  *  A valid Tx queue identifier on this port.
  * @param offset
- *  The offset of the descriptor starting from tail (0 is the place where
- *  the next packet will be send).
+ *  The offset of the descriptor starting from tail (0 is the last written
+ *  descriptor).
  *
  * @return
  *  - (RTE_ETH_TX_DESC_FULL) Descriptor is being processed by the hw, i.e.
Signature · 1 line
2.11.0
#31 lore raw

Re: [dpdk-dev] [PATCH 19.11 V3 02/12] ethdev: fix description of tx descriptor status

Quoted 19 lines · Thierry Herbelot

From: Olivier Matz <[email protected]>

The API comment of rte_eth_tx_descriptor_status() was incorrect. The reference descriptor (when offset = 0) is not where the next packet will be sent, but where the latest packet has been enqueued.

Fixes 52f5cdd2e897 ("ethdev: add descriptor status API")
---
  lib/librte_ethdev/rte_ethdev.h | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_ethdev/rte_ethdev.h +0 -0
index dc6596bc93b4..b423e71050e9 100644

lib/librte_ethdev/rte_ethdev.h rte_eth_rx_descriptor_status(uint16_t port_id, uint16_t queue_id, · line 4252

Quoted 10 lines
lib/librte_ethdev/rte_ethdev.h +2 -2
@@ -4245,8 +4245,8 @@ rte_eth_rx_descriptor_status(uint16_t port_id, uint16_t queue_id,
   * @param queue_id
   *  A valid Tx queue identifier on this port.
   * @param offset
- *  The offset of the descriptor starting from tail (0 is the place where
- *  the next packet will be send).
+ *  The offset of the descriptor starting from tail (0 is the last written
+ *  descriptor).
   *
   * @return
   *  - (RTE_ETH_TX_DESC_FULL) Descriptor is being processed by the hw, i.e.

The patch lacks explanation why it was wrong before and why it is correct after the patch.

I think there is a space here for description improvements since it is still confusing. I'd suggest to remove 'tail' term from the description or write down full description to explain what/where is head and tail.

Rx case:

         V-->offset
=========***********----------------00000000000==
 UNAVAIL     DONE        AVAIL        UNUSED

i.e. V points to the next packet to be received by the driver, i.e. the first * (the descriptor is done by HW and packet is ready to be processed). DONE(*) are descriptors filled in by HW, but not processed by the driver yet. AVAIL(-) are available for HW to fill in with received packets. UNUSED(0) (right now it is UNAVAIL as well) are descriptors which can be done AVAIL on Rx queue refill
UNAVAIL(=) are reserved Rx ring space which cannot be used

The key question for Rx is the length of DONE. Is CPU fast enough to handle incoming traffic?

Tx case (where is tail V and why):

=========************--------------00000000000==
 UNAVAIL     DONE          FULL       UNUSED

DONE(*) are descriptors processed by HW and should be processed by the driver to free associated mbuf (rte_eth_tx_done_cleanup())
FULL(-) are available for HW to send these packets out, but not processed yet. UNUSED(0) is available space in Tx ring to make it FULL
UNAVAIL(=) is reserved Tx ring space which cannot be used

I'd say that it should be similar to Rx here, but it looks different in accordance with current description. Above patch fixes it from the first '-' (or the first '0'?) to be the last '-'.

What is the main question for Tx status descriptor user?

#32 lore raw

PATCH v3 3/12 net/e1000: fix Tx descriptor status api (igb)

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The Tx descriptor status api was not behaving as expected. This API is used to inspect the content of the descriptors in the Tx ring to determine the length of the Tx queue.

Since the software advances the tail pointer and the hardware advances the head pointer, the Tx queue is located before txq->tx_tail in the ring. Therefore, a call to rte_eth_tx_descriptor_status(..., offset=20) should inspect the 20th descriptor before the tail, not after.

Fixes 978f8eea1719 ("net/e1000: implement descriptor status API (igb)")
---
 drivers/net/e1000/igb_rxtx.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/net/e1000/igb_rxtx.c +5 -4
index c5606de5d7a0..c22118e59a21 100644
@@ -1835,14 +1835,15 @@ eth_igb_tx_descriptor_status(void *tx_queue, uint16_t offset)
 {
 	struct igb_tx_queue *txq = tx_queue;
 	volatile uint32_t *status;
-	uint32_t desc;
+	int32_t desc;
 
 	if (unlikely(offset >= txq->nb_tx_desc))
 		return -EINVAL;
 
-	desc = txq->tx_tail + offset;
-	if (desc >= txq->nb_tx_desc)
-		desc -= txq->nb_tx_desc;
+	desc = txq->tx_tail - offset - 1;
+	if (desc < 0)
+		desc += txq->nb_tx_desc;
+	desc = txq->sw_ring[desc].last_id;
 
 	status = &txq->tx_ring[desc].wb.status;
 	if (*status & rte_cpu_to_le_32(E1000_TXD_STAT_DD))
Signature · 1 line
2.11.0
#33 lore raw

PATCH v3 4/12 net/e1000: fix Tx descriptor status api (em)

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The Tx descriptor status api was not behaving as expected. This API is used to inspect the content of the descriptors in the Tx ring to determine the length of the Tx queue.

Since the software advances the tail pointer and the hardware advances the head pointer, the Tx queue is located before txq->tx_tail in the ring. Therefore, a call to rte_eth_tx_descriptor_status(..., offset=20) should inspect the 20th descriptor before the tail, not after.

As before, we still need to take care about only checking descriptors that have the RS bit.

Additionally, we can avoid an access to the ring if offset is greater or equal to nb_tx_desc - nb_tx_free.

Fixes b9082317cab3 ("net/e1000: implement descriptor status API (em)")
---
 drivers/net/e1000/em_rxtx.c | 33 +++++++++++++++++++++++----------
 1 file changed, 23 insertions(+), 10 deletions(-)
drivers/net/e1000/em_rxtx.c +23 -10
index 5925e490641b..3061998c7768 100644
@@ -152,6 +152,7 @@ struct em_tx_queue {
 	uint64_t               tx_ring_phys_addr; /**< TX ring DMA address. */
 	struct em_tx_entry    *sw_ring; /**< virtual address of SW ring. */
 	volatile uint32_t      *tdt_reg_addr; /**< Address of TDT register. */
+	volatile uint32_t      *tdh_reg_addr; /**< Address of TDH register. */
 	uint16_t               nb_tx_desc;    /**< number of TX descriptors. */
 	uint16_t               tx_tail;  /**< Current value of TDT register. */
 	/**< Start freeing TX buffers if there are less free descriptors than
@@ -1304,6 +1305,7 @@ eth_em_tx_queue_setup(struct rte_eth_dev *dev,
 	txq->port_id = dev->data->port_id;
 
 	txq->tdt_reg_addr = E1000_PCI_REG_ADDR(hw, E1000_TDT(queue_idx));
+	txq->tdh_reg_addr = E1000_PCI_REG_ADDR(hw, E1000_TDH(queue_idx));
 	txq->tx_ring_phys_addr = tz->iova;
 	txq->tx_ring = (struct e1000_data_desc *) tz->addr;
 
@@ -1557,22 +1559,33 @@ eth_em_tx_descriptor_status(void *tx_queue, uint16_t offset)
 {
 	struct em_tx_queue *txq = tx_queue;
 	volatile uint8_t *status;
-	uint32_t desc;
+	int32_t desc, dd;
 
 	if (unlikely(offset >= txq->nb_tx_desc))
 		return -EINVAL;
+	if (offset >= txq->nb_tx_desc - txq->nb_tx_free)
+		return RTE_ETH_TX_DESC_DONE;
 
-	desc = txq->tx_tail + offset;
-	/* go to next desc that has the RS bit */
-	desc = ((desc + txq->tx_rs_thresh - 1) / txq->tx_rs_thresh) *
-		txq->tx_rs_thresh;
-	if (desc >= txq->nb_tx_desc) {
-		desc -= txq->nb_tx_desc;
-		if (desc >= txq->nb_tx_desc)
-			desc -= txq->nb_tx_desc;
+	desc = txq->tx_tail - offset - 1;
+	if (desc < 0)
+		desc += txq->nb_tx_desc;
+
+	/* offset is too small, no other way than reading PCI reg */
+	if (unlikely(offset < txq->tx_rs_thresh)) {
+		int16_t tx_head, queue_size;
+		tx_head = e1000_read_addr(txq->tdh_reg_addr);
+		queue_size = txq->tx_tail - tx_head;
+		if (queue_size < 0)
+			queue_size += txq->nb_tx_desc;
+		return queue_size > offset ? RTE_ETH_TX_DESC_FULL :
+			RTE_ETH_TX_DESC_DONE;
 	}
 
-	status = &txq->tx_ring[desc].upper.fields.status;
+	/* index of the dd bit to look at */
+	dd = (desc / txq->tx_rs_thresh + 1) * txq->tx_rs_thresh - 1;
+	dd = txq->sw_ring[dd].last_id;
+
+	status = &txq->tx_ring[dd].upper.fields.status;
 	if (*status & E1000_TXD_STAT_DD)
 		return RTE_ETH_TX_DESC_DONE;
 
Signature · 1 line
2.11.0
#34 lore raw

PATCH v3 5/12 net/ixgbe: fix Tx descriptor status api

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The Tx descriptor status api was not behaving as expected. This API is used to inspect the content of the descriptors in the Tx ring to determine the length of the Tx queue.

Since the software advances the tail pointer and the hardware advances the head pointer, the Tx queue is located before txq->tx_tail in the ring. Therefore, a call to rte_eth_tx_descriptor_status(..., offset=20) should inspect the 20th descriptor before the tail, not after.

As before, we still need to take care about only checking descriptors that have the RS bit.

Additionally, we can avoid an access to the ring if offset is greater or equal to nb_tx_desc - nb_tx_free.

Fixes 5da8b8814178 ("net/ixgbe: implement descriptor status API")
---
 drivers/net/ixgbe/ixgbe_rxtx.c | 45 +++++++++++++++++++++++++++++++-----------
 drivers/net/ixgbe/ixgbe_rxtx.h |  1 +
 2 files changed, 34 insertions(+), 12 deletions(-)
drivers/net/ixgbe/ixgbe_rxtx.c +33 -12
index edcfa60cec98..4abc2fe37488 100644
@@ -2627,10 +2627,15 @@ ixgbe_dev_tx_queue_setup(struct rte_eth_dev *dev,
 	    hw->mac.type == ixgbe_mac_X540_vf ||
 	    hw->mac.type == ixgbe_mac_X550_vf ||
 	    hw->mac.type == ixgbe_mac_X550EM_x_vf ||
-	    hw->mac.type == ixgbe_mac_X550EM_a_vf)
+	    hw->mac.type == ixgbe_mac_X550EM_a_vf) {
 		txq->tdt_reg_addr = IXGBE_PCI_REG_ADDR(hw, IXGBE_VFTDT(queue_idx));
-	else
+		txq->tdh_reg_addr = IXGBE_PCI_REG_ADDR(hw,
+						       IXGBE_VFTDH(queue_idx));
+	} else {
 		txq->tdt_reg_addr = IXGBE_PCI_REG_ADDR(hw, IXGBE_TDT(txq->reg_idx));
+		txq->tdh_reg_addr = IXGBE_PCI_REG_ADDR(hw,
+						       IXGBE_TDH(txq->reg_idx));
+	}
 
 	txq->tx_ring_phys_addr = tz->iova;
 	txq->tx_ring = (union ixgbe_adv_tx_desc *) tz->addr;
@@ -3163,22 +3168,38 @@ ixgbe_dev_tx_descriptor_status(void *tx_queue, uint16_t offset)
 {
 	struct ixgbe_tx_queue *txq = tx_queue;
 	volatile uint32_t *status;
-	uint32_t desc;
+	int32_t desc, dd;
 
 	if (unlikely(offset >= txq->nb_tx_desc))
 		return -EINVAL;
+	if (offset >= txq->nb_tx_desc - txq->nb_tx_free)
+		return RTE_ETH_TX_DESC_DONE;
+
+	desc = txq->tx_tail - offset - 1;
+	if (desc < 0)
+		desc += txq->nb_tx_desc;
 
-	desc = txq->tx_tail + offset;
-	/* go to next desc that has the RS bit */
-	desc = ((desc + txq->tx_rs_thresh - 1) / txq->tx_rs_thresh) *
-		txq->tx_rs_thresh;
-	if (desc >= txq->nb_tx_desc) {
-		desc -= txq->nb_tx_desc;
-		if (desc >= txq->nb_tx_desc)
-			desc -= txq->nb_tx_desc;
+	/* offset is too small, no other way than reading PCI reg */
+	if (unlikely(offset < txq->tx_rs_thresh)) {
+		int16_t tx_head, queue_size;
+		tx_head = ixgbe_read_addr(txq->tdh_reg_addr);
+		queue_size = txq->tx_tail - tx_head;
+		if (queue_size < 0)
+			queue_size += txq->nb_tx_desc;
+		return queue_size > offset ? RTE_ETH_TX_DESC_FULL :
+			RTE_ETH_TX_DESC_DONE;
 	}
 
-	status = &txq->tx_ring[desc].wb.status;
+	/* index of the dd bit to look at */
+	dd = (desc / txq->tx_rs_thresh + 1) * txq->tx_rs_thresh - 1;
+
+	/* In full featured mode, RS bit is only set in the last descriptor */
+	/* of a multisegments packet */
+	if (!(txq->offloads == 0 &&
+	      txq->tx_rs_thresh >= RTE_PMD_IXGBE_TX_MAX_BURST))
+		dd = txq->sw_ring[dd].last_id;
+
+	status = &txq->tx_ring[dd].wb.status;
 	if (*status & rte_cpu_to_le_32(IXGBE_ADVTXD_STAT_DD))
 		return RTE_ETH_TX_DESC_DONE;
 
drivers/net/ixgbe/ixgbe_rxtx.h +1 -0
index 505d344b9cee..05fd4167576c 100644
@@ -201,6 +201,7 @@ struct ixgbe_tx_queue {
 		struct ixgbe_tx_entry_v *sw_ring_v; /**< address of SW ring for vector PMD */
 	};
 	volatile uint32_t   *tdt_reg_addr; /**< Address of TDT register. */
+	volatile uint32_t   *tdh_reg_addr; /**< Address of TDH register. */
 	uint16_t            nb_tx_desc;    /**< number of TX descriptors. */
 	uint16_t            tx_tail;       /**< current value of TDT reg. */
 	/**< Start freeing TX buffers if there are less free descriptors than
Signature · 1 line
2.11.0
#35 lore raw

PATCH v3 6/12 net/i40e: fix Tx descriptor status api

In reply to Thierry Herbelot

From: Olivier Matz <[email protected]>

The Tx descriptor status api was not behaving as expected. This API is used to inspect the content of the descriptors in the Tx ring to determine the length of the Tx queue.

Since the software advances the tail pointer and the hardware advances the head pointer, the Tx queue is located before txq->tx_tail in the ring. Therefore, a call to rte_eth_tx_descriptor_status(..., offset=20) should inspect the 20th descriptor before the tail, not after.

As before, we still need to take care about only checking descriptors that have the RS bit.

Additionally, we can avoid an access to the ring if offset is greater or equal to nb_tx_desc - nb_tx_free.

Fixes a9dd9af6f38e ("net/i40e: implement descriptor status API")
---
 drivers/net/i40e/i40e_rxtx.c | 37 +++++++++++++++++++++++++++----------
 1 file changed, 27 insertions(+), 10 deletions(-)
drivers/net/i40e/i40e_rxtx.c +27 -10
index 692c3bab4b5f..d84a97732f1e 100644
@@ -2031,22 +2031,39 @@ i40e_dev_tx_descriptor_status(void *tx_queue, uint16_t offset)
 	struct i40e_tx_queue *txq = tx_queue;
 	volatile uint64_t *status;
 	uint64_t mask, expect;
-	uint32_t desc;
+	int32_t desc, dd;
 
 	if (unlikely(offset >= txq->nb_tx_desc))
 		return -EINVAL;
+	if (offset >= txq->nb_tx_desc - txq->nb_tx_free)
+		return RTE_ETH_TX_DESC_DONE;
+
+	desc = txq->tx_tail - offset - 1;
+	if (desc < 0)
+		desc += txq->nb_tx_desc;
 
-	desc = txq->tx_tail + offset;
-	/* go to next desc that has the RS bit */
-	desc = ((desc + txq->tx_rs_thresh - 1) / txq->tx_rs_thresh) *
-		txq->tx_rs_thresh;
-	if (desc >= txq->nb_tx_desc) {
-		desc -= txq->nb_tx_desc;
-		if (desc >= txq->nb_tx_desc)
-			desc -= txq->nb_tx_desc;
+	/* offset is too small, no other way than reading PCI reg */
+	if (unlikely(offset < txq->tx_rs_thresh)) {
+		int16_t tx_head, queue_size;
+		tx_head = I40E_READ_REG(I40E_VSI_TO_HW(txq->vsi),
+					I40E_QTX_HEAD(txq->reg_idx));
+		queue_size = txq->tx_tail - tx_head;
+		if (queue_size < 0)
+			queue_size += txq->nb_tx_desc;
+		return queue_size > offset ? RTE_ETH_TX_DESC_FULL :
+			RTE_ETH_TX_DESC_DONE;
 	}
 
-	status = &txq->tx_ring[desc].cmd_type_offset_bsz;
+	/* index of the dd bit to look at */
+	dd = (desc / txq->tx_rs_thresh + 1) * txq->tx_rs_thresh - 1;
+
+	/* In full featured mode, RS bit is only set in the last descriptor */
+	/* of a multisegments packet */
+	if (!(txq->offloads == 0 &&
+	      txq->tx_rs_thresh >= RTE_PMD_I40E_TX_MAX_BURST))
+		dd = txq->sw_ring[dd].last_id;
+
+	status = &txq->tx_ring[dd].cmd_type_offset_bsz;
 	mask = rte_le_to_cpu_64(I40E_TXD_QW1_DTYPE_MASK);
 	expect = rte_cpu_to_le_64(
 		I40E_TX_DESC_DTYPE_DESC_DONE << I40E_TXD_QW1_DTYPE_SHIFT);
Signature · 1 line
2.11.0
#36 lore raw

PATCH v3 7/12 net/i40e: set speed to undefined for default case in link update

In reply to Thierry Herbelot

From: Laurent Hardy <[email protected]>

During PF/VF link update, a default speed value of 100M will be set if get_link_info has failed or speed is unknown.

Consequently if PF is put in no-carrier state, VFs will switch to "in carrier" state due to a link up + a link speed set to 100M (default value if no speed detected).

To be consistent with linux drivers on which PF and VFs are in same carrier state, sets default speed to undefined (instead of 100M) and updates a link status of VF only if link is up and speed is different from undefined.

Fixes 4861cde46116 ('i40e: new poll mode driver')
---
 drivers/net/i40e/i40e_ethdev.c    | 4 ++--
 drivers/net/i40e/i40e_ethdev_vf.c | 8 +++++---
 2 files changed, 7 insertions(+), 5 deletions(-)
drivers/net/i40e/i40e_ethdev.c +2 -2
index 4e40b7ab5250..76abe8209a10 100644
@@ -2743,7 +2743,7 @@ update_link_aq(struct i40e_hw *hw, struct rte_eth_link *link,
 		status = i40e_aq_get_link_info(hw, enable_lse,
 						&link_status, NULL);
 		if (unlikely(status != I40E_SUCCESS)) {
-			link->link_speed = ETH_SPEED_NUM_100M;
+			link->link_speed = ETH_SPEED_NUM_NONE;
 			link->link_duplex = ETH_LINK_FULL_DUPLEX;
 			PMD_DRV_LOG(ERR, "Failed to get link info");
 			return;
@@ -2777,7 +2777,7 @@ update_link_aq(struct i40e_hw *hw, struct rte_eth_link *link,
 		link->link_speed = ETH_SPEED_NUM_40G;
 		break;
 	default:
-		link->link_speed = ETH_SPEED_NUM_100M;
+		link->link_speed = ETH_SPEED_NUM_NONE;
 		break;
 	}
 }
drivers/net/i40e/i40e_ethdev_vf.c +5 -3
index 308fb9835ab1..bf707e57b29b 100644
@@ -2143,13 +2143,15 @@ i40evf_dev_link_update(struct rte_eth_dev *dev,
 		new_link.link_speed = ETH_SPEED_NUM_40G;
 		break;
 	default:
-		new_link.link_speed = ETH_SPEED_NUM_100M;
+		new_link.link_speed = ETH_SPEED_NUM_NONE;
 		break;
 	}
 	/* full duplex only */
 	new_link.link_duplex = ETH_LINK_FULL_DUPLEX;
-	new_link.link_status = vf->link_up ? ETH_LINK_UP :
-					     ETH_LINK_DOWN;
+	new_link.link_status = vf->link_up &&
+				new_link.link_speed != ETH_SPEED_NUM_NONE
+				? ETH_LINK_UP
+				: ETH_LINK_DOWN;
 	new_link.link_autoneg =
 		!(dev->data->dev_conf.link_speeds & ETH_LINK_SPEED_FIXED);
 
Signature · 1 line
2.11.0
#37 lore raw

PATCH v3 8/12 virtio: fix rx stats with vectorized functions

In reply to Thierry Herbelot

From: Thibaut Collet <[email protected]>

With vectorized functions, only the rx stats for number of packets is incremented.
Update also the other statistics.
Performance impact is about 2%

Fixes fc3d66212fed ("virtio: add vector Rx")
---
 drivers/net/virtio/virtio_rxtx.c             | 2 +-
 drivers/net/virtio/virtio_rxtx.h             | 2 ++
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 5 +++++
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 5 +++++
 4 files changed, 13 insertions(+), 1 deletion(-)
drivers/net/virtio/virtio_rxtx.c +1 -1
index 27ead19fbe81..6dd62bf51863 100644
@@ -1083,7 +1083,7 @@ virtio_discard_rxbuf_inorder(struct virtqueue *vq, struct rte_mbuf *m)
 	}
 }
 
-static inline void
+void
 virtio_update_packet_stats(struct virtnet_stats *stats, struct rte_mbuf *mbuf)
 {
 	uint32_t s = mbuf->pkt_len;
drivers/net/virtio/virtio_rxtx.h +2 -0
index 685cc4f8104c..1eb8dae227ee 100644
@@ -59,5 +59,7 @@ struct virtnet_ctl {
 };
 
 int virtio_rxq_vec_setup(struct virtnet_rx *rxvq);
+void virtio_update_packet_stats(struct virtnet_stats *stats,
+				struct rte_mbuf *mbuf);
 
 #endif /* _VIRTIO_RXTX_H_ */
drivers/net/virtio/virtio_rxtx_simple_neon.c +5 -0
index cdc2a4d28ed5..70e89fc428e0 100644
@@ -47,6 +47,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
 	struct rte_mbuf **sw_ring_end;
+	struct rte_mbuf **ref_rx_pkts;
 	uint16_t nb_pkts_received = 0;
 
 	uint8x16_t shuf_msk1 = {
@@ -105,6 +106,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
 		nb_pkts_received < nb_used;) {
 		uint64x2_t desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
@@ -204,5 +206,8 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	vq->vq_used_cons_idx += nb_pkts_received;
 	vq->vq_free_cnt += nb_pkts_received;
 	rxvq->stats.packets += nb_pkts_received;
+	for (nb_used = 0; nb_used < nb_pkts_received; nb_used++)
+		virtio_update_packet_stats(&rxvq->stats, ref_rx_pkts[nb_used]);
+
 	return nb_pkts_received;
 }
drivers/net/virtio/virtio_rxtx_simple_sse.c +5 -0
index af76708d66ae..cb1610e71563 100644
@@ -48,6 +48,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
 	struct rte_mbuf **sw_ring_end;
+	struct rte_mbuf **ref_rx_pkts;
 	uint16_t nb_pkts_received = 0;
 	__m128i shuf_msk1, shuf_msk2, len_adjust;
 
@@ -107,6 +108,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
 		nb_pkts_received < nb_used;) {
 		__m128i desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
@@ -190,5 +192,8 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	vq->vq_used_cons_idx += nb_pkts_received;
 	vq->vq_free_cnt += nb_pkts_received;
 	rxvq->stats.packets += nb_pkts_received;
+	for (nb_used = 0; nb_used < nb_pkts_received; nb_used++)
+		virtio_update_packet_stats(&rxvq->stats, ref_rx_pkts[nb_used]);
+
 	return nb_pkts_received;
 }
Signature · 1 line
2.11.0
#38 lore raw

PATCH v3 9/12 virtio: get all pending rx packets with vectorized functions

In reply to Thierry Herbelot

From: Thibaut Collet <[email protected]>

The loop to read packets does not take all packets as the number of available packets (nb_used) is decremented in the loop. Take all available packets provides a performance improvement of 3%.

Fixes fc3d66212fed ("virtio: add vector Rx")
---
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 5 +++--
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 5 +++--
 2 files changed, 6 insertions(+), 4 deletions(-)
drivers/net/virtio/virtio_rxtx_simple_neon.c +3 -2
index 70e89fc428e0..992e71f010eb 100644
@@ -42,7 +42,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct virtnet_rx *rxvq = rx_queue;
 	struct virtqueue *vq = rxvq->vq;
 	struct virtio_hw *hw = vq->hw;
-	uint16_t nb_used;
+	uint16_t nb_used, nb_total;
 	uint16_t desc_idx;
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
@@ -106,9 +106,10 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	nb_total = nb_used;
 	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
-		nb_pkts_received < nb_used;) {
+		nb_pkts_received < nb_total;) {
 		uint64x2_t desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		uint64x2_t mbp[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		uint64x2_t pkt_mb[RTE_VIRTIO_DESC_PER_LOOP];
drivers/net/virtio/virtio_rxtx_simple_sse.c +3 -2
index cb1610e71563..f9ec4ae69986 100644
@@ -43,7 +43,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct virtnet_rx *rxvq = rx_queue;
 	struct virtqueue *vq = rxvq->vq;
 	struct virtio_hw *hw = vq->hw;
-	uint16_t nb_used;
+	uint16_t nb_used, nb_total;
 	uint16_t desc_idx;
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
@@ -108,9 +108,10 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	nb_total = nb_used;
 	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
-		nb_pkts_received < nb_used;) {
+		nb_pkts_received < nb_total;) {
 		__m128i desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		__m128i mbp[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		__m128i pkt_mb[RTE_VIRTIO_DESC_PER_LOOP];
Signature · 1 line
2.11.0
#39 lore raw

PATCH v3 10/12 drivers/crypto/openssl: use a local copy for the session contexts

In reply to Thierry Herbelot

Session contexts are used for temporary storage when processing a packet.
If packets for the same session are to be processed simultaneously on multiple cores, separate contexts must be used.

Note: with openssl 1.1.1 EVP_CIPHER_CTX can no longer be defined as a variable on the stack: it must be allocated. This in turn reduces the performance.

Fixes d61f70b4c918 ('crypto/libcrypto: add driver for OpenSSL library')
---
 drivers/crypto/openssl/rte_openssl_pmd.c | 34 +++++++++++++++++++++++---------
 1 file changed, 25 insertions(+), 9 deletions(-)
drivers/crypto/openssl/rte_openssl_pmd.c +25 -9
index 2f5552840741..ce2d12347737 100644
@@ -1290,6 +1290,7 @@ process_openssl_combined_op
 	int srclen, aadlen, status = -1;
 	uint32_t offset;
 	uint8_t taglen;
+	EVP_CIPHER_CTX *ctx_copy;
 
 	/*
 	 * Segmented destination buffer is not supported for
@@ -1326,6 +1327,8 @@ process_openssl_combined_op
 	}
 
 	taglen = sess->auth.digest_length;
+	ctx_copy = EVP_CIPHER_CTX_new();
+	EVP_CIPHER_CTX_copy(ctx_copy, sess->cipher.ctx);
 
 	if (sess->cipher.direction == RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
 		if (sess->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC ||
@@ -1333,12 +1336,12 @@ process_openssl_combined_op
 			status = process_openssl_auth_encryption_gcm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, sess->cipher.ctx);
+					dst, tag, ctx_copy);
 		else
 			status = process_openssl_auth_encryption_ccm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, taglen, sess->cipher.ctx);
+					dst, tag, taglen, ctx_copy);
 
 	} else {
 		if (sess->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC ||
@@ -1346,14 +1349,15 @@ process_openssl_combined_op
 			status = process_openssl_auth_decryption_gcm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, sess->cipher.ctx);
+					dst, tag, ctx_copy);
 		else
 			status = process_openssl_auth_decryption_ccm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, taglen, sess->cipher.ctx);
+					dst, tag, taglen, ctx_copy);
 	}
 
+	EVP_CIPHER_CTX_free(ctx_copy);
 	if (status != 0) {
 		if (status == (-EFAULT) &&
 				sess->auth.operation ==
@@ -1372,6 +1376,7 @@ process_openssl_cipher_op
 {
 	uint8_t *dst, *iv;
 	int srclen, status;
+	EVP_CIPHER_CTX *ctx_copy;
 
 	/*
 	 * Segmented destination buffer is not supported for
@@ -1388,22 +1393,25 @@ process_openssl_cipher_op
 
 	iv = rte_crypto_op_ctod_offset(op, uint8_t *,
 			sess->iv.offset);
+	ctx_copy = EVP_CIPHER_CTX_new();
+	EVP_CIPHER_CTX_copy(ctx_copy, sess->cipher.ctx);
 
 	if (sess->cipher.mode == OPENSSL_CIPHER_LIB)
 		if (sess->cipher.direction == RTE_CRYPTO_CIPHER_OP_ENCRYPT)
 			status = process_openssl_cipher_encrypt(mbuf_src, dst,
 					op->sym->cipher.data.offset, iv,
-					srclen, sess->cipher.ctx);
+					srclen, ctx_copy);
 		else
 			status = process_openssl_cipher_decrypt(mbuf_src, dst,
 					op->sym->cipher.data.offset, iv,
-					srclen, sess->cipher.ctx);
+					srclen, ctx_copy);
 	else
 		status = process_openssl_cipher_des3ctr(mbuf_src, dst,
 				op->sym->cipher.data.offset, iv,
 				sess->cipher.key.data, srclen,
-				sess->cipher.ctx);
+				ctx_copy);
 
+	EVP_CIPHER_CTX_free(ctx_copy);
 	if (status != 0)
 		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
 }
@@ -1507,6 +1515,8 @@ process_openssl_auth_op(struct openssl_qp *qp, struct rte_crypto_op *op,
 {
 	uint8_t *dst;
 	int srclen, status;
+	EVP_MD_CTX *ctx_a;
+	HMAC_CTX *ctx_h;
 
 	srclen = op->sym->auth.data.length;
 
@@ -1514,14 +1524,20 @@ process_openssl_auth_op(struct openssl_qp *qp, struct rte_crypto_op *op,
 
 	switch (sess->auth.mode) {
 	case OPENSSL_AUTH_AS_AUTH:
+		ctx_a = EVP_MD_CTX_create();
+		EVP_MD_CTX_copy_ex(ctx_a, sess->auth.auth.ctx);
 		status = process_openssl_auth(mbuf_src, dst,
 				op->sym->auth.data.offset, NULL, NULL, srclen,
-				sess->auth.auth.ctx, sess->auth.auth.evp_algo);
+				ctx_a, sess->auth.auth.evp_algo);
+		EVP_MD_CTX_destroy(ctx_a);
 		break;
 	case OPENSSL_AUTH_AS_HMAC:
+		ctx_h = HMAC_CTX_new();
+		HMAC_CTX_copy(ctx_h, sess->auth.hmac.ctx);
 		status = process_openssl_auth_hmac(mbuf_src, dst,
 				op->sym->auth.data.offset, srclen,
-				sess->auth.hmac.ctx);
+				ctx_h);
+		HMAC_CTX_free(ctx_h);
 		break;
 	default:
 		status = -1;
Signature · 1 line
2.11.0
#40 lore raw

PATCH v3 11/12 drivers/crypto/dpaa_sec: update DPAA iova table in dpaa_mem_vtop

In reply to Thierry Herbelot

dpaa_sec needs translations between physical and virtual addresses. V to P translation is relatively fast, as memory is managed in contiguous segments.

The result of each V to P translation is used to update the DPAA iova table, which should be updated by a Mem event callback, but is not. Then the DPAA iova table has entries for all needed memory ranges.

With this patch, dpaa_mem_ptov will always use dpaax_iova_table_get_va, which ensures optimal performance.

Fixes 5a7dbb934d75 ('dpaa: enable dpaax library')
---
 drivers/crypto/dpaa_sec/dpaa_sec.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/crypto/dpaa_sec/dpaa_sec.c +4 -1
index 122c80a072ff..22b8b1d63ce0 100644
@@ -38,6 +38,7 @@
 #include <rte_dpaa_bus.h>
 #include <dpaa_sec.h>
 #include <dpaa_sec_log.h>
+#include <dpaax_iova_table.h>
 
 enum rta_sec_era rta_sec_era;
 
@@ -100,8 +101,10 @@ dpaa_mem_vtop(void *vaddr)
 	const struct rte_memseg *ms;
 
 	ms = rte_mem_virt2memseg(vaddr, NULL);
-	if (ms)
+	if (ms) {
+		dpaax_iova_table_update(ms->iova, (void *)ms->addr_64, ms->len);
 		return ms->iova + RTE_PTR_DIFF(vaddr, ms->addr);
+	}
 	return (size_t)NULL;
 }
 
Signature · 1 line
2.11.0
#41 lore raw

PATCH v3 12/12 drivers/crypto/octeontx: enable unbinding for the OcteonTx crypto engines

In reply to Thierry Herbelot

Like for Ethernet ports, the OcteonTx crypto engines must first be unbound from their kernel module, then rebound to vfio-pci, before being usable in DPDK.

As this capability is detected at runtime by dpdk-pmdinfo, add the info in the PMD registering directives.

Then an external script can be used for bind and unbind.

Fixes bfe2ae495ee268 ('crypto/octeontx: add PMD skeleton')
---
 drivers/crypto/octeontx/otx_cryptodev.c | 1 +
 1 file changed, 1 insertion(+)
drivers/crypto/octeontx/otx_cryptodev.c +1 -0
index fc64a5f3041f..16f1909966d0 100644
@@ -118,6 +118,7 @@ static struct cryptodev_driver otx_cryptodev_drv;
 
 RTE_PMD_REGISTER_PCI(CRYPTODEV_NAME_OCTEONTX_PMD, otx_cryptodev_pmd);
 RTE_PMD_REGISTER_PCI_TABLE(CRYPTODEV_NAME_OCTEONTX_PMD, pci_id_cpt_table);
+RTE_PMD_REGISTER_KMOD_DEP(CRYPTODEV_NAME_OCTEONTX_PMD, "* igb_uio | uio_pci_generic | vfio-pci");
 RTE_PMD_REGISTER_CRYPTO_DRIVER(otx_cryptodev_drv, otx_cryptodev_pmd.driver,
 		otx_cryptodev_driver_id);
 
Signature · 1 line
2.11.0
#42 lore raw
In reply to Thierry Herbelot
Quoted 4 lines · Thierry Herbelot

This series of patches includes fixes for issues seen with 6WIND fast path, built on DPDK.
The patches are in 6WIND version of DPDK, and should have been upstreamed a long time ago.

In fact, I will re-submit another series of patches, as some are not yet ready.

Please disregard the current series.

Thierry

Quoted 49 lines

V3 changes:
fix double increment of stats in virtio vectorized functions

V2 changes:
fix checkpatch issues

Guo Fengtian (1):
net/ixgbevf: fix stats update after a PF reset

Laurent Hardy (1):
net/i40e: set speed to undefined for default case in link update

Olivier Matz (5):
ethdev: fix description of tx descriptor status net/e1000: fix Tx descriptor status api (igb) net/e1000: fix Tx descriptor status api (em) net/ixgbe: fix Tx descriptor status api net/i40e: fix Tx descriptor status api

Thibaut Collet (2):
virtio: fix rx stats with vectorized functions
virtio: get all pending rx packets with vectorized functions

Thierry Herbelot (3):
drivers/crypto/openssl: use a local copy for the session contexts drivers/crypto/dpaa_sec: update DPAA iova table in dpaa_mem_vtop drivers/crypto/octeontx: enable unbinding for the OcteonTx crypto
engines

  drivers/crypto/dpaa_sec/dpaa_sec.c           |  5 +++-
  drivers/crypto/octeontx/otx_cryptodev.c      |  1 +
  drivers/crypto/openssl/rte_openssl_pmd.c     | 34 +++++++++++++++------
  drivers/net/e1000/em_rxtx.c                  | 33 +++++++++++++-------
  drivers/net/e1000/igb_rxtx.c                 |  9 +++---
  drivers/net/i40e/i40e_ethdev.c               |  4 +--
  drivers/net/i40e/i40e_ethdev_vf.c            |  8 +++--
  drivers/net/i40e/i40e_rxtx.c                 | 37 ++++++++++++++++-------
  drivers/net/ixgbe/ixgbe_ethdev.c             |  6 ++--
  drivers/net/ixgbe/ixgbe_rxtx.c               | 45 ++++++++++++++++++++--------
  drivers/net/ixgbe/ixgbe_rxtx.h               |  1 +
  drivers/net/virtio/virtio_rxtx.c             |  2 +-
  drivers/net/virtio/virtio_rxtx.h             |  2 ++
  drivers/net/virtio/virtio_rxtx_simple_neon.c | 10 +++++--
  drivers/net/virtio/virtio_rxtx_simple_sse.c  | 10 +++++--
  lib/librte_ethdev/rte_ethdev.h               |  4 +--
  16 files changed, 151 insertions(+), 60 deletions(-)
Signature · 2 lines
Thierry Herbelot
6WIND
#43 lore raw

08/08/2019 15:19, Thierry Herbelot:

Quoted 8 lines · Thierry Herbelot
Quoted 4 lines

This series of patches includes fixes for issues seen with 6WIND fast path, built on DPDK.
The patches are in 6WIND version of DPDK, and should have been upstreamed a long time ago.

In fact, I will re-submit another series of patches, as some are not yet ready.

No hurry to send 3 versions of this series while we are still closing 19.08.

Quoted 1 line

Please disregard the current series.

#44 lore raw
Quoted 8 lines · Thomas Monjalon

08/08/2019 15:19, Thierry Herbelot:

Quoted 6 lines · Thierry Herbelot
Quoted 4 lines

This series of patches includes fixes for issues seen with 6WIND fast path, built on DPDK.
The patches are in 6WIND version of DPDK, and should have been upstreamed a long time ago.

Quoted 4 lines
Quoted 2 lines

In fact, I will re-submit another series of patches, as some are not yet ready.

No hurry to send 3 versions of this series while we are still closing 19.08.

I can see some patches went through the virtio subtree, is this series still valid for 19.11?
Please update these patches status in patchwork.

Thanks.

--
David Marchand

#45 lore raw
Quoted 9 lines · David Marchand
Quoted 8 lines · Thomas Monjalon

08/08/2019 15:19, Thierry Herbelot:

Quoted 6 lines · Thierry Herbelot
Quoted 4 lines

This series of patches includes fixes for issues seen with 6WIND fast path, built on DPDK.
The patches are in 6WIND version of DPDK, and should have been upstreamed a long time ago.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

In fact, I will re-submit another series of patches, as some are not yet ready.

No hurry to send 3 versions of this series while we are still closing 19.08.

I can see some patches went through the virtio subtree, is this series still valid for 19.11?
Please update these patches status in patchwork.

done: all have been set as superseded.

Th

Quoted 6 lines

Thanks.

--
David Marchand

Signature · 3 lines
Thierry Herbelot
6WIND
Senior Software Engineer