Skip to contentSkip to conversation

net/ena: fixes and missing version/doc udate

PATCH In review Michal Krawczyk 2 participants 8 messages Oct 30, 2020 Network drivers

lore
Share & tools
lore
#0 lore raw

cover net/ena: fixes and missing version/doc udate

Hi,

This patchset contains few fixes, and two of them are very important:
* Fix for setting Rx checksum flags
* align IO cq allocation to 4k

Also it adds missing release notes regarding the ENI metrics which should go with the patches upstreamed previously, and also increases the driver version to v2.2.0

Michal Krawczyk (6):
net/ena: fix getting xstats global stats offset net/ena: fix setting Rx checksum flags in mbuf net/ena: change name of the supported PCI device IDs net/ena/base: align IO cq allocation to a 4K
doc: mark ARMv8 as supported by ENA PMD net/ena: upgrade driver version to v2.2.0

 doc/guides/nics/ena.rst                |  2 +-
 doc/guides/nics/features/ena.ini       |  1 +
 doc/guides/rel_notes/release_20_11.rst |  7 +++++
 drivers/net/ena/base/ena_com.c         | 26 ++++++++++---------
 drivers/net/ena/base/ena_com.h         |  2 ++
 drivers/net/ena/base/ena_plat_dpdk.h   | 36 ++++++++++++++++++++------
 drivers/net/ena/ena_ethdev.c           | 28 ++++++++++----------
 7 files changed, 68 insertions(+), 34 deletions(-)
Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/6 net/ena: fix getting xstats global stats offset

There was a bug in a code, which was reading stat_offset value from the ena_stats_rx_strings array instead of ena_stats_global_strings.

It wasn't causing real problems just because ena_stats_rx_strings was not smaller than ena_stats_global_strings and both arrays hold the same offsets.

Fixes 7830e905b7c9 ("net/ena: expose extended stats")
---
 drivers/net/ena/ena_ethdev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ena/ena_ethdev.c +1 -1
index c513faf202..7ab9b93ba9 100644
@@ -2724,7 +2724,7 @@ static int ena_xstats_get(struct rte_eth_dev *dev,
 		return 0;
 
 	for (stat = 0; stat < ENA_STATS_ARRAY_GLOBAL; stat++, count++) {
-		stat_offset = ena_stats_rx_strings[stat].stat_offset;
+		stat_offset = ena_stats_global_strings[stat].stat_offset;
 		stats_begin = &adapter->dev_stats;
 
 		xstats[count].id = count;
Signature · 1 line
2.25.1
#2 lore raw

PATCH 2/6 net/ena: fix setting Rx checksum flags in mbuf

In reply to Michal Krawczyk

The driver was never setting PKT_RX_*_CKSUM_GOOD flags, so the only way of checking if the checksum was checked was by testing for the PKT_RX_*_CKSUM_BAD. In that situation, the application couldn't detect if the checksum was valid or unknown, as unknown flag is equal to 0.

Moreover, the l3_csum_err value is only valid if the l3_proto is indicating IPv4, so it shouldn't be checked for other protocols.

Fixes 1173fca25af9 ("ena: add polling-mode driver")
---
 drivers/net/ena/ena_ethdev.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
drivers/net/ena/ena_ethdev.c +10 -8
index 7ab9b93ba9..ad593c882c 100644
@@ -296,21 +296,23 @@ static inline void ena_rx_mbuf_prepare(struct rte_mbuf *mbuf,
 	else if (ena_rx_ctx->l4_proto == ENA_ETH_IO_L4_PROTO_UDP)
 		packet_type |= RTE_PTYPE_L4_UDP;
 
-	if (ena_rx_ctx->l3_proto == ENA_ETH_IO_L3_PROTO_IPV4)
+	if (ena_rx_ctx->l3_proto == ENA_ETH_IO_L3_PROTO_IPV4) {
 		packet_type |= RTE_PTYPE_L3_IPV4;
-	else if (ena_rx_ctx->l3_proto == ENA_ETH_IO_L3_PROTO_IPV6)
+		if (unlikely(ena_rx_ctx->l3_csum_err))
+			ol_flags |= PKT_RX_IP_CKSUM_BAD;
+		else
+			ol_flags |= PKT_RX_IP_CKSUM_GOOD;
+	} else if (ena_rx_ctx->l3_proto == ENA_ETH_IO_L3_PROTO_IPV6) {
 		packet_type |= RTE_PTYPE_L3_IPV6;
+	}
 
-	if (!ena_rx_ctx->l4_csum_checked)
+	if (!ena_rx_ctx->l4_csum_checked || ena_rx_ctx->frag)
 		ol_flags |= PKT_RX_L4_CKSUM_UNKNOWN;
 	else
-		if (unlikely(ena_rx_ctx->l4_csum_err) && !ena_rx_ctx->frag)
+		if (unlikely(ena_rx_ctx->l4_csum_err))
 			ol_flags |= PKT_RX_L4_CKSUM_BAD;
 		else
-			ol_flags |= PKT_RX_L4_CKSUM_UNKNOWN;
-
-	if (unlikely(ena_rx_ctx->l3_csum_err))
-		ol_flags |= PKT_RX_IP_CKSUM_BAD;
+			ol_flags |= PKT_RX_L4_CKSUM_GOOD;
 
 	mbuf->ol_flags = ol_flags;
 	mbuf->packet_type = packet_type;
Signature · 1 line
2.25.1
#3 lore raw

PATCH 3/6 net/ena: change name of the supported PCI device IDs

In reply to Michal Krawczyk

The ID 0xEC21 is not associated with LLQ feature of the device, so it would be misleading for the user. Because of that, the current identifier is more precise.

Together with code update, the documentation was changed to reflect current changes

---
 doc/guides/nics/ena.rst      | 2 +-
 drivers/net/ena/ena_ethdev.c | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)
doc/guides/nics/ena.rst +1 -1
index 3312acba52..d7fe32c340 100644
@@ -118,7 +118,7 @@ Supported ENA adapters
 Current ENA PMD supports the following ENA adapters including:
 
 * ``1d0f:ec20`` - ENA VF
-* ``1d0f:ec21`` - ENA VF with LLQ support
+* ``1d0f:ec21`` - ENA VF RSERV0
 
 Supported Operating Systems
 ---------------------------
drivers/net/ena/ena_ethdev.c +3 -3
index ad593c882c..b1c215b63e 100644
@@ -140,8 +140,8 @@ static const struct ena_stats ena_stats_rx_strings[] = {
 /** Vendor ID used by Amazon devices */
 #define PCI_VENDOR_ID_AMAZON 0x1D0F
 /** Amazon devices */
-#define PCI_DEVICE_ID_ENA_VF	0xEC20
-#define PCI_DEVICE_ID_ENA_LLQ_VF	0xEC21
+#define PCI_DEVICE_ID_ENA_VF		0xEC20
+#define PCI_DEVICE_ID_ENA_VF_RSERV0	0xEC21
 
 #define	ENA_TX_OFFLOAD_MASK	(\
 	PKT_TX_L4_MASK |         \
@@ -155,7 +155,7 @@ static const struct ena_stats ena_stats_rx_strings[] = {
 
 static const struct rte_pci_id pci_id_ena_map[] = {
 	{ RTE_PCI_DEVICE(PCI_VENDOR_ID_AMAZON, PCI_DEVICE_ID_ENA_VF) },
-	{ RTE_PCI_DEVICE(PCI_VENDOR_ID_AMAZON, PCI_DEVICE_ID_ENA_LLQ_VF) },
+	{ RTE_PCI_DEVICE(PCI_VENDOR_ID_AMAZON, PCI_DEVICE_ID_ENA_VF_RSERV0) },
 	{ .device_id = 0 },
 };
 
Signature · 1 line
2.25.1
#4 lore raw

PATCH 4/6 net/ena/base: align IO cq allocation to a 4K

In reply to Michal Krawczyk

Latest generation HW requires IO completion queue descriptors to be aligned to a 4K in order to achieve the best performance.

Because of that, the new allocation macros were added, which allows driver to allocate the memory with specified alignemnt.

The previous allocation macros are now wrappers around the macros doing the alignment, with the alignment value equal to cacheline size.

Fixes b68309be44c0 ("net/ena/base: update communication layer for the ENAv2")
---
 drivers/net/ena/base/ena_com.c       | 26 ++++++++++----------
 drivers/net/ena/base/ena_com.h       |  2 ++
 drivers/net/ena/base/ena_plat_dpdk.h | 36 +++++++++++++++++++++-------
 3 files changed, 44 insertions(+), 20 deletions(-)
drivers/net/ena/base/ena_com.c +14 -12
index 3686ae05c4..aae68721fb 100644
@@ -413,19 +413,21 @@ static int ena_com_init_io_cq(struct ena_com_dev *ena_dev,
 	size = io_cq->cdesc_entry_size_in_bytes * io_cq->q_depth;
 	io_cq->bus = ena_dev->bus;
 
-	ENA_MEM_ALLOC_COHERENT_NODE(ena_dev->dmadev,
-			size,
-			io_cq->cdesc_addr.virt_addr,
-			io_cq->cdesc_addr.phys_addr,
-			io_cq->cdesc_addr.mem_handle,
-			ctx->numa_node,
-			prev_node);
+	ENA_MEM_ALLOC_COHERENT_NODE_ALIGNED(ena_dev->dmadev,
+					    size,
+					    io_cq->cdesc_addr.virt_addr,
+					    io_cq->cdesc_addr.phys_addr,
+					    io_cq->cdesc_addr.mem_handle,
+					    ctx->numa_node,
+					    prev_node,
+					    ENA_CDESC_RING_SIZE_ALIGNMENT);
 	if (!io_cq->cdesc_addr.virt_addr) {
-		ENA_MEM_ALLOC_COHERENT(ena_dev->dmadev,
-				       size,
-				       io_cq->cdesc_addr.virt_addr,
-				       io_cq->cdesc_addr.phys_addr,
-				       io_cq->cdesc_addr.mem_handle);
+		ENA_MEM_ALLOC_COHERENT_ALIGNED(ena_dev->dmadev,
+					       size,
+					       io_cq->cdesc_addr.virt_addr,
+					       io_cq->cdesc_addr.phys_addr,
+					       io_cq->cdesc_addr.mem_handle,
+					       ENA_CDESC_RING_SIZE_ALIGNMENT);
 	}
 
 	if (!io_cq->cdesc_addr.virt_addr) {
drivers/net/ena/base/ena_com.h +2 -0
index 8eacaeab0e..64d8f247cb 100644
@@ -23,6 +23,8 @@
 #define ADMIN_CQ_SIZE(depth)	((depth) * sizeof(struct ena_admin_acq_entry))
 #define ADMIN_AENQ_SIZE(depth)	((depth) * sizeof(struct ena_admin_aenq_entry))
 
+#define ENA_CDESC_RING_SIZE_ALIGNMENT	(1 << 12) /* 4K */
+
 /*****************************************************************************/
 /*****************************************************************************/
 /* ENA adaptive interrupt moderation settings */
drivers/net/ena/base/ena_plat_dpdk.h +28 -8
index a6782f3732..48c77f0c19 100644
@@ -172,7 +172,8 @@ do {                                                                   \
  */
 extern rte_atomic32_t ena_alloc_cnt;
 
-#define ENA_MEM_ALLOC_COHERENT(dmadev, size, virt, phys, handle)	\
+#define ENA_MEM_ALLOC_COHERENT_ALIGNED(					\
+	dmadev, size, virt, phys, handle, alignment)			\
 	do {								\
 		const struct rte_memzone *mz = NULL;			\
 		ENA_TOUCH(dmadev); ENA_TOUCH(handle);			\
@@ -181,9 +182,10 @@ extern rte_atomic32_t ena_alloc_cnt;
 			snprintf(z_name, sizeof(z_name),		\
 			 "ena_alloc_%d",				\
 			 rte_atomic32_add_return(&ena_alloc_cnt, 1));	\
-			mz = rte_memzone_reserve(z_name, size,		\
+			mz = rte_memzone_reserve_aligned(z_name, size,	\
 					SOCKET_ID_ANY,			\
-					RTE_MEMZONE_IOVA_CONTIG);	\
+					RTE_MEMZONE_IOVA_CONTIG,	\
+					alignment);			\
 			handle = mz;					\
 		}							\
 		if (mz == NULL) {					\
@@ -195,13 +197,21 @@ extern rte_atomic32_t ena_alloc_cnt;
 			phys = mz->iova;				\
 		}							\
 	} while (0)
+#define ENA_MEM_ALLOC_COHERENT(dmadev, size, virt, phys, handle)	\
+		ENA_MEM_ALLOC_COHERENT_ALIGNED(				\
+			dmadev,						\
+			size,						\
+			virt,						\
+			phys,						\
+			handle,						\
+			RTE_CACHE_LINE_SIZE)
 #define ENA_MEM_FREE_COHERENT(dmadev, size, virt, phys, handle) 	\
 		({ ENA_TOUCH(size); ENA_TOUCH(phys);			\
 		   ENA_TOUCH(dmadev);					\
 		   rte_memzone_free(handle); })
 
-#define ENA_MEM_ALLOC_COHERENT_NODE(					\
-	dmadev, size, virt, phys, mem_handle, node, dev_node)		\
+#define ENA_MEM_ALLOC_COHERENT_NODE_ALIGNED(				\
+	dmadev, size, virt, phys, mem_handle, node, dev_node, alignment) \
 	do {								\
 		const struct rte_memzone *mz = NULL;			\
 		ENA_TOUCH(dmadev); ENA_TOUCH(dev_node);			\
@@ -210,8 +220,8 @@ extern rte_atomic32_t ena_alloc_cnt;
 			snprintf(z_name, sizeof(z_name),		\
 			 "ena_alloc_%d",				\
 			 rte_atomic32_add_return(&ena_alloc_cnt, 1));   \
-			mz = rte_memzone_reserve(z_name, size, node,	\
-				RTE_MEMZONE_IOVA_CONTIG);		\
+			mz = rte_memzone_reserve_aligned(z_name, size, node, \
+				RTE_MEMZONE_IOVA_CONTIG, alignment);	\
 			mem_handle = mz;				\
 		}							\
 		if (mz == NULL) {					\
@@ -223,7 +233,17 @@ extern rte_atomic32_t ena_alloc_cnt;
 			phys = mz->iova;				\
 		}							\
 	} while (0)
-
+#define ENA_MEM_ALLOC_COHERENT_NODE(					\
+	dmadev, size, virt, phys, mem_handle, node, dev_node)		\
+		ENA_MEM_ALLOC_COHERENT_NODE_ALIGNED(			\
+			dmadev,						\
+			size,						\
+			virt,						\
+			phys,						\
+			mem_handle,					\
+			node,						\
+			dev_node,					\
+			RTE_CACHE_LINE_SIZE)
 #define ENA_MEM_ALLOC_NODE(dmadev, size, virt, node, dev_node) \
 	do {								\
 		ENA_TOUCH(dmadev); ENA_TOUCH(dev_node);			\
Signature · 1 line
2.25.1
#5 lore raw

PATCH 5/6 doc: mark ARMv8 as supported by ENA PMD

In reply to Michal Krawczyk

The ARMv8 platform support was tested and works fine with the ENA PMD.

It can be used on the AWS a1.* and m6g.* instances.

The ARMv8 support in ENA is at least from v19.11, where the VFIO DPDK driver was fixed to work with 32-bit applications compiled for arm.

---
 doc/guides/nics/features/ena.ini | 1 +
 1 file changed, 1 insertion(+)
doc/guides/nics/features/ena.ini +1 -0
index 4300dd012c..7cd6d50f98 100644
@@ -18,6 +18,7 @@ Basic stats          = Y
 Extended stats       = Y
 Linux UIO            = Y
 Linux VFIO           = Y
+ARMv8                = Y
 x86-32               = Y
 x86-64               = Y
 Usage doc            = Y
Signature · 1 line
2.25.1
#6 lore raw

PATCH 6/6 net/ena: upgrade driver version to v2.2.0

In reply to Michal Krawczyk

The v2.2.0 adds support for network interface metrics, includes some bug fixes and updates HAL to the latest version.

---
 doc/guides/rel_notes/release_20_11.rst | 7 +++++++
 drivers/net/ena/ena_ethdev.c           | 2 +-
 2 files changed, 8 insertions(+), 1 deletion(-)
doc/guides/rel_notes/release_20_11.rst +7 -0
index c242d7409f..6148a0c2f5 100644
@@ -373,6 +373,13 @@ New Features
   leverage IOAT DMA channel with vhost asynchronous APIs.
   See the :doc:`../sample_app_ug/vhost` for more details.
 
+* **Updated the Amazon ena driver.**
+
+  Updated the ena PMD with new features and improvements, including:
+
+  * Added network interface metrics which can be read using xstats.
+
+
 Removed Items
 -------------
 
drivers/net/ena/ena_ethdev.c +1 -1
index b1c215b63e..20ff3653c6 100644
@@ -27,7 +27,7 @@
 #include <ena_eth_io_defs.h>
 
 #define DRV_MODULE_VER_MAJOR	2
-#define DRV_MODULE_VER_MINOR	1
+#define DRV_MODULE_VER_MINOR	2
 #define DRV_MODULE_VER_SUBMINOR	0
 
 #define ENA_IO_TXQ_IDX(q)	(2 * (q))
Signature · 1 line
2.25.1
#7 lore raw
In reply to Michal Krawczyk
Quoted 10 lines · Michal Krawczyk

Hi,

This patchset contains few fixes, and two of them are very important:
* Fix for setting Rx checksum flags
* align IO cq allocation to 4k

Also it adds missing release notes regarding the ENI metrics which should go with the patches upstreamed previously, and also increases the driver version to v2.2.0

Quoted 8 lines

Michal Krawczyk (6):
net/ena: fix getting xstats global stats offset net/ena: fix setting Rx checksum flags in mbuf net/ena: change name of the supported PCI device IDs net/ena/base: align IO cq allocation to a 4K
doc: mark ARMv8 as supported by ENA PMD net/ena: upgrade driver version to v2.2.0

Series applied to dpdk-next-net/main, thanks.