Skip to contentSkip to conversation

gve: adopt page pool

PATCH Applied Praveen Kaligineedi 4 participants 10 messages Oct 14, 2024 – Oct 17, 2024

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v3 cover gve: adopt page pool

From: Harshitha Ramamurthy <[email protected]>

This patchset implements page pool support for gve. The first patch deals with movement of code to make page pool adoption easier in the next patch. The second patch adopts the page pool API. The third patch adds basic per queue stats which includes page pool allocation failures as well.

Changes in v3:
-Add patch 3 for per queue stats to track page pool alloc failures (Jakub Kicinski)
-Remove ethtool stat for page pool alloc failures in patch 2 (Jakub Kicinski)

Changes in v2:
-Set allow_direct parameter to true in napi context and false in others (Shannon Nelson)
-Set the napi pointer in page pool params (Jakub Kicinski) -Track page pool alloc failures per ring (Jakub Kicinski) -Don't exceed 80 char limit (Jakub Kicinski)

Harshitha Ramamurthy (3):
gve: move DQO rx buffer management related code to a new file
gve: adopt page pool for DQ RDA mode
gve: add support for basic queue stats

 drivers/net/ethernet/google/Kconfig           |   1 +
 drivers/net/ethernet/google/gve/Makefile      |   3 +-
 drivers/net/ethernet/google/gve/gve.h         |  36 ++
 .../ethernet/google/gve/gve_buffer_mgmt_dqo.c | 312 +++++++++++++++++
 drivers/net/ethernet/google/gve/gve_main.c    |  49 +++
 drivers/net/ethernet/google/gve/gve_rx_dqo.c  | 314 +++---------------
 6 files changed, 446 insertions(+), 269 deletions(-)
 create mode 100644 drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c
Signature · 1 line
2.47.0.rc1.288.g06298d1525-goog
#1 lore raw

PATCH v3 1/3 gve: move DQO rx buffer management related code to a new file

From: Harshitha Ramamurthy <[email protected]>

In preparation for the upcoming page pool adoption for DQO raw addressing mode, move RX buffer management code to a new file. In the follow on patches, page pool code will be added to this file.

No functional change, just movement of code.

---
 drivers/net/ethernet/google/gve/Makefile      |   3 +-
 drivers/net/ethernet/google/gve/gve.h         |  18 ++
 .../ethernet/google/gve/gve_buffer_mgmt_dqo.c | 230 ++++++++++++++++++
 drivers/net/ethernet/google/gve/gve_rx_dqo.c  | 225 -----------------
 4 files changed, 250 insertions(+), 226 deletions(-)
 create mode 100644 drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c
drivers/net/ethernet/google/gve/Makefile +2 -1
index 9ed07080b38a..4520f1c07a63 100644
@@ -1,4 +1,5 @@
 # Makefile for the Google virtual Ethernet (gve) driver
 
 obj-$(CONFIG_GVE) += gve.o
-gve-objs := gve_main.o gve_tx.o gve_tx_dqo.o gve_rx.o gve_rx_dqo.o gve_ethtool.o gve_adminq.o gve_utils.o gve_flow_rule.o
+gve-objs := gve_main.o gve_tx.o gve_tx_dqo.o gve_rx.o gve_rx_dqo.o gve_ethtool.o gve_adminq.o gve_utils.o gve_flow_rule.o \
+	    gve_buffer_mgmt_dqo.o
drivers/net/ethernet/google/gve/gve.h +18 -0
index 301fa1ea4f51..bd684c7d996a 100644
@@ -1162,6 +1162,24 @@ void gve_rx_stop_ring_gqi(struct gve_priv *priv, int idx);
 u16 gve_get_pkt_buf_size(const struct gve_priv *priv, bool enable_hplit);
 bool gve_header_split_supported(const struct gve_priv *priv);
 int gve_set_hsplit_config(struct gve_priv *priv, u8 tcp_data_split);
+/* rx buffer handling */
+int gve_buf_ref_cnt(struct gve_rx_buf_state_dqo *bs);
+void gve_free_page_dqo(struct gve_priv *priv, struct gve_rx_buf_state_dqo *bs,
+		       bool free_page);
+struct gve_rx_buf_state_dqo *gve_alloc_buf_state(struct gve_rx_ring *rx);
+bool gve_buf_state_is_allocated(struct gve_rx_ring *rx,
+				struct gve_rx_buf_state_dqo *buf_state);
+void gve_free_buf_state(struct gve_rx_ring *rx,
+			struct gve_rx_buf_state_dqo *buf_state);
+struct gve_rx_buf_state_dqo *gve_dequeue_buf_state(struct gve_rx_ring *rx,
+						   struct gve_index_list *list);
+void gve_enqueue_buf_state(struct gve_rx_ring *rx, struct gve_index_list *list,
+			   struct gve_rx_buf_state_dqo *buf_state);
+struct gve_rx_buf_state_dqo *gve_get_recycled_buf_state(struct gve_rx_ring *rx);
+int gve_alloc_page_dqo(struct gve_rx_ring *rx,
+		       struct gve_rx_buf_state_dqo *buf_state);
+void gve_try_recycle_buf(struct gve_priv *priv, struct gve_rx_ring *rx,
+			 struct gve_rx_buf_state_dqo *buf_state);
 /* Reset */
 void gve_schedule_reset(struct gve_priv *priv);
 int gve_reset(struct gve_priv *priv, bool attempt_teardown);
drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c new file +230 -0

230 lines not on this page · raw message

drivers/net/ethernet/google/gve/gve_rx_dqo.c +0 -225

237 lines not on this page · raw message

Signature · 1 line
2.47.0.rc1.288.g06298d1525-goog
#2 lore raw

PATCH v3 2/3 gve: adopt page pool for DQ RDA mode

In reply to Praveen Kaligineedi

From: Harshitha Ramamurthy <[email protected]>

For DQ queue format in raw DMA addressing(RDA) mode, implement page pool recycling of buffers by leveraging a few helper functions.

DQ QPL mode will continue to use the exisiting recycling logic. This is because in QPL mode, the pages come from a constant set of pages that the driver pre-allocates and registers with the device.

---

Changes in v3:
-Removed ethtool stat tracking page pool alloc failures (Jakub Kicinski)

Changes in v2:
-Set allow_direct parameter to true in napi context and false in others (Shannon Nelson)
-Set the napi pointer in page pool params (Jakub Kicinski) -Track page pool alloc failures per ring (Jakub Kicinski) -Don't exceed 80 char limit (Jakub Kicinski)

---
 drivers/net/ethernet/google/Kconfig           |   1 +
 drivers/net/ethernet/google/gve/gve.h         |  22 ++-
 .../ethernet/google/gve/gve_buffer_mgmt_dqo.c | 180 +++++++++++++-----
 drivers/net/ethernet/google/gve/gve_rx_dqo.c  |  89 ++++-----
 4 files changed, 198 insertions(+), 94 deletions(-)
drivers/net/ethernet/google/Kconfig +1 -0
index 8641a00f8e63..564862a57124 100644
@@ -18,6 +18,7 @@ if NET_VENDOR_GOOGLE
 config GVE
 	tristate "Google Virtual NIC (gVNIC) support"
 	depends on (PCI_MSI && (X86 || CPU_LITTLE_ENDIAN))
+	select PAGE_POOL
 	help
 	  This driver supports Google Virtual NIC (gVNIC)"
 
drivers/net/ethernet/google/gve/gve.h +20 -2
index bd684c7d996a..dd92949bb214 100644
@@ -13,6 +13,7 @@
 #include <linux/netdevice.h>
 #include <linux/pci.h>
 #include <linux/u64_stats_sync.h>
+#include <net/page_pool/helpers.h>
 #include <net/xdp.h>
 
 #include "gve_desc.h"
@@ -60,6 +61,8 @@
 
 #define GVE_DEFAULT_RX_BUFFER_OFFSET 2048
 
+#define GVE_PAGE_POOL_SIZE_MULTIPLIER 4
+
 #define GVE_FLOW_RULES_CACHE_SIZE \
 	(GVE_ADMINQ_BUFFER_SIZE / sizeof(struct gve_adminq_queried_flow_rule))
 #define GVE_FLOW_RULE_IDS_CACHE_SIZE \
@@ -102,6 +105,7 @@ struct gve_rx_slot_page_info {
 	struct page *page;
 	void *page_address;
 	u32 page_offset; /* offset to write to in page */
+	unsigned int buf_size;
 	int pagecnt_bias; /* expected pagecnt if only the driver has a ref */
 	u16 pad; /* adjustment for rx padding */
 	u8 can_flip; /* tracks if the networking stack is using the page */
@@ -273,6 +277,8 @@ struct gve_rx_ring {
 
 			/* Address info of the buffers for header-split */
 			struct gve_header_buf hdr_bufs;
+
+			struct page_pool *page_pool;
 		} dqo;
 	};
 
@@ -1176,10 +1182,22 @@ struct gve_rx_buf_state_dqo *gve_dequeue_buf_state(struct gve_rx_ring *rx,
 void gve_enqueue_buf_state(struct gve_rx_ring *rx, struct gve_index_list *list,
 			   struct gve_rx_buf_state_dqo *buf_state);
 struct gve_rx_buf_state_dqo *gve_get_recycled_buf_state(struct gve_rx_ring *rx);
-int gve_alloc_page_dqo(struct gve_rx_ring *rx,
-		       struct gve_rx_buf_state_dqo *buf_state);
 void gve_try_recycle_buf(struct gve_priv *priv, struct gve_rx_ring *rx,
 			 struct gve_rx_buf_state_dqo *buf_state);
+void gve_free_to_page_pool(struct gve_rx_ring *rx,
+			   struct gve_rx_buf_state_dqo *buf_state,
+			   bool allow_direct);
+int gve_alloc_qpl_page_dqo(struct gve_rx_ring *rx,
+			   struct gve_rx_buf_state_dqo *buf_state);
+void gve_free_qpl_page_dqo(struct gve_rx_buf_state_dqo *buf_state);
+void gve_reuse_buffer(struct gve_rx_ring *rx,
+		      struct gve_rx_buf_state_dqo *buf_state);
+void gve_free_buffer(struct gve_rx_ring *rx,
+		     struct gve_rx_buf_state_dqo *buf_state);
+int gve_alloc_buffer(struct gve_rx_ring *rx, struct gve_rx_desc_dqo *desc);
+struct page_pool *gve_rx_create_page_pool(struct gve_priv *priv,
+					  struct gve_rx_ring *rx);
+
 /* Reset */
 void gve_schedule_reset(struct gve_priv *priv);
 int gve_reset(struct gve_priv *priv, bool attempt_teardown);
drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c +131 -49

212 lines not on this page · raw message

drivers/net/ethernet/google/gve/gve_rx_dqo.c +46 -43

180 lines not on this page · raw message

Signature · 1 line
2.47.0.rc1.288.g06298d1525-goog
#3 lore raw

Re: [PATCH net-next v3 2/3] gve: adopt page pool for DQ RDA mode

On 2024/10/15 4:21, Praveen Kaligineedi wrote:

...

drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c void gve_try_recycle_buf(struct gve_priv *priv, struct gve_rx_ring *rx, · line 214

Quoted 11 lines
drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c +11 -0
+void gve_free_to_page_pool(struct gve_rx_ring *rx,
+			   struct gve_rx_buf_state_dqo *buf_state,
+			   bool allow_direct)
+{
+	struct page *page = buf_state->page_info.page;
+
+	if (!page)
+		return;
+
+	page_pool_put_page(page->pp, page, buf_state->page_info.buf_size,
+			   allow_direct);

page_pool_put_full_page() might be a better option here for now when page_pool is created with PP_FLAG_DMA_SYNC_DEV flag and frag API like page_pool_alloc() is used in gve_alloc_from_page_pool(), as explained in below:

https://lore.kernel.org/netdev/[email protected]/T/#mdaba23284a37affc2c46ef846674ae6aa49f8f04

Quoted 42 lines
drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c +42 -0
+	buf_state->page_info.page = NULL;
+}
+
+static int gve_alloc_from_page_pool(struct gve_rx_ring *rx,
+				    struct gve_rx_buf_state_dqo *buf_state)
+{
+	struct gve_priv *priv = rx->gve;
+	struct page *page;
+
+	buf_state->page_info.buf_size = priv->data_buffer_size_dqo;
+	page = page_pool_alloc(rx->dqo.page_pool,
+			       &buf_state->page_info.page_offset,
+			       &buf_state->page_info.buf_size, GFP_ATOMIC);
+
+	if (!page)
+		return -ENOMEM;
+
+	buf_state->page_info.page = page;
+	buf_state->page_info.page_address = page_address(page);
+	buf_state->addr = page_pool_get_dma_addr(page);
+
+	return 0;
+}
+
+struct page_pool *gve_rx_create_page_pool(struct gve_priv *priv,
+					  struct gve_rx_ring *rx)
+{
+	u32 ntfy_id = gve_rx_idx_to_ntfy(priv, rx->q_num);
+	struct page_pool_params pp = {
+		.flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV,
+		.order = 0,
+		.pool_size = GVE_PAGE_POOL_SIZE_MULTIPLIER * priv->rx_desc_cnt,
+		.dev = &priv->pdev->dev,
+		.netdev = priv->dev,
+		.napi = &priv->ntfy_blocks[ntfy_id].napi,
+		.max_len = PAGE_SIZE,
+		.dma_dir = DMA_FROM_DEVICE,
+	};
+
+	return page_pool_create(&pp);
+}
+
#4 lore raw

Re: [PATCH net-next v3 2/3] gve: adopt page pool for DQ RDA mode

Thanks Yunsheng. One thing that's not clear to me - the GVE driver does not call page_pool_put_page with dma_sync_size of 0 anywhere. Is this still an issue in that case?

Thanks,
Praveen

Quoted 67 lines · Yunsheng Lin

On 2024/10/15 4:21, Praveen Kaligineedi wrote:

...

drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c void gve_try_recycle_buf(struct gve_priv *priv, struct gve_rx_ring *rx, · line 214

Quoted 11 lines
drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c +11 -0
+void gve_free_to_page_pool(struct gve_rx_ring *rx,
+                        struct gve_rx_buf_state_dqo *buf_state,
+                        bool allow_direct)
+{
+     struct page *page = buf_state->page_info.page;
+
+     if (!page)
+             return;
+
+     page_pool_put_page(page->pp, page, buf_state->page_info.buf_size,
+                        allow_direct);

page_pool_put_full_page() might be a better option here for now when page_pool is created with PP_FLAG_DMA_SYNC_DEV flag and frag API like page_pool_alloc() is used in gve_alloc_from_page_pool(), as explained in below:

https://lore.kernel.org/netdev/[email protected]/T/#mdaba23284a37affc2c46ef846674ae6aa49f8f04

Quoted 42 lines
drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c +42 -0
+     buf_state->page_info.page = NULL;
+}
+
+static int gve_alloc_from_page_pool(struct gve_rx_ring *rx,
+                                 struct gve_rx_buf_state_dqo *buf_state)
+{
+     struct gve_priv *priv = rx->gve;
+     struct page *page;
+
+     buf_state->page_info.buf_size = priv->data_buffer_size_dqo;
+     page = page_pool_alloc(rx->dqo.page_pool,
+                            &buf_state->page_info.page_offset,
+                            &buf_state->page_info.buf_size, GFP_ATOMIC);
+
+     if (!page)
+             return -ENOMEM;
+
+     buf_state->page_info.page = page;
+     buf_state->page_info.page_address = page_address(page);
+     buf_state->addr = page_pool_get_dma_addr(page);
+
+     return 0;
+}
+
+struct page_pool *gve_rx_create_page_pool(struct gve_priv *priv,
+                                       struct gve_rx_ring *rx)
+{
+     u32 ntfy_id = gve_rx_idx_to_ntfy(priv, rx->q_num);
+     struct page_pool_params pp = {
+             .flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV,
+             .order = 0,
+             .pool_size = GVE_PAGE_POOL_SIZE_MULTIPLIER * priv->rx_desc_cnt,
+             .dev = &priv->pdev->dev,
+             .netdev = priv->dev,
+             .napi = &priv->ntfy_blocks[ntfy_id].napi,
+             .max_len = PAGE_SIZE,
+             .dma_dir = DMA_FROM_DEVICE,
+     };
+
+     return page_pool_create(&pp);
+}
+
#5 lore raw

Re: [PATCH net-next v3 2/3] gve: adopt page pool for DQ RDA mode

Quoted 3 lines · Praveen Kaligineedi

Thanks Yunsheng. One thing that's not clear to me - the GVE driver does not call page_pool_put_page with dma_sync_size of 0 anywhere. Is this still an issue in that case?

It depends on what's value of 'dma_sync_size', as the value of the below 'page_info.buf_size' seems to be the size of one fragment, so it might end up only doing the dma_sync operation for the first fragment, and what we want might be to dma sync all the fragments in the same page.

The doc about that in Documentation/networking/page_pool.rst seems a little outdated, but what it meant is still true as my understanding:

https://elixir.bootlin.com/linux/v6.11.3/source/Documentation/networking/page_pool.rst#L101

Quoted 73 lines

Thanks,
Praveen

Quoted 67 lines · Yunsheng Lin

On 2024/10/15 4:21, Praveen Kaligineedi wrote:

...

drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c void gve_try_recycle_buf(struct gve_priv *priv, struct gve_rx_ring *rx, · line 214

Quoted 11 lines
drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c +11 -0
+void gve_free_to_page_pool(struct gve_rx_ring *rx,
+                        struct gve_rx_buf_state_dqo *buf_state,
+                        bool allow_direct)
+{
+     struct page *page = buf_state->page_info.page;
+
+     if (!page)
+             return;
+
+     page_pool_put_page(page->pp, page, buf_state->page_info.buf_size,
+                        allow_direct);

page_pool_put_full_page() might be a better option here for now when page_pool is created with PP_FLAG_DMA_SYNC_DEV flag and frag API like page_pool_alloc() is used in gve_alloc_from_page_pool(), as explained in below:

https://lore.kernel.org/netdev/[email protected]/T/#mdaba23284a37affc2c46ef846674ae6aa49f8f04

Quoted 42 lines
drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c +42 -0
+     buf_state->page_info.page = NULL;
+}
+
+static int gve_alloc_from_page_pool(struct gve_rx_ring *rx,
+                                 struct gve_rx_buf_state_dqo *buf_state)
+{
+     struct gve_priv *priv = rx->gve;
+     struct page *page;
+
+     buf_state->page_info.buf_size = priv->data_buffer_size_dqo;
+     page = page_pool_alloc(rx->dqo.page_pool,
+                            &buf_state->page_info.page_offset,
+                            &buf_state->page_info.buf_size, GFP_ATOMIC);
+
+     if (!page)
+             return -ENOMEM;
+
+     buf_state->page_info.page = page;
+     buf_state->page_info.page_address = page_address(page);
+     buf_state->addr = page_pool_get_dma_addr(page);
+
+     return 0;
+}
+
+struct page_pool *gve_rx_create_page_pool(struct gve_priv *priv,
+                                       struct gve_rx_ring *rx)
+{
+     u32 ntfy_id = gve_rx_idx_to_ntfy(priv, rx->q_num);
+     struct page_pool_params pp = {
+             .flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV,
+             .order = 0,
+             .pool_size = GVE_PAGE_POOL_SIZE_MULTIPLIER * priv->rx_desc_cnt,
+             .dev = &priv->pdev->dev,
+             .netdev = priv->dev,
+             .napi = &priv->ntfy_blocks[ntfy_id].napi,
+             .max_len = PAGE_SIZE,
+             .dma_dir = DMA_FROM_DEVICE,
+     };
+
+     return page_pool_create(&pp);
+}
+
#6 lore raw

Re: [PATCH net-next v3 2/3] gve: adopt page pool for DQ RDA mode

Thanks Yunsheng for the clarification. It makes sense to me now. We will send a patch calling page_pool_put_full_page instead of page_pool_put_page.

--Praveen

Quoted 89 lines · Yunsheng Lin
Quoted 3 lines · Praveen Kaligineedi

Thanks Yunsheng. One thing that's not clear to me - the GVE driver does not call page_pool_put_page with dma_sync_size of 0 anywhere. Is this still an issue in that case?

It depends on what's value of 'dma_sync_size', as the value of the below 'page_info.buf_size' seems to be the size of one fragment, so it might end up only doing the dma_sync operation for the first fragment, and what we want might be to dma sync all the fragments in the same page.

The doc about that in Documentation/networking/page_pool.rst seems a little outdated, but what it meant is still true as my understanding:

https://elixir.bootlin.com/linux/v6.11.3/source/Documentation/networking/page_pool.rst#L101

Quoted 73 lines

Thanks,
Praveen

Quoted 67 lines · Yunsheng Lin

On 2024/10/15 4:21, Praveen Kaligineedi wrote:

...

drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c void gve_try_recycle_buf(struct gve_priv *priv, struct gve_rx_ring *rx, · line 214

Quoted 11 lines
drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c +11 -0
+void gve_free_to_page_pool(struct gve_rx_ring *rx,
+                        struct gve_rx_buf_state_dqo *buf_state,
+                        bool allow_direct)
+{
+     struct page *page = buf_state->page_info.page;
+
+     if (!page)
+             return;
+
+     page_pool_put_page(page->pp, page, buf_state->page_info.buf_size,
+                        allow_direct);

page_pool_put_full_page() might be a better option here for now when page_pool is created with PP_FLAG_DMA_SYNC_DEV flag and frag API like page_pool_alloc() is used in gve_alloc_from_page_pool(), as explained in below:

https://lore.kernel.org/netdev/[email protected]/T/#mdaba23284a37affc2c46ef846674ae6aa49f8f04

Quoted 42 lines
drivers/net/ethernet/google/gve/gve_buffer_mgmt_dqo.c +42 -0
+     buf_state->page_info.page = NULL;
+}
+
+static int gve_alloc_from_page_pool(struct gve_rx_ring *rx,
+                                 struct gve_rx_buf_state_dqo *buf_state)
+{
+     struct gve_priv *priv = rx->gve;
+     struct page *page;
+
+     buf_state->page_info.buf_size = priv->data_buffer_size_dqo;
+     page = page_pool_alloc(rx->dqo.page_pool,
+                            &buf_state->page_info.page_offset,
+                            &buf_state->page_info.buf_size, GFP_ATOMIC);
+
+     if (!page)
+             return -ENOMEM;
+
+     buf_state->page_info.page = page;
+     buf_state->page_info.page_address = page_address(page);
+     buf_state->addr = page_pool_get_dma_addr(page);
+
+     return 0;
+}
+
+struct page_pool *gve_rx_create_page_pool(struct gve_priv *priv,
+                                       struct gve_rx_ring *rx)
+{
+     u32 ntfy_id = gve_rx_idx_to_ntfy(priv, rx->q_num);
+     struct page_pool_params pp = {
+             .flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV,
+             .order = 0,
+             .pool_size = GVE_PAGE_POOL_SIZE_MULTIPLIER * priv->rx_desc_cnt,
+             .dev = &priv->pdev->dev,
+             .netdev = priv->dev,
+             .napi = &priv->ntfy_blocks[ntfy_id].napi,
+             .max_len = PAGE_SIZE,
+             .dma_dir = DMA_FROM_DEVICE,
+     };
+
+     return page_pool_create(&pp);
+}
+
#7 lore raw

PATCH v3 3/3 gve: add support for basic queue stats

In reply to Praveen Kaligineedi

From: Harshitha Ramamurthy <[email protected]>

Implement netdev_stats_ops to export basic per-queue stats.

With page pool support for DQO added in the previous patches, rx-alloc-fail captures failures in page pool allocations as well since the rx_buf_alloc_fail stat tracked in the driver is incremented when gve_alloc_buffer returns error.

---

Changes in v3:
-Add this patch to the series for per queue stats (Jakub Kicinski)

---
 drivers/net/ethernet/google/gve/gve_main.c | 49 ++++++++++++++++++++++
 1 file changed, 49 insertions(+)
drivers/net/ethernet/google/gve/gve_main.c +49 -0
index 294ddcd0bf6c..e171ca248f9a 100644
@@ -2561,6 +2561,54 @@ static const struct netdev_queue_mgmt_ops gve_queue_mgmt_ops = {
 	.ndo_queue_stop		=	gve_rx_queue_stop,
 };
 
+static void gve_get_rx_queue_stats(struct net_device *dev, int idx,
+				   struct netdev_queue_stats_rx *rx_stats)
+{
+	struct gve_priv *priv = netdev_priv(dev);
+	struct gve_rx_ring *rx = &priv->rx[idx];
+	unsigned int start;
+
+	do {
+		start = u64_stats_fetch_begin(&rx->statss);
+		rx_stats->packets = rx->rpackets;
+		rx_stats->bytes = rx->rbytes;
+		rx_stats->alloc_fail = rx->rx_skb_alloc_fail +
+				       rx->rx_buf_alloc_fail;
+	} while (u64_stats_fetch_retry(&rx->statss, start));
+}
+
+static void gve_get_tx_queue_stats(struct net_device *dev, int idx,
+				   struct netdev_queue_stats_tx *tx_stats)
+{
+	struct gve_priv *priv = netdev_priv(dev);
+	struct gve_tx_ring *tx = &priv->tx[idx];
+	unsigned int start;
+
+	do {
+		start = u64_stats_fetch_begin(&tx->statss);
+		tx_stats->packets = tx->pkt_done;
+		tx_stats->bytes = tx->bytes_done;
+	} while (u64_stats_fetch_retry(&tx->statss, start));
+}
+
+static void gve_get_base_stats(struct net_device *dev,
+			       struct netdev_queue_stats_rx *rx,
+			       struct netdev_queue_stats_tx *tx)
+{
+	rx->packets = 0;
+	rx->bytes = 0;
+	rx->alloc_fail = 0;
+
+	tx->packets = 0;
+	tx->bytes = 0;
+}
+
+static const struct netdev_stat_ops gve_stat_ops = {
+	.get_queue_stats_rx	= gve_get_rx_queue_stats,
+	.get_queue_stats_tx	= gve_get_tx_queue_stats,
+	.get_base_stats		= gve_get_base_stats,
+};
+
 static int gve_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 {
 	int max_tx_queues, max_rx_queues;
@@ -2616,6 +2664,7 @@ static int gve_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 	dev->ethtool_ops = &gve_ethtool_ops;
 	dev->netdev_ops = &gve_netdev_ops;
 	dev->queue_mgmt_ops = &gve_queue_mgmt_ops;
+	dev->stat_ops = &gve_stat_ops;
 
 	/* Set default and supported features.
 	 *
Signature · 1 line
2.47.0.rc1.288.g06298d1525-goog
Reviewed-by on the v3 series
#8 lore raw
In reply to Praveen Kaligineedi
Quoted 19 lines · Praveen Kaligineedi

From: Harshitha Ramamurthy <[email protected]>

This patchset implements page pool support for gve. The first patch deals with movement of code to make page pool adoption easier in the next patch. The second patch adopts the page pool API. The third patch adds basic per queue stats which includes page pool allocation failures as well.

Changes in v3:
-Add patch 3 for per queue stats to track page pool alloc failures (Jakub Kicinski)
-Remove ethtool stat for page pool alloc failures in patch 2 (Jakub Kicinski)

Changes in v2:
-Set allow_direct parameter to true in napi context and false in others (Shannon Nelson)

Review

Quoted 3 lines
(quoted diff) +0 -3
-Set the napi pointer in page pool params (Jakub Kicinski)
-Track page pool alloc failures per ring (Jakub Kicinski)
-Don't exceed 80 char limit (Jakub Kicinski)
#9 lore raw
In reply to Praveen Kaligineedi
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Quoted 10 lines · On Mon, 14 Oct 2024 13:21:05 -0700 you wrote:

From: Harshitha Ramamurthy <[email protected]>

This patchset implements page pool support for gve. The first patch deals with movement of code to make page pool adoption easier in the next patch. The second patch adopts the page pool API. The third patch adds basic per queue stats which includes page pool allocation failures as well.

[...]

Here is the summary with links:
- [net-next,v3,1/3] gve: move DQO rx buffer management related code to a new file

    https://git.kernel.org/netdev/net-next/c/93c68f1275f9
  - [net-next,v3,2/3] gve: adopt page pool for DQ RDA mode
    https://git.kernel.org/netdev/net-next/c/ebdfae0d377b
  - [net-next,v3,3/3] gve: add support for basic queue stats
    https://git.kernel.org/netdev/net-next/c/2e5e0932dff5

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html