Skip to contentSkip to conversation

net: page_pool: add netlink-based introspection

RFC In review Jakub Kicinski 5 participants 29 messages Aug 16, 2023 – Aug 18, 2023

Subsystems (9)
lore
Share & tools
lore

Jakub Kicinski

author maintainer
#0 lore raw

cover net: page_pool: add netlink-based introspection

As the page pool functionality grows we will be increasingly constrained by the lack of any uAPI. This series is a first step towards such a uAPI, it creates a way for users to query information about page pools and associated statistics.

I'm purposefully exposing only the information which I'd find useful when monitoring production workloads.

For the SET part (which to be clear isn't addressed by this series at all) I think we'll need to turn to something more along the lines of "netdev-level policy". Instead of configuring page pools, which are somewhat ephemeral, and hard to change at runtime, we should set the "expected page pool parameters" at the netdev level, and have the driver consult those when instantiating pools. My ramblings from yesterday about Queue API may make this more or less clear...
https://lore.kernel.org/all/[email protected]/

Jakub Kicinski (13):
net: page_pool: split the page_pool_params into fast and slow
net: page_pool: avoid touching slow on the fastpath
net: page_pool: factor out uninit
net: page_pool: id the page pools
net: page_pool: record pools per netdev
net: page_pool: stash the NAPI ID for easier access
eth: link netdev to pp
net: page_pool: add nlspec for basic access to page pools
net: page_pool: implement GET in the netlink API
net: page_pool: add netlink notifications for state changes
net: page_pool: report when page pool was destroyed
net: page_pool: expose page pool stats via netlink
tools: netdev: regen after page pool changes

 Documentation/netlink/specs/netdev.yaml       | 158 +++++++
 Documentation/networking/page_pool.rst        |   5 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt.c     |   1 +
 .../net/ethernet/mellanox/mlx5/core/en_main.c |   1 +
 drivers/net/ethernet/microsoft/mana/mana_en.c |   1 +
 include/linux/list.h                          |  20 +
 include/linux/netdevice.h                     |   4 +
 include/net/page_pool/helpers.h               |   8 +-
 include/net/page_pool/types.h                 |  43 +-
 include/uapi/linux/netdev.h                   |  37 ++
 net/core/Makefile                             |   2 +-
 net/core/netdev-genl-gen.c                    |  41 ++
 net/core/netdev-genl-gen.h                    |  11 +
 net/core/page_pool.c                          |  56 ++-
 net/core/page_pool_priv.h                     |  12 +
 net/core/page_pool_user.c                     | 409 +++++++++++++++++
 tools/include/uapi/linux/netdev.h             |  37 ++
 tools/net/ynl/generated/netdev-user.c         | 415 ++++++++++++++++++
 tools/net/ynl/generated/netdev-user.h         | 169 +++++++
 19 files changed, 1391 insertions(+), 39 deletions(-)
 create mode 100644 net/core/page_pool_priv.h
 create mode 100644 net/core/page_pool_user.c
Signature · 1 line
2.41.0
author maintainer
#1 lore raw

RFC 1/13 net: page_pool: split the page_pool_params into fast and slow

struct page_pool is rather performance critical and we use 16B of the first cache line to store 2 pointers used only by test code. Future patches will add more informational (non-fast path) attributes.

It's convenient for the user of the API to not have to worry which fields are fast and which are slow path. Use struct groups to split the params into the two categories internally.

---
 include/net/page_pool/types.h | 31 +++++++++++++++++++------------
 net/core/page_pool.c          |  7 ++++---
 2 files changed, 23 insertions(+), 15 deletions(-)
include/net/page_pool/types.h +19 -12
index 887e7946a597..1c16b95de62f 100644
@@ -56,18 +56,22 @@ struct pp_alloc_cache {
  * @offset:	DMA sync address offset for PP_FLAG_DMA_SYNC_DEV
  */
 struct page_pool_params {
-	unsigned int	flags;
-	unsigned int	order;
-	unsigned int	pool_size;
-	int		nid;
-	struct device	*dev;
-	struct napi_struct *napi;
-	enum dma_data_direction dma_dir;
-	unsigned int	max_len;
-	unsigned int	offset;
+	struct_group_tagged(page_pool_params_fast, fast,
+		unsigned int	flags;
+		unsigned int	order;
+		unsigned int	pool_size;
+		int		nid;
+		struct device	*dev;
+		struct napi_struct *napi;
+		enum dma_data_direction dma_dir;
+		unsigned int	max_len;
+		unsigned int	offset;
+	);
+	struct_group_tagged(page_pool_params_slow, slow,
 /* private: used by test code only */
-	void (*init_callback)(struct page *page, void *arg);
-	void *init_arg;
+		void (*init_callback)(struct page *page, void *arg);
+		void *init_arg;
+	);
 };
 
 #ifdef CONFIG_PAGE_POOL_STATS
@@ -121,7 +125,7 @@ struct page_pool_stats {
 #endif
 
 struct page_pool {
-	struct page_pool_params p;
+	struct page_pool_params_fast p;
 
 	long frag_users;
 	struct page *frag_page;
@@ -180,6 +184,9 @@ struct page_pool {
 	refcount_t user_cnt;
 
 	u64 destroy_cnt;
+
+	/* Slow/Control-path information follows */
+	struct page_pool_params_slow slow;
 };
 
 struct page *page_pool_alloc_pages(struct page_pool *pool, gfp_t gfp);
net/core/page_pool.c +4 -3
index 77cb75e63aca..ffe7782d7fc0 100644
@@ -173,7 +173,8 @@ static int page_pool_init(struct page_pool *pool,
 {
 	unsigned int ring_qsize = 1024; /* Default */
 
-	memcpy(&pool->p, params, sizeof(pool->p));
+	memcpy(&pool->p, &params->fast, sizeof(pool->p));
+	memcpy(&pool->slow, &params->slow, sizeof(pool->slow));
 
 	/* Validate only known flags were used */
 	if (pool->p.flags & ~(PP_FLAG_ALL))
@@ -372,8 +373,8 @@ static void page_pool_set_pp_info(struct page_pool *pool,
 {
 	page->pp = pool;
 	page->pp_magic |= PP_SIGNATURE;
-	if (pool->p.init_callback)
-		pool->p.init_callback(page, pool->p.init_arg);
+	if (pool->slow.init_callback)
+		pool->slow.init_callback(page, pool->slow.init_arg);
 }
 
 static void page_pool_clear_pp_info(struct page *page)
Signature · 1 line
2.41.0
maintainer Acked-by on v1 1/13
#2 lore raw

Re: [RFC net-next 01/13] net: page_pool: split the page_pool_params into fast and slow

Quoted 9 lines · Jakub Kicinski

struct page_pool is rather performance critical and we use 16B of the first cache line to store 2 pointers used only by test code. Future patches will add more informational (non-fast path) attributes.

It's convenient for the user of the API to not have to worry which fields are fast and which are slow path. Use struct groups to split the params into the two categories internally.

Quoted 5 lines
---
  include/net/page_pool/types.h | 31 +++++++++++++++++++------------
  net/core/page_pool.c          |  7 ++++---
  2 files changed, 23 insertions(+), 15 deletions(-)

With changes in 2/13 where you add a bool to avoid accessing "slow":

maintainer Acked-by on v1 1/13
#3 lore raw

Re: [RFC net-next 01/13] net: page_pool: split the page_pool_params into fast and slow

In reply to Jakub Kicinski

Hi Jakub,

Quoted 9 lines · Jakub Kicinski

struct page_pool is rather performance critical and we use 16B of the first cache line to store 2 pointers used only by test code. Future patches will add more informational (non-fast path) attributes.

It's convenient for the user of the API to not have to worry which fields are fast and which are slow path. Use struct groups to split the params into the two categories internally.

LGTM and valuable, since we've been struggling to explain where new variables should be placed, in order to affect the cache line placement as little as possible.

Quoted 92 lines
---
 include/net/page_pool/types.h | 31 +++++++++++++++++++------------
 net/core/page_pool.c          |  7 ++++---
 2 files changed, 23 insertions(+), 15 deletions(-)
include/net/page_pool/types.h +19 -12
index 887e7946a597..1c16b95de62f 100644
@@ -56,18 +56,22 @@ struct pp_alloc_cache {
  * @offset:    DMA sync address offset for PP_FLAG_DMA_SYNC_DEV
  */
 struct page_pool_params {
-       unsigned int    flags;
-       unsigned int    order;
-       unsigned int    pool_size;
-       int             nid;
-       struct device   *dev;
-       struct napi_struct *napi;
-       enum dma_data_direction dma_dir;
-       unsigned int    max_len;
-       unsigned int    offset;
+       struct_group_tagged(page_pool_params_fast, fast,
+               unsigned int    flags;
+               unsigned int    order;
+               unsigned int    pool_size;
+               int             nid;
+               struct device   *dev;
+               struct napi_struct *napi;
+               enum dma_data_direction dma_dir;
+               unsigned int    max_len;
+               unsigned int    offset;
+       );
+       struct_group_tagged(page_pool_params_slow, slow,
 /* private: used by test code only */
-       void (*init_callback)(struct page *page, void *arg);
-       void *init_arg;
+               void (*init_callback)(struct page *page, void *arg);
+               void *init_arg;
+       );
 };
 
 #ifdef CONFIG_PAGE_POOL_STATS
@@ -121,7 +125,7 @@ struct page_pool_stats {
 #endif
 
 struct page_pool {
-       struct page_pool_params p;
+       struct page_pool_params_fast p;
 
        long frag_users;
        struct page *frag_page;
@@ -180,6 +184,9 @@ struct page_pool {
        refcount_t user_cnt;
 
        u64 destroy_cnt;
+
+       /* Slow/Control-path information follows */
+       struct page_pool_params_slow slow;
 };
 
 struct page *page_pool_alloc_pages(struct page_pool *pool, gfp_t gfp);
net/core/page_pool.c +4 -4
index 77cb75e63aca..ffe7782d7fc0 100644
@@ -173,7 +173,8 @@ static int page_pool_init(struct page_pool *pool,
 {
        unsigned int ring_qsize = 1024; /* Default */
 
-       memcpy(&pool->p, params, sizeof(pool->p));
+       memcpy(&pool->p, &params->fast, sizeof(pool->p));
+       memcpy(&pool->slow, &params->slow, sizeof(pool->slow));
 
        /* Validate only known flags were used */
        if (pool->p.flags & ~(PP_FLAG_ALL))
@@ -372,8 +373,8 @@ static void page_pool_set_pp_info(struct page_pool *pool,
 {
        page->pp = pool;
        page->pp_magic |= PP_SIGNATURE;
-       if (pool->p.init_callback)
-               pool->p.init_callback(page, pool->p.init_arg);
+       if (pool->slow.init_callback)
+               pool->slow.init_callback(page, pool->slow.init_arg);
 }
 
 static void page_pool_clear_pp_info(struct page *page)
--

2.41.0

Reviewed-by on v1 1/13
#4 lore raw

Re: [RFC net-next 01/13] net: page_pool: split the page_pool_params into fast and slow

In reply to Jakub Kicinski
Quoted 11 lines · Jakub Kicinski

struct page_pool is rather performance critical and we use 16B of the first cache line to store 2 pointers used only by test code. Future patches will add more informational (non-fast path) attributes.

It's convenient for the user of the API to not have to worry which fields are fast and which are slow path. Use struct groups to split the params into the two categories internally.

Quoted 90 lines
---
 include/net/page_pool/types.h | 31 +++++++++++++++++++------------
 net/core/page_pool.c          |  7 ++++---
 2 files changed, 23 insertions(+), 15 deletions(-)
include/net/page_pool/types.h +19 -12
index 887e7946a597..1c16b95de62f 100644
@@ -56,18 +56,22 @@ struct pp_alloc_cache {
  * @offset:    DMA sync address offset for PP_FLAG_DMA_SYNC_DEV
  */
 struct page_pool_params {
-       unsigned int    flags;
-       unsigned int    order;
-       unsigned int    pool_size;
-       int             nid;
-       struct device   *dev;
-       struct napi_struct *napi;
-       enum dma_data_direction dma_dir;
-       unsigned int    max_len;
-       unsigned int    offset;
+       struct_group_tagged(page_pool_params_fast, fast,
+               unsigned int    flags;
+               unsigned int    order;
+               unsigned int    pool_size;
+               int             nid;
+               struct device   *dev;
+               struct napi_struct *napi;
+               enum dma_data_direction dma_dir;
+               unsigned int    max_len;
+               unsigned int    offset;
+       );
+       struct_group_tagged(page_pool_params_slow, slow,
 /* private: used by test code only */
-       void (*init_callback)(struct page *page, void *arg);
-       void *init_arg;
+               void (*init_callback)(struct page *page, void *arg);
+               void *init_arg;
+       );
 };
 
 #ifdef CONFIG_PAGE_POOL_STATS
@@ -121,7 +125,7 @@ struct page_pool_stats {
 #endif
 
 struct page_pool {
-       struct page_pool_params p;
+       struct page_pool_params_fast p;
 
        long frag_users;
        struct page *frag_page;
@@ -180,6 +184,9 @@ struct page_pool {
        refcount_t user_cnt;
 
        u64 destroy_cnt;
+
+       /* Slow/Control-path information follows */
+       struct page_pool_params_slow slow;
 };
 
 struct page *page_pool_alloc_pages(struct page_pool *pool, gfp_t gfp);
net/core/page_pool.c +4 -4
index 77cb75e63aca..ffe7782d7fc0 100644
@@ -173,7 +173,8 @@ static int page_pool_init(struct page_pool *pool,
 {
        unsigned int ring_qsize = 1024; /* Default */
 
-       memcpy(&pool->p, params, sizeof(pool->p));
+       memcpy(&pool->p, &params->fast, sizeof(pool->p));
+       memcpy(&pool->slow, &params->slow, sizeof(pool->slow));
 
        /* Validate only known flags were used */
        if (pool->p.flags & ~(PP_FLAG_ALL))
@@ -372,8 +373,8 @@ static void page_pool_set_pp_info(struct page_pool *pool,
 {
        page->pp = pool;
        page->pp_magic |= PP_SIGNATURE;
-       if (pool->p.init_callback)
-               pool->p.init_callback(page, pool->p.init_arg);
+       if (pool->slow.init_callback)
+               pool->slow.init_callback(page, pool->slow.init_arg);
 }
 
 static void page_pool_clear_pp_info(struct page *page)
--

2.41.0

Signature · 2 lines
Thanks,
Mina
author maintainer
#5 lore raw

RFC 2/13 net: page_pool: avoid touching slow on the fastpath

In reply to Jakub Kicinski

To fully benefit from previous commit add one byte of state in the first cache line recording if we need to look at the slow part.

The packing isn't all that impressive right now, we create a 7B hole. I'm expecting Olek's rework will reshuffle this, anyway.

---
 include/net/page_pool/types.h | 2 ++
 net/core/page_pool.c          | 4 +++-
 2 files changed, 5 insertions(+), 1 deletion(-)
include/net/page_pool/types.h +2 -0
index 1c16b95de62f..1ac7ce25fbd4 100644
@@ -127,6 +127,8 @@ struct page_pool_stats {
 struct page_pool {
 	struct page_pool_params_fast p;
 
+	bool has_init_callback;
+
 	long frag_users;
 	struct page *frag_page;
 	unsigned int frag_offset;
net/core/page_pool.c +3 -1
index ffe7782d7fc0..2c14445a353a 100644
@@ -216,6 +216,8 @@ static int page_pool_init(struct page_pool *pool,
 	    pool->p.flags & PP_FLAG_PAGE_FRAG)
 		return -EINVAL;
 
+	pool->has_init_callback = !!pool->slow.init_callback;
+
 #ifdef CONFIG_PAGE_POOL_STATS
 	pool->recycle_stats = alloc_percpu(struct page_pool_recycle_stats);
 	if (!pool->recycle_stats)
@@ -373,7 +375,7 @@ static void page_pool_set_pp_info(struct page_pool *pool,
 {
 	page->pp = pool;
 	page->pp_magic |= PP_SIGNATURE;
-	if (pool->slow.init_callback)
+	if (pool->has_init_callback)
 		pool->slow.init_callback(page, pool->slow.init_arg);
 }
 
Signature · 1 line
2.41.0
maintainer Acked-by on v1 2/13
#6 lore raw

Re: [RFC net-next 02/13] net: page_pool: avoid touching slow on the fastpath

Quoted 8 lines · Jakub Kicinski

To fully benefit from previous commit add one byte of state in the first cache line recording if we need to look at the slow part.

The packing isn't all that impressive right now, we create a 7B hole. I'm expecting Olek's rework will reshuffle this, anyway.

Quoted 5 lines
---
  include/net/page_pool/types.h | 2 ++
  net/core/page_pool.c          | 4 +++-
  2 files changed, 5 insertions(+), 1 deletion(-)

As followup to 1/13 LGTM

maintainer Acked-by on v1 2/13
#7 lore raw

Re: [RFC net-next 02/13] net: page_pool: avoid touching slow on the fastpath

In reply to Jakub Kicinski
Quoted 51 lines · Jakub Kicinski

To fully benefit from previous commit add one byte of state in the first cache line recording if we need to look at the slow part.

The packing isn't all that impressive right now, we create a 7B hole. I'm expecting Olek's rework will reshuffle this, anyway.

---
 include/net/page_pool/types.h | 2 ++
 net/core/page_pool.c          | 4 +++-
 2 files changed, 5 insertions(+), 1 deletion(-)
include/net/page_pool/types.h +2 -0
index 1c16b95de62f..1ac7ce25fbd4 100644
@@ -127,6 +127,8 @@ struct page_pool_stats {
 struct page_pool {
        struct page_pool_params_fast p;
 
+       bool has_init_callback;
+
        long frag_users;
        struct page *frag_page;
        unsigned int frag_offset;
net/core/page_pool.c +3 -2
index ffe7782d7fc0..2c14445a353a 100644
@@ -216,6 +216,8 @@ static int page_pool_init(struct page_pool *pool,
            pool->p.flags & PP_FLAG_PAGE_FRAG)
                return -EINVAL;
 
+       pool->has_init_callback = !!pool->slow.init_callback;
+
 #ifdef CONFIG_PAGE_POOL_STATS
        pool->recycle_stats = alloc_percpu(struct page_pool_recycle_stats);
        if (!pool->recycle_stats)
@@ -373,7 +375,7 @@ static void page_pool_set_pp_info(struct page_pool *pool,
 {
        page->pp = pool;
        page->pp_magic |= PP_SIGNATURE;
-       if (pool->slow.init_callback)
+       if (pool->has_init_callback)
                pool->slow.init_callback(page, pool->slow.init_arg);
 }
 
--
Quoted 2 lines

2.41.0

Reviewed-by on v1 2/13
#8 lore raw

Re: [RFC net-next 02/13] net: page_pool: avoid touching slow on the fastpath

In reply to Jakub Kicinski
Quoted 20 lines · Jakub Kicinski

To fully benefit from previous commit add one byte of state in the first cache line recording if we need to look at the slow part.

The packing isn't all that impressive right now, we create a 7B hole. I'm expecting Olek's rework will reshuffle this, anyway.

---
 include/net/page_pool/types.h | 2 ++
 net/core/page_pool.c          | 4 +++-
 2 files changed, 5 insertions(+), 1 deletion(-)
include/net/page_pool/types.h +0 -0
index 1c16b95de62f..1ac7ce25fbd4 100644

include/net/page_pool/types.h struct page_pool_stats { · line 131

Quoted 5 lines
include/net/page_pool/types.h +2 -0
@@ -127,6 +127,8 @@ struct page_pool_stats {
 struct page_pool {
        struct page_pool_params_fast p;
 
+       bool has_init_callback;
+

Nit: I wonder if it's slightly more appropriate for this to be has_slow, given how I understand the intent of usage, but it doesn't really matter. Either way,:

Quoted 28 lines
include/net/page_pool/types.h +0 -0
        long frag_users;
        struct page *frag_page;
        unsigned int frag_offset;
net/core/page_pool.c +3 -2
index ffe7782d7fc0..2c14445a353a 100644
@@ -216,6 +216,8 @@ static int page_pool_init(struct page_pool *pool,
            pool->p.flags & PP_FLAG_PAGE_FRAG)
                return -EINVAL;
 
+       pool->has_init_callback = !!pool->slow.init_callback;
+
 #ifdef CONFIG_PAGE_POOL_STATS
        pool->recycle_stats = alloc_percpu(struct page_pool_recycle_stats);
        if (!pool->recycle_stats)
@@ -373,7 +375,7 @@ static void page_pool_set_pp_info(struct page_pool *pool,
 {
        page->pp = pool;
        page->pp_magic |= PP_SIGNATURE;
-       if (pool->slow.init_callback)
+       if (pool->has_init_callback)
                pool->slow.init_callback(page, pool->slow.init_arg);
 }
 
--

2.41.0

Signature · 2 lines
Thanks,
Mina
author maintainer
#9 lore raw

RFC 3/13 net: page_pool: factor out uninit

In reply to Jakub Kicinski

We'll soon need a fuller unwind path in page_pool_create() so create the inverse of page_pool_init().

---
 net/core/page_pool.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)
net/core/page_pool.c +13 -8
index 2c14445a353a..8e71e116224d 100644
@@ -238,6 +238,18 @@ static int page_pool_init(struct page_pool *pool,
 	return 0;
 }
 
+static void page_pool_uninit(struct page_pool *pool)
+{
+	ptr_ring_cleanup(&pool->ring, NULL);
+
+	if (pool->p.flags & PP_FLAG_DMA_MAP)
+		put_device(pool->p.dev);
+
+#ifdef CONFIG_PAGE_POOL_STATS
+	free_percpu(pool->recycle_stats);
+#endif
+}
+
 /**
  * page_pool_create() - create a page pool.
  * @params: parameters, see struct page_pool_params
@@ -806,14 +818,7 @@ static void page_pool_free(struct page_pool *pool)
 	if (pool->disconnect)
 		pool->disconnect(pool);
 
-	ptr_ring_cleanup(&pool->ring, NULL);
-
-	if (pool->p.flags & PP_FLAG_DMA_MAP)
-		put_device(pool->p.dev);
-
-#ifdef CONFIG_PAGE_POOL_STATS
-	free_percpu(pool->recycle_stats);
-#endif
+	page_pool_uninit(pool);
 	kfree(pool);
 }
 
Signature · 1 line
2.41.0
maintainer
#10 lore raw

Re: [RFC net-next 03/13] net: page_pool: factor out uninit

Hi Jakub,

[...]

net/core/page_pool.c static int page_pool_init(struct page_pool *pool, · line 251

Quoted 13 lines

}

net/core/page_pool.c +11 -0
+static void page_pool_uninit(struct page_pool *pool)
+{
+       ptr_ring_cleanup(&pool->ring, NULL);
+
+       if (pool->p.flags & PP_FLAG_DMA_MAP)
+               put_device(pool->p.dev);
+
+#ifdef CONFIG_PAGE_POOL_STATS
+       free_percpu(pool->recycle_stats);
+#endif
+}

I am not sure I am following the reasoning here. The only extra thing page_pool_free() does is disconnect the pool. So I assume no one will call page_pool_uninit() directly. Do you expect page_pool_free() to grow in the future, so factoring out the uninit makes the code easier to read?

Thanks
/Ilias

Quoted 23 lines
net/core/page_pool.c +2 -9
+
 /**
  * page_pool_create() - create a page pool.
  * @params: parameters, see struct page_pool_params
@@ -806,14 +818,7 @@ static void page_pool_free(struct page_pool *pool)
        if (pool->disconnect)
                pool->disconnect(pool);
 
-       ptr_ring_cleanup(&pool->ring, NULL);
-
-       if (pool->p.flags & PP_FLAG_DMA_MAP)
-               put_device(pool->p.dev);
-
-#ifdef CONFIG_PAGE_POOL_STATS
-       free_percpu(pool->recycle_stats);
-#endif
+       page_pool_uninit(pool);
        kfree(pool);
 }
 
--

2.41.0

author maintainer
#11 lore raw

Re: [RFC net-next 03/13] net: page_pool: factor out uninit

Quoted 9 lines · On Thu, 17 Aug 2023 10:40:09 +0300 Ilias Apalodimas wrote:
Quoted 9 lines
net/core/page_pool.c +9 -0
+static void page_pool_uninit(struct page_pool *pool)
+{
+       ptr_ring_cleanup(&pool->ring, NULL);
+
+       if (pool->p.flags & PP_FLAG_DMA_MAP)
+               put_device(pool->p.dev);
+
+#ifdef CONFIG_PAGE_POOL_STATS
+       free_percpu(pool->recycle_stats);
Quoted 8 lines

net/core/page_pool.c static int page_pool_init(struct page_pool *pool, · line 251

Quoted 2 lines
net/core/page_pool.c +2 -0
+#endif
+}  

I am not sure I am following the reasoning here. The only extra thing page_pool_free() does is disconnect the pool. So I assume no one will call page_pool_uninit() directly. Do you expect page_pool_free() to grow in the future, so factoring out the uninit makes the code easier to read?

I'm calling it from the unwind patch of page_pool_create() in the next patch, because I'm adding another setup state after page_pool_init(). I can't put the free into _uninit() because on the unwind path of _create() that's an individual step.

maintainer Reviewed-by
#12 lore raw

Re: [RFC net-next 03/13] net: page_pool: factor out uninit

Quoted 20 lines · Jakub Kicinski
Quoted 9 lines · On Thu, 17 Aug 2023 10:40:09 +0300 Ilias Apalodimas wrote:
Quoted 9 lines
net/core/page_pool.c +9 -0
+static void page_pool_uninit(struct page_pool *pool)
+{
+       ptr_ring_cleanup(&pool->ring, NULL);
+
+       if (pool->p.flags & PP_FLAG_DMA_MAP)
+               put_device(pool->p.dev);
+
+#ifdef CONFIG_PAGE_POOL_STATS
+       free_percpu(pool->recycle_stats);
Quoted 8 lines

net/core/page_pool.c static int page_pool_init(struct page_pool *pool, · line 251

Quoted 2 lines
net/core/page_pool.c +2 -0
+#endif
+}

I am not sure I am following the reasoning here. The only extra thing page_pool_free() does is disconnect the pool. So I assume no one will call page_pool_uninit() directly. Do you expect page_pool_free() to grow in the future, so factoring out the uninit makes the code easier to read?

Quoted 4 lines

I'm calling it from the unwind patch of page_pool_create() in the next patch, because I'm adding another setup state after page_pool_init(). I can't put the free into _uninit() because on the unwind path of _create() that's an individual step.

Yea fair enough, I went through that patch a few minutes ago, so this one makes sense

author maintainer
#13 lore raw

RFC 4/13 net: page_pool: id the page pools

In reply to Jakub Kicinski

To give ourselves the flexibility of creating netlink commands and ability to refer to page pool instances in uAPIs create IDs for page pools.

---
 include/net/page_pool/types.h |  4 ++++
 net/core/Makefile             |  2 +-
 net/core/page_pool.c          | 21 +++++++++++++++-----
 net/core/page_pool_priv.h     |  9 +++++++++
 net/core/page_pool_user.c     | 36 +++++++++++++++++++++++++++++++++++
 5 files changed, 66 insertions(+), 6 deletions(-)
 create mode 100644 net/core/page_pool_priv.h
 create mode 100644 net/core/page_pool_user.c
include/net/page_pool/types.h +4 -0
index 1ac7ce25fbd4..9fadf15dadfa 100644
@@ -189,6 +189,10 @@ struct page_pool {
 
 	/* Slow/Control-path information follows */
 	struct page_pool_params_slow slow;
+	/* User-facing fields, protected by page_pools_lock */
+	struct {
+		u32 id;
+	} user;
 };
 
 struct page *page_pool_alloc_pages(struct page_pool *pool, gfp_t gfp);
net/core/Makefile +1 -1
index 731db2eaa610..4ae3d83f67d5 100644
@@ -18,7 +18,7 @@ obj-y		     += dev.o dev_addr_lists.o dst.o netevent.o \
 obj-$(CONFIG_NETDEV_ADDR_LIST_TEST) += dev_addr_lists_test.o
 
 obj-y += net-sysfs.o
-obj-$(CONFIG_PAGE_POOL) += page_pool.o
+obj-$(CONFIG_PAGE_POOL) += page_pool.o page_pool_user.o
 obj-$(CONFIG_PROC_FS) += net-procfs.o
 obj-$(CONFIG_NET_PKTGEN) += pktgen.o
 obj-$(CONFIG_NETPOLL) += netpoll.o
net/core/page_pool.c +16 -5
index 8e71e116224d..de199c356043 100644
@@ -23,6 +23,8 @@
 
 #include <trace/events/page_pool.h>
 
+#include "page_pool_priv.h"
+
 #define DEFER_TIME (msecs_to_jiffies(1000))
 #define DEFER_WARN_INTERVAL (60 * HZ)
 
@@ -264,13 +266,21 @@ struct page_pool *page_pool_create(const struct page_pool_params *params)
 		return ERR_PTR(-ENOMEM);
 
 	err = page_pool_init(pool, params);
-	if (err < 0) {
-		pr_warn("%s() gave up with errno %d\n", __func__, err);
-		kfree(pool);
-		return ERR_PTR(err);
-	}
+	if (err < 0)
+		goto err_free;
+
+	err = page_pool_list(pool);
+	if (err)
+		goto err_uninit;
 
 	return pool;
+
+err_uninit:
+	page_pool_uninit(pool);
+err_free:
+	pr_warn("%s() gave up with errno %d\n", __func__, err);
+	kfree(pool);
+	return ERR_PTR(err);
 }
 EXPORT_SYMBOL(page_pool_create);
 
@@ -818,6 +828,7 @@ static void page_pool_free(struct page_pool *pool)
 	if (pool->disconnect)
 		pool->disconnect(pool);
 
+	page_pool_unlist(pool);
 	page_pool_uninit(pool);
 	kfree(pool);
 }
net/core/page_pool_priv.h new file +9 -0
new file mode 100644
index 000000000000..6c4e4aeed02a
@@ -0,0 +1,9 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#ifndef __PAGE_POOL_PRIV_H
+#define __PAGE_POOL_PRIV_H
+
+int page_pool_list(struct page_pool *pool);
+void page_pool_unlist(struct page_pool *pool);
+
+#endif
net/core/page_pool_user.c new file +36 -0
new file mode 100644
index 000000000000..af4ac38a2de1
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#include <linux/mutex.h>
+#include <linux/xarray.h>
+#include <net/page_pool/types.h>
+
+#include "page_pool_priv.h"
+
+static DEFINE_XARRAY_FLAGS(page_pools, XA_FLAGS_ALLOC1);
+static DEFINE_MUTEX(page_pools_lock);
+
+int page_pool_list(struct page_pool *pool)
+{
+	static u32 id_alloc_next;
+	int err;
+
+	mutex_lock(&page_pools_lock);
+	err = xa_alloc_cyclic(&page_pools, &pool->user.id, pool, xa_limit_32b,
+			      &id_alloc_next, GFP_KERNEL);
+	if (err < 0)
+		goto err_unlock;
+
+	mutex_unlock(&page_pools_lock);
+	return 0;
+
+err_unlock:
+	mutex_unlock(&page_pools_lock);
+	return err;
+}
+
+void page_pool_unlist(struct page_pool *pool)
+{
+	mutex_lock(&page_pools_lock);
+	xa_erase(&page_pools, pool->user.id);
+	mutex_unlock(&page_pools_lock);
+}
Signature · 1 line
2.41.0
#14 lore raw

Re: [RFC net-next 04/13] net: page_pool: id the page pools

Quoted 5 lines · Jakub Kicinski

To give ourselves the flexibility of creating netlink commands and ability to refer to page pool instances in uAPIs create IDs for page pools.

Sorry maybe I'm missing something, but it's a bit curious to me that this ID is needed. An rx queue can only ever have 1 page-pool associated with it at a time, right? Could you instead add a pointer to the page_pool into 'struct netdev_rx_queue', and then page-pool can be referred to by the netdev id & the rx-queue number? Wouldn't that make the implementation much simpler? I also believe the userspace refers to the rx-queue by its index number for the ethtool APIs like adding flow steering rules, so extending that to here maybe makes sense.

Quoted 147 lines
---
 include/net/page_pool/types.h |  4 ++++
 net/core/Makefile             |  2 +-
 net/core/page_pool.c          | 21 +++++++++++++++-----
 net/core/page_pool_priv.h     |  9 +++++++++
 net/core/page_pool_user.c     | 36 +++++++++++++++++++++++++++++++++++
 5 files changed, 66 insertions(+), 6 deletions(-)
 create mode 100644 net/core/page_pool_priv.h
 create mode 100644 net/core/page_pool_user.c
include/net/page_pool/types.h +4 -0
index 1ac7ce25fbd4..9fadf15dadfa 100644
@@ -189,6 +189,10 @@ struct page_pool {
 
        /* Slow/Control-path information follows */
        struct page_pool_params_slow slow;
+       /* User-facing fields, protected by page_pools_lock */
+       struct {
+               u32 id;
+       } user;
 };
 
 struct page *page_pool_alloc_pages(struct page_pool *pool, gfp_t gfp);
net/core/Makefile +1 -1
index 731db2eaa610..4ae3d83f67d5 100644
@@ -18,7 +18,7 @@ obj-y              += dev.o dev_addr_lists.o dst.o netevent.o \
 obj-$(CONFIG_NETDEV_ADDR_LIST_TEST) += dev_addr_lists_test.o
 
 obj-y += net-sysfs.o
-obj-$(CONFIG_PAGE_POOL) += page_pool.o
+obj-$(CONFIG_PAGE_POOL) += page_pool.o page_pool_user.o
 obj-$(CONFIG_PROC_FS) += net-procfs.o
 obj-$(CONFIG_NET_PKTGEN) += pktgen.o
 obj-$(CONFIG_NETPOLL) += netpoll.o
net/core/page_pool.c +16 -5
index 8e71e116224d..de199c356043 100644
@@ -23,6 +23,8 @@
 
 #include <trace/events/page_pool.h>
 
+#include "page_pool_priv.h"
+
 #define DEFER_TIME (msecs_to_jiffies(1000))
 #define DEFER_WARN_INTERVAL (60 * HZ)
 
@@ -264,13 +266,21 @@ struct page_pool *page_pool_create(const struct page_pool_params *params)
                return ERR_PTR(-ENOMEM);
 
        err = page_pool_init(pool, params);
-       if (err < 0) {
-               pr_warn("%s() gave up with errno %d\n", __func__, err);
-               kfree(pool);
-               return ERR_PTR(err);
-       }
+       if (err < 0)
+               goto err_free;
+
+       err = page_pool_list(pool);
+       if (err)
+               goto err_uninit;
 
        return pool;
+
+err_uninit:
+       page_pool_uninit(pool);
+err_free:
+       pr_warn("%s() gave up with errno %d\n", __func__, err);
+       kfree(pool);
+       return ERR_PTR(err);
 }
 EXPORT_SYMBOL(page_pool_create);
 
@@ -818,6 +828,7 @@ static void page_pool_free(struct page_pool *pool)
        if (pool->disconnect)
                pool->disconnect(pool);
 
+       page_pool_unlist(pool);
        page_pool_uninit(pool);
        kfree(pool);
 }
net/core/page_pool_priv.h new file +9 -0
new file mode 100644
index 000000000000..6c4e4aeed02a
@@ -0,0 +1,9 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#ifndef __PAGE_POOL_PRIV_H
+#define __PAGE_POOL_PRIV_H
+
+int page_pool_list(struct page_pool *pool);
+void page_pool_unlist(struct page_pool *pool);
+
+#endif
net/core/page_pool_user.c new file +36 -1
new file mode 100644
index 000000000000..af4ac38a2de1
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#include <linux/mutex.h>
+#include <linux/xarray.h>
+#include <net/page_pool/types.h>
+
+#include "page_pool_priv.h"
+
+static DEFINE_XARRAY_FLAGS(page_pools, XA_FLAGS_ALLOC1);
+static DEFINE_MUTEX(page_pools_lock);
+
+int page_pool_list(struct page_pool *pool)
+{
+       static u32 id_alloc_next;
+       int err;
+
+       mutex_lock(&page_pools_lock);
+       err = xa_alloc_cyclic(&page_pools, &pool->user.id, pool, xa_limit_32b,
+                             &id_alloc_next, GFP_KERNEL);
+       if (err < 0)
+               goto err_unlock;
+
+       mutex_unlock(&page_pools_lock);
+       return 0;
+
+err_unlock:
+       mutex_unlock(&page_pools_lock);
+       return err;
+}
+
+void page_pool_unlist(struct page_pool *pool)
+{
+       mutex_lock(&page_pools_lock);
+       xa_erase(&page_pools, pool->user.id);
+       mutex_unlock(&page_pools_lock);
+}
--

2.41.0

Signature · 2 lines
Thanks,
Mina
author maintainer
#15 lore raw

Re: [RFC net-next 04/13] net: page_pool: id the page pools

Quoted 2 lines · On Thu, 17 Aug 2023 14:56:01 -0700 Mina Almasry wrote:

Sorry maybe I'm missing something, but it's a bit curious to me that this ID is needed.

Right.. now I realize that I haven't explained the queue side.

I expect a similar API will be added to dump queue info, and then page pool ID will be added to the queue object, to link which queue is being fed from which page pool. I guess I should have said that in the cover letter...

Quoted 3 lines

An rx queue can only ever have 1 page-pool
associated with it at a time, right? Could you instead add a pointer to the page_pool into 'struct netdev_rx_queue',

100% my intention, which is why I moved the location of that struct recently :)

Quoted 2 lines

and then page-pool can be referred to by the netdev id & the rx-queue number?

And use/type (headers vs payloads). And then nothing stops a driver from using one page pool for multiple queues of the same type within a NAPI. And then the page pools can die and linger (see patch 11).

Quoted 4 lines

Wouldn't that make the implementation much simpler? I also believe the userspace refers to the rx-queue by its index number for the ethtool APIs like adding flow steering rules, so extending that to here maybe makes sense.

author maintainer
#16 lore raw

RFC 5/13 net: page_pool: record pools per netdev

In reply to Jakub Kicinski

Link the page pools with netdevs. This needs to be netns compatible so we have two options. Either we record the pools per netns and have to worry about moving them as the netdev gets moved. Or we record them directly on the netdev so they move with the netdev without any extra work.

Implement the latter option. Since pools may outlast netdev we need a place to store orphans. In time honored tradition use loopback for this purpose.

---
 include/linux/list.h          | 20 ++++++++
 include/linux/netdevice.h     |  4 ++
 include/net/page_pool/types.h |  4 ++
 net/core/page_pool_user.c     | 86 +++++++++++++++++++++++++++++++++++
 4 files changed, 114 insertions(+)
include/linux/list.h +20 -0
index f10344dbad4d..a65e3017a39b 100644
@@ -1030,6 +1030,26 @@ static inline void hlist_move_list(struct hlist_head *old,
 	old->first = NULL;
 }
 
+/**
+ * hlist_splice_init() - move all entries from one list to another
+ * @from: hlist_head from which entries will be moved
+ * @last: last entry on the @from list
+ * @to:   hlist_head to which entries will be moved
+ *
+ * @to can be empty, @from must contain at least @last.
+ */
+static inline void hlist_splice_init(struct hlist_head *from,
+				     struct hlist_node *last,
+				     struct hlist_head *to)
+{
+	if (to->first)
+		to->first->pprev = &last->next;
+	last->next = to->first;
+	to->first = from->first;
+	from->first->pprev = &to->first;
+	from->first = NULL;
+}
+
 #define hlist_entry(ptr, type, member) container_of(ptr,type,member)
 
 #define hlist_for_each(pos, head) \
include/linux/netdevice.h +4 -0
index 0896aaa91dd7..7576cd43b49e 100644
@@ -2405,6 +2405,10 @@ struct net_device {
 	struct rtnl_hw_stats64	*offload_xstats_l3;
 
 	struct devlink_port	*devlink_port;
+#if IS_ENABLED(CONFIG_PAGE_POOL)
+	/** @page_pools: page pools created for this netdevice */
+	struct hlist_head	page_pools;
+#endif
 };
 #define to_net_dev(d) container_of(d, struct net_device, dev)
 
include/net/page_pool/types.h +4 -0
index 9fadf15dadfa..b9db612708e4 100644
@@ -5,6 +5,7 @@
 
 #include <linux/dma-direction.h>
 #include <linux/ptr_ring.h>
+#include <linux/types.h>
 
 #define PP_FLAG_DMA_MAP		BIT(0) /* Should page_pool do the DMA
 					* map/unmap
@@ -50,6 +51,7 @@ struct pp_alloc_cache {
  * @pool_size:	size of the ptr_ring
  * @nid:	NUMA node id to allocate from pages from
  * @dev:	device, for DMA pre-mapping purposes
+ * @netdev:	netdev this pool will serve (leave as NULL if none or multiple)
  * @napi:	NAPI which is the sole consumer of pages, otherwise NULL
  * @dma_dir:	DMA mapping direction
  * @max_len:	max DMA sync memory size for PP_FLAG_DMA_SYNC_DEV
@@ -68,6 +70,7 @@ struct page_pool_params {
 		unsigned int	offset;
 	);
 	struct_group_tagged(page_pool_params_slow, slow,
+		struct net_device *netdev;
 /* private: used by test code only */
 		void (*init_callback)(struct page *page, void *arg);
 		void *init_arg;
@@ -191,6 +194,7 @@ struct page_pool {
 	struct page_pool_params_slow slow;
 	/* User-facing fields, protected by page_pools_lock */
 	struct {
+		struct hlist_node list;
 		u32 id;
 	} user;
 };
net/core/page_pool_user.c +86 -0

111 lines not on this page · raw message

Signature · 1 line
2.41.0
reviewer
#17 lore raw

Re: [RFC net-next 05/13] net: page_pool: record pools per netdev

Quoted 11 lines · Jakub Kicinski

Link the page pools with netdevs. This needs to be netns compatible so we have two options. Either we record the pools per netns and have to worry about moving them as the netdev gets moved. Or we record them directly on the netdev so they move with the netdev without any extra work.

Implement the latter option. Since pools may outlast netdev we need a place to store orphans. In time honored tradition use loopback for this purpose.

...

Quoted 1 line
net/core/page_pool_user.c +0 -0

...

Quoted 10 lines
net/core/page_pool_user.c +10 -0
+static void page_pool_unreg_netdev(struct net_device *netdev)
+{
+	struct page_pool *pool, *last;
+	struct net_device *lo;
+
+	lo = __dev_get_by_index(dev_net(netdev), 1);
+	if (!lo) {
+		netdev_err_once(netdev,
+				"can't get lo to store orphan page pools\n");
+		page_pool_unreg_netdev_wipe(netdev);

net/core/page_pool_user.c void page_pool_unlist(struct page_pool *pool) · line 89

Quoted 10 lines
net/core/page_pool_user.c +10 -0
+		return;
+	}
+
+	mutex_lock(&page_pools_lock);
+	hlist_for_each_entry(pool, &netdev->page_pools, user.list) {
+		pool->slow.netdev = lo;
+		last = pool;
+	}
+
+	hlist_splice_init(&netdev->page_pools, &last->user.list,

Hi Jakub.

I'm not sure if it is possible, but if the hlist loop above iterates zero times then last will be uninitialised here.

Flagged by Smatch.

Quoted 3 lines
net/core/page_pool_user.c +3 -0
+			  &lo->page_pools);
+	mutex_unlock(&page_pools_lock);
+}

...

author maintainer
#18 lore raw

Re: [RFC net-next 05/13] net: page_pool: record pools per netdev

Quoted 4 lines · On Thu, 17 Aug 2023 09:26:09 +0200 Simon Horman wrote:

I'm not sure if it is possible, but if the hlist loop above iterates zero times then last will be uninitialised here.

Flagged by Smatch.

Hm, the caller checks if the list is empty but there may be a race condition since we don't hold the lock, yet. Thanks for flagging!

author maintainer
#19 lore raw

RFC 6/13 net: page_pool: stash the NAPI ID for easier access

In reply to Jakub Kicinski

To avoid any issues with race conditions on accessing napi and having to think about the lifetime of NAPI objects in netlink GET - stash the napi_id to which page pool was linked at creation time.

---
 include/net/page_pool/types.h | 1 +
 net/core/page_pool_user.c     | 4 +++-
 2 files changed, 4 insertions(+), 1 deletion(-)
include/net/page_pool/types.h +1 -0
index b9db612708e4..3017557e0c59 100644
@@ -195,6 +195,7 @@ struct page_pool {
 	/* User-facing fields, protected by page_pools_lock */
 	struct {
 		struct hlist_node list;
+		u32 napi_id;
 		u32 id;
 	} user;
 };
net/core/page_pool_user.c +3 -1
index 25977ce18e2b..6632fc19a86f 100644
@@ -37,9 +37,11 @@ int page_pool_list(struct page_pool *pool)
 	if (err < 0)
 		goto err_unlock;
 
-	if (pool->slow.netdev)
+	if (pool->slow.netdev) {
 		hlist_add_head(&pool->user.list,
 			       &pool->slow.netdev->page_pools);
+		pool->user.napi_id = pool->p.napi ? pool->p.napi->napi_id : 0;
+	}
 
 	mutex_unlock(&page_pools_lock);
 	return 0;
Signature · 1 line
2.41.0
author maintainer
#20 lore raw

RFC 7/13 eth: link netdev to pp

In reply to Jakub Kicinski

Link page pool instances to netdev for the drivers which already link to NAPI. Unless the driver is doing something very weird per-NAPI should imply per-netdev.

---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c         | 1 +
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 1 +
 drivers/net/ethernet/microsoft/mana/mana_en.c     | 1 +
 3 files changed, 3 insertions(+)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +1 -0
index 7be917a8da48..06776196c05e 100644
@@ -3247,6 +3247,7 @@ static int bnxt_alloc_rx_page_pool(struct bnxt *bp,
 	pp.pool_size = bp->rx_ring_size;
 	pp.nid = dev_to_node(&bp->pdev->dev);
 	pp.napi = &rxr->bnapi->napi;
+	pp.netdev = bp->dev;
 	pp.dev = &bp->pdev->dev;
 	pp.dma_dir = DMA_BIDIRECTIONAL;
 	if (PAGE_SIZE > BNXT_RX_PAGE_SIZE)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +1 -0
index bc9d5a5bea01..f9ca600f47e2 100644
@@ -839,6 +839,7 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params,
 		pp_params.nid       = node;
 		pp_params.dev       = rq->pdev;
 		pp_params.napi      = rq->cq.napi;
+		pp_params.netdev    = rq->netdev;
 		pp_params.dma_dir   = rq->buff.map_dir;
 		pp_params.max_len   = PAGE_SIZE;
 
drivers/net/ethernet/microsoft/mana/mana_en.c +1 -0
index 4a16ebff3d1d..25286490e237 100644
@@ -2056,6 +2056,7 @@ static int mana_create_page_pool(struct mana_rxq *rxq, struct gdma_context *gc)
 	pprm.pool_size = RX_BUFFERS_PER_QUEUE;
 	pprm.nid = gc->numa_node;
 	pprm.napi = &rxq->rx_cq.napi;
+	pprm.netdev = rxq->ndev;
 
 	rxq->page_pool = page_pool_create(&pprm);
 
Signature · 1 line
2.41.0
author maintainer
#21 lore raw

RFC 8/13 net: page_pool: add nlspec for basic access to page pools

In reply to Jakub Kicinski

Add a Netlink spec in YAML for getting very basic information about page pools.

---
 Documentation/netlink/specs/netdev.yaml | 41 +++++++++++++++++++++++++
 1 file changed, 41 insertions(+)
Documentation/netlink/specs/netdev.yaml +41 -0
index 1c7284fd535b..ded77e61df7b 100644
@@ -68,6 +68,30 @@ name: netdev
         type: u32
         checks:
           min: 1
+  -
+    name: page-pool
+    attributes:
+      -
+        name: id
+        doc: Unique ID of a Page Pool instance.
+        type: u32
+        checks:
+          min: 1
+      -
+        name: pad
+        type: pad
+      -
+        name: ifindex
+        doc: |
+          ifindex of the netdev to which the pool belongs.
+          May be reported as 0 if the page pool was allocated for a netdev
+          which got destroyed already (page pools may outlast their netdevs
+          because they wait for all memory to be returned).
+        type: u32
+      -
+        name: napi-id
+        doc: Id of NAPI using this Page Pool instance.
+        type: u32
 
 operations:
   list:
@@ -101,6 +125,23 @@ name: netdev
       doc: Notification about device configuration being changed.
       notify: dev-get
       mcgrp: mgmt
+    -
+      name: page-pool-get
+      doc: |
+        Get / dump information about Page Pools.
+        (Only Page Pools associated with a net_device can be listed.)
+      attribute-set: page-pool
+      do:
+        request:
+          attributes:
+            - id
+        reply: &pp-reply
+          attributes:
+            - id
+            - ifindex
+            - napi-id
+      dump:
+        reply: *pp-reply
 
 mcast-groups:
   list:
Signature · 1 line
2.41.0
author maintainer
#22 lore raw

RFC 9/13 net: page_pool: implement GET in the netlink API

In reply to Jakub Kicinski

Expose the very basic page pool information via netlink.

Example using ynl-py for a system with 9 queues:

$ ./cli.py --no-schema --spec netlink/specs/netdev.yaml \
           --dump page-pool-get
[{'id': 19, 'ifindex': 2, 'napi-id': 147},
 {'id': 18, 'ifindex': 2, 'napi-id': 146},
 {'id': 17, 'ifindex': 2, 'napi-id': 145},
 {'id': 16, 'ifindex': 2, 'napi-id': 144},
 {'id': 15, 'ifindex': 2, 'napi-id': 143},
 {'id': 14, 'ifindex': 2, 'napi-id': 142},
 {'id': 13, 'ifindex': 2, 'napi-id': 141},
 {'id': 12, 'ifindex': 2, 'napi-id': 140},
 {'id': 11, 'ifindex': 2, 'napi-id': 139},
 {'id': 10, 'ifindex': 2, 'napi-id': 138}]
---
 include/uapi/linux/netdev.h |  11 ++++
 net/core/netdev-genl-gen.c  |  17 +++++
 net/core/netdev-genl-gen.h  |   3 +
 net/core/page_pool_user.c   | 127 ++++++++++++++++++++++++++++++++++++
 4 files changed, 158 insertions(+)
include/uapi/linux/netdev.h +11 -0
index c1634b95c223..98999e0650a3 100644
@@ -48,11 +48,22 @@ enum {
 	NETDEV_A_DEV_MAX = (__NETDEV_A_DEV_MAX - 1)
 };
 
+enum {
+	NETDEV_A_PAGE_POOL_ID = 1,
+	NETDEV_A_PAGE_POOL_PAD,
+	NETDEV_A_PAGE_POOL_IFINDEX,
+	NETDEV_A_PAGE_POOL_NAPI_ID,
+
+	__NETDEV_A_PAGE_POOL_MAX,
+	NETDEV_A_PAGE_POOL_MAX = (__NETDEV_A_PAGE_POOL_MAX - 1)
+};
+
 enum {
 	NETDEV_CMD_DEV_GET = 1,
 	NETDEV_CMD_DEV_ADD_NTF,
 	NETDEV_CMD_DEV_DEL_NTF,
 	NETDEV_CMD_DEV_CHANGE_NTF,
+	NETDEV_CMD_PAGE_POOL_GET,
 
 	__NETDEV_CMD_MAX,
 	NETDEV_CMD_MAX = (__NETDEV_CMD_MAX - 1)
net/core/netdev-genl-gen.c +17 -0
index ea9231378aa6..c74fe69dd241 100644
@@ -15,6 +15,11 @@ static const struct nla_policy netdev_dev_get_nl_policy[NETDEV_A_DEV_IFINDEX + 1
 	[NETDEV_A_DEV_IFINDEX] = NLA_POLICY_MIN(NLA_U32, 1),
 };
 
+/* NETDEV_CMD_PAGE_POOL_GET - do */
+static const struct nla_policy netdev_page_pool_get_nl_policy[NETDEV_A_PAGE_POOL_ID + 1] = {
+	[NETDEV_A_PAGE_POOL_ID] = NLA_POLICY_MIN(NLA_U32, 1),
+};
+
 /* Ops table for netdev */
 static const struct genl_split_ops netdev_nl_ops[] = {
 	{
@@ -29,6 +34,18 @@ static const struct genl_split_ops netdev_nl_ops[] = {
 		.dumpit	= netdev_nl_dev_get_dumpit,
 		.flags	= GENL_CMD_CAP_DUMP,
 	},
+	{
+		.cmd		= NETDEV_CMD_PAGE_POOL_GET,
+		.doit		= netdev_nl_page_pool_get_doit,
+		.policy		= netdev_page_pool_get_nl_policy,
+		.maxattr	= NETDEV_A_PAGE_POOL_ID,
+		.flags		= GENL_CMD_CAP_DO,
+	},
+	{
+		.cmd	= NETDEV_CMD_PAGE_POOL_GET,
+		.dumpit	= netdev_nl_page_pool_get_dumpit,
+		.flags	= GENL_CMD_CAP_DUMP,
+	},
 };
 
 static const struct genl_multicast_group netdev_nl_mcgrps[] = {
net/core/netdev-genl-gen.h +3 -0
index 7b370c073e7d..a011d12abff4 100644
@@ -13,6 +13,9 @@
 
 int netdev_nl_dev_get_doit(struct sk_buff *skb, struct genl_info *info);
 int netdev_nl_dev_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb);
+int netdev_nl_page_pool_get_doit(struct sk_buff *skb, struct genl_info *info);
+int netdev_nl_page_pool_get_dumpit(struct sk_buff *skb,
+				   struct netlink_callback *cb);
 
 enum {
 	NETDEV_NLGRP_MGMT,
net/core/page_pool_user.c +127 -0

141 lines not on this page · raw message

Signature · 1 line
2.41.0
author maintainer
#23 lore raw

RFC 10/13 net: page_pool: add netlink notifications for state changes

In reply to Jakub Kicinski

Generate netlink notifications about page pool state changes.

---
 Documentation/netlink/specs/netdev.yaml | 17 ++++++++++++
 include/uapi/linux/netdev.h             |  4 +++
 net/core/netdev-genl-gen.c              |  1 +
 net/core/netdev-genl-gen.h              |  1 +
 net/core/page_pool_user.c               | 36 +++++++++++++++++++++++++
 5 files changed, 59 insertions(+)
Documentation/netlink/specs/netdev.yaml +17 -0
index ded77e61df7b..47ae2a45daea 100644
@@ -142,8 +142,25 @@ name: netdev
             - napi-id
       dump:
         reply: *pp-reply
+    -
+      name: page-pool-add-ntf
+      doc: Notification about page pool appearing.
+      notify: page-pool-get
+      mcgrp: page-pool
+    -
+      name: page-pool-del-ntf
+      doc: Notification about page pool disappearing.
+      notify: page-pool-get
+      mcgrp: page-pool
+    -
+      name: page-pool-change-ntf
+      doc: Notification about page pool configuration being changed.
+      notify: page-pool-get
+      mcgrp: page-pool
 
 mcast-groups:
   list:
     -
       name: mgmt
+    -
+      name: page-pool
include/uapi/linux/netdev.h +4 -0
index 98999e0650a3..3c9818c1962a 100644
@@ -64,11 +64,15 @@ enum {
 	NETDEV_CMD_DEV_DEL_NTF,
 	NETDEV_CMD_DEV_CHANGE_NTF,
 	NETDEV_CMD_PAGE_POOL_GET,
+	NETDEV_CMD_PAGE_POOL_ADD_NTF,
+	NETDEV_CMD_PAGE_POOL_DEL_NTF,
+	NETDEV_CMD_PAGE_POOL_CHANGE_NTF,
 
 	__NETDEV_CMD_MAX,
 	NETDEV_CMD_MAX = (__NETDEV_CMD_MAX - 1)
 };
 
 #define NETDEV_MCGRP_MGMT	"mgmt"
+#define NETDEV_MCGRP_PAGE_POOL	"page-pool"
 
 #endif /* _UAPI_LINUX_NETDEV_H */
net/core/netdev-genl-gen.c +1 -0
index c74fe69dd241..23d5bd111ddd 100644
@@ -50,6 +50,7 @@ static const struct genl_split_ops netdev_nl_ops[] = {
 
 static const struct genl_multicast_group netdev_nl_mcgrps[] = {
 	[NETDEV_NLGRP_MGMT] = { "mgmt", },
+	[NETDEV_NLGRP_PAGE_POOL] = { "page-pool", },
 };
 
 struct genl_family netdev_nl_family __ro_after_init = {
net/core/netdev-genl-gen.h +1 -0
index a011d12abff4..738097847100 100644
@@ -19,6 +19,7 @@ int netdev_nl_page_pool_get_dumpit(struct sk_buff *skb,
 
 enum {
 	NETDEV_NLGRP_MGMT,
+	NETDEV_NLGRP_PAGE_POOL,
 };
 
 extern struct genl_family netdev_nl_family;
net/core/page_pool_user.c +36 -0
index 3bc434f7825f..ba2f27f15495 100644
@@ -135,6 +135,37 @@ page_pool_nl_fill(struct sk_buff *rsp, const struct page_pool *pool,
 	return -EMSGSIZE;
 }
 
+static void netdev_nl_page_pool_event(const struct page_pool *pool, u32 cmd)
+{
+	struct genl_info info;
+	struct sk_buff *ntf;
+	struct net *net;
+
+	lockdep_assert_held(&page_pools_lock);
+
+	/* 'invisible' page pools don't matter */
+	if (hlist_unhashed(&pool->user.list))
+		return;
+	net = dev_net(pool->slow.netdev);
+
+	if (!genl_has_listeners(&netdev_nl_family, net, NETDEV_NLGRP_PAGE_POOL))
+		return;
+
+	genl_info_init_ntf(&info, &netdev_nl_family, cmd);
+
+	ntf = genlmsg_new(GENLMSG_DEFAULT_SIZE, GFP_KERNEL);
+	if (!ntf)
+		return;
+
+	if (page_pool_nl_fill(ntf, pool, &info)) {
+		nlmsg_free(ntf);
+		return;
+	}
+
+	genlmsg_multicast_netns(&netdev_nl_family, net, ntf,
+				0, NETDEV_NLGRP_PAGE_POOL, GFP_KERNEL);
+}
+
 int netdev_nl_page_pool_get_doit(struct sk_buff *skb, struct genl_info *info)
 {
 	u32 id;
@@ -168,6 +199,8 @@ int page_pool_list(struct page_pool *pool)
 		hlist_add_head(&pool->user.list,
 			       &pool->slow.netdev->page_pools);
 		pool->user.napi_id = pool->p.napi ? pool->p.napi->napi_id : 0;
+
+		netdev_nl_page_pool_event(pool, NETDEV_CMD_PAGE_POOL_ADD_NTF);
 	}
 
 	mutex_unlock(&page_pools_lock);
@@ -181,6 +214,7 @@ int page_pool_list(struct page_pool *pool)
 void page_pool_unlist(struct page_pool *pool)
 {
 	mutex_lock(&page_pools_lock);
+	netdev_nl_page_pool_event(pool, NETDEV_CMD_PAGE_POOL_DEL_NTF);
 	xa_erase(&page_pools, pool->user.id);
 	hlist_del(&pool->user.list);
 	mutex_unlock(&page_pools_lock);
@@ -212,6 +246,8 @@ static void page_pool_unreg_netdev(struct net_device *netdev)
 	mutex_lock(&page_pools_lock);
 	hlist_for_each_entry(pool, &netdev->page_pools, user.list) {
 		pool->slow.netdev = lo;
+		netdev_nl_page_pool_event(pool,
+					  NETDEV_CMD_PAGE_POOL_CHANGE_NTF);
 		last = pool;
 	}
 
Signature · 1 line
2.41.0
author maintainer
#24 lore raw

RFC 11/13 net: page_pool: report when page pool was destroyed

In reply to Jakub Kicinski

Report when page pool was destroyed and number of inflight references. This allows user to proactively check for dead page pools without waiting to see if errors messages will be printed in dmesg.

Only provide inflight for pools which were already "destroyed". inflight information could also be interesting for "live" pools but we don't want to have to deal with the potential negative values due to races.

Example output for a fake leaked page pool using some hacks in netdevsim (one "live" pool, and one "leaked" on the same dev):

$ ./cli.py --no-schema --spec netlink/specs/netdev.yaml \
           --dump page-pool-get
[{'id': 2, 'ifindex': 3},
 {'id': 1, 'ifindex': 3, 'destroyed': 133, 'inflight': 1}]
---
 Documentation/netlink/specs/netdev.yaml | 16 ++++++++++++++++
 include/net/page_pool/types.h           |  1 +
 include/uapi/linux/netdev.h             |  2 ++
 net/core/page_pool.c                    |  3 ++-
 net/core/page_pool_priv.h               |  3 +++
 net/core/page_pool_user.c               | 16 ++++++++++++++++
 6 files changed, 40 insertions(+), 1 deletion(-)
Documentation/netlink/specs/netdev.yaml +16 -0
index 47ae2a45daea..1fd3b4235251 100644
@@ -92,6 +92,20 @@ name: netdev
         name: napi-id
         doc: Id of NAPI using this Page Pool instance.
         type: u32
+      -
+        name: destroyed
+        type: u64
+        doc: |
+          Seconds in CLOCK_BOOTTIME of when Page Pool was destroyed.
+          Page Pools wait for all the memory allocated from them to be freed
+          before truly disappearing.
+          Absent if Page Pool hasn't been destroyed.
+      -
+        name: inflight
+        type: u32
+        doc: |
+          Number of outstanding references to this page pool (allocated
+          but yet to be freed pages).
 
 operations:
   list:
@@ -140,6 +154,8 @@ name: netdev
             - id
             - ifindex
             - napi-id
+            - destroyed
+            - inflight
       dump:
         reply: *pp-reply
     -
include/net/page_pool/types.h +1 -0
index 3017557e0c59..490c7419a474 100644
@@ -195,6 +195,7 @@ struct page_pool {
 	/* User-facing fields, protected by page_pools_lock */
 	struct {
 		struct hlist_node list;
+		u64 destroyed;
 		u32 napi_id;
 		u32 id;
 	} user;
include/uapi/linux/netdev.h +2 -0
index 3c9818c1962a..aac840a3849b 100644
@@ -53,6 +53,8 @@ enum {
 	NETDEV_A_PAGE_POOL_PAD,
 	NETDEV_A_PAGE_POOL_IFINDEX,
 	NETDEV_A_PAGE_POOL_NAPI_ID,
+	NETDEV_A_PAGE_POOL_DESTROYED,
+	NETDEV_A_PAGE_POOL_INFLIGHT,
 
 	__NETDEV_A_PAGE_POOL_MAX,
 	NETDEV_A_PAGE_POOL_MAX = (__NETDEV_A_PAGE_POOL_MAX - 1)
net/core/page_pool.c +2 -1
index de199c356043..733ca2198d94 100644
@@ -513,7 +513,7 @@ EXPORT_SYMBOL(page_pool_alloc_pages);
  */
 #define _distance(a, b)	(s32)((a) - (b))
 
-static s32 page_pool_inflight(struct page_pool *pool)
+s32 page_pool_inflight(const struct page_pool *pool)
 {
 	u32 release_cnt = atomic_read(&pool->pages_state_release_cnt);
 	u32 hold_cnt = READ_ONCE(pool->pages_state_hold_cnt);
@@ -933,6 +933,7 @@ void page_pool_destroy(struct page_pool *pool)
 	if (!page_pool_release(pool))
 		return;
 
+	page_pool_destroyed(pool);
 	pool->defer_start = jiffies;
 	pool->defer_warn  = jiffies + DEFER_WARN_INTERVAL;
 
net/core/page_pool_priv.h +3 -0
index 6c4e4aeed02a..892e0cddc400 100644
@@ -3,7 +3,10 @@
 #ifndef __PAGE_POOL_PRIV_H
 #define __PAGE_POOL_PRIV_H
 
+s32 page_pool_inflight(const struct page_pool *pool);
+
 int page_pool_list(struct page_pool *pool);
+void page_pool_destroyed(struct page_pool *pool);
 void page_pool_unlist(struct page_pool *pool);
 
 #endif
net/core/page_pool_user.c +16 -0
index ba2f27f15495..a74d6f18caac 100644
@@ -126,6 +126,14 @@ page_pool_nl_fill(struct sk_buff *rsp, const struct page_pool *pool,
 	if (pool->user.napi_id &&
 	    nla_put_u32(rsp, NETDEV_A_PAGE_POOL_NAPI_ID, pool->user.napi_id))
 		goto err_cancel;
+	if (pool->user.destroyed) {
+		if (nla_put_u64_64bit(rsp, NETDEV_A_PAGE_POOL_DESTROYED,
+				      pool->user.destroyed,
+				      NETDEV_A_PAGE_POOL_PAD) ||
+		    nla_put_u32(rsp, NETDEV_A_PAGE_POOL_INFLIGHT,
+				page_pool_inflight(pool)))
+			goto err_cancel;
+	}
 
 	genlmsg_end(rsp, hdr);
 
@@ -211,6 +219,14 @@ int page_pool_list(struct page_pool *pool)
 	return err;
 }
 
+void page_pool_destroyed(struct page_pool *pool)
+{
+	mutex_lock(&page_pools_lock);
+	pool->user.destroyed = ktime_get_boottime_seconds();
+	netdev_nl_page_pool_event(pool, NETDEV_CMD_PAGE_POOL_CHANGE_NTF);
+	mutex_unlock(&page_pools_lock);
+}
+
 void page_pool_unlist(struct page_pool *pool)
 {
 	mutex_lock(&page_pools_lock);
Signature · 1 line
2.41.0
author maintainer
#25 lore raw

RFC 12/13 net: page_pool: expose page pool stats via netlink

In reply to Jakub Kicinski

Dump the stats into netlink. More clever approaches like dumping the stats per-CPU for each CPU individually to see where the packets get consumed can be implemented in the future.

A trimmed example from a real (but recently booted system):

$ ./cli.py --no-schema --spec netlink/specs/netdev.yaml \
           --dump page-pool-stats-get
[{'info': {'id': 19, 'ifindex': 2},
  'alloc-empty': 48,
  'alloc-fast': 3024,
  'alloc-refill': 0,
  'alloc-slow': 48,
  'alloc-slow-high-order': 0,
  'alloc-waive': 0,
  'recycle-cache-full': 0,
  'recycle-cached': 0,
  'recycle-released-refcnt': 0,
  'recycle-ring': 0,
  'recycle-ring-full': 0},
 {'info': {'id': 18, 'ifindex': 2},
  'alloc-empty': 66,
  'alloc-fast': 11811,
  'alloc-refill': 35,
  'alloc-slow': 66,
  'alloc-slow-high-order': 0,
  'alloc-waive': 0,
  'recycle-cache-full': 1145,
  'recycle-cached': 6541,
  'recycle-released-refcnt': 0,
  'recycle-ring': 1275,
  'recycle-ring-full': 0},
 {'info': {'id': 17, 'ifindex': 2},
  'alloc-empty': 73,
  'alloc-fast': 62099,
  'alloc-refill': 413,
...
---
 Documentation/netlink/specs/netdev.yaml |  84 +++++++++++++++++++
 Documentation/networking/page_pool.rst  |   5 +-
 include/net/page_pool/helpers.h         |   8 +-
 include/uapi/linux/netdev.h             |  20 +++++
 net/core/netdev-genl-gen.c              |  23 +++++
 net/core/netdev-genl-gen.h              |   7 ++
 net/core/page_pool.c                    |   2 +-
 net/core/page_pool_user.c               | 106 ++++++++++++++++++++++++
 8 files changed, 246 insertions(+), 9 deletions(-)
Documentation/netlink/specs/netdev.yaml +84 -0
index 1fd3b4235251..412e500d5eaa 100644
@@ -106,6 +106,66 @@ name: netdev
         doc: |
           Number of outstanding references to this page pool (allocated
           but yet to be freed pages).
+  -
+    name: page-pool-info
+    subset-of: page-pool
+    attributes:
+      -
+        name: id
+        type: u32
+        checks:
+          min: 1
+      -
+        name: ifindex
+        type: u32
+  -
+    name: page-pool-stats
+    doc: |
+      Page pool statistics, see docs for struct page_pool_stats
+      for information about individual statistics.
+    attributes:
+      -
+        name: info
+        doc: Page pool identifying information.
+        type: nest
+        nested-attributes: page-pool-info
+      -
+        name: pad
+        type: pad
+      -
+        name: alloc-fast
+        type: u64
+        value: 8 # reserve some attr ids in case we need more metadata later
+      -
+        name: alloc-slow
+        type: u64
+      -
+        name: alloc-slow-high-order
+        type: u64
+      -
+        name: alloc-empty
+        type: u64
+      -
+        name: alloc-refill
+        type: u64
+      -
+        name: alloc-waive
+        type: u64
+      -
+        name: recycle-cached
+        type: u64
+      -
+        name: recycle-cache-full
+        type: u64
+      -
+        name: recycle-ring
+        type: u64
+      -
+        name: recycle-ring-full
+        type: u64
+      -
+        name: recycle-released-refcnt
+        type: u64
 
 operations:
   list:
@@ -173,6 +233,30 @@ name: netdev
       doc: Notification about page pool configuration being changed.
       notify: page-pool-get
       mcgrp: page-pool
+    -
+      name: page-pool-stats-get
+      doc: Get page pool statistics.
+      attribute-set: page-pool-stats
+      do:
+        request:
+          attributes:
+            - info
+        reply: &pp-stats-reply
+          attributes:
+            - info
+            - alloc-fast
+            - alloc-slow
+            - alloc-slow-high-order
+            - alloc-empty
+            - alloc-refill
+            - alloc-waive
+            - recycle-cached
+            - recycle-cache-full
+            - recycle-ring
+            - recycle-ring-full
+            - recycle-released-refcnt
+      dump:
+        reply: *pp-stats-reply
 
 mcast-groups:
   list:
Documentation/networking/page_pool.rst +3 -2
index 215ebc92752c..d5566b7c5e7c 100644
@@ -105,8 +105,9 @@ page_pool_get_stats() and structures described below are available.
 It takes a  pointer to a ``struct page_pool`` and a pointer to a struct
 page_pool_stats allocated by the caller.
 
-The API will fill in the provided struct page_pool_stats with
-statistics about the page_pool.
+Older drivers expose page pool statistics via ethtool or debugfs.
+The same statistics are accessible via the netlink netdev family
+in a driver-independent fashion.
 
 .. kernel-doc:: include/net/page_pool/types.h
    :identifiers: struct page_pool_recycle_stats
include/net/page_pool/helpers.h +2 -6
index 94231533a369..b004eef7071a 100644
@@ -32,16 +32,12 @@
 #include <net/page_pool/types.h>
 
 #ifdef CONFIG_PAGE_POOL_STATS
+/* Deprecated driver-facing API, use netlink instead */
 int page_pool_ethtool_stats_get_count(void);
 u8 *page_pool_ethtool_stats_get_strings(u8 *data);
 u64 *page_pool_ethtool_stats_get(u64 *data, void *stats);
 
-/*
- * Drivers that wish to harvest page pool stats and report them to users
- * (perhaps via ethtool, debugfs, or another mechanism) can allocate a
- * struct page_pool_stats call page_pool_get_stats to get stats for the specified pool.
- */
-bool page_pool_get_stats(struct page_pool *pool,
+bool page_pool_get_stats(const struct page_pool *pool,
 			 struct page_pool_stats *stats);
 #else
 static inline int page_pool_ethtool_stats_get_count(void)
include/uapi/linux/netdev.h +20 -0
index aac840a3849b..25754237eb2f 100644
@@ -60,6 +60,25 @@ enum {
 	NETDEV_A_PAGE_POOL_MAX = (__NETDEV_A_PAGE_POOL_MAX - 1)
 };
 
+enum {
+	NETDEV_A_PAGE_POOL_STATS_INFO = 1,
+	NETDEV_A_PAGE_POOL_STATS_PAD,
+	NETDEV_A_PAGE_POOL_STATS_ALLOC_FAST = 8,
+	NETDEV_A_PAGE_POOL_STATS_ALLOC_SLOW,
+	NETDEV_A_PAGE_POOL_STATS_ALLOC_SLOW_HIGH_ORDER,
+	NETDEV_A_PAGE_POOL_STATS_ALLOC_EMPTY,
+	NETDEV_A_PAGE_POOL_STATS_ALLOC_REFILL,
+	NETDEV_A_PAGE_POOL_STATS_ALLOC_WAIVE,
+	NETDEV_A_PAGE_POOL_STATS_RECYCLE_CACHED,
+	NETDEV_A_PAGE_POOL_STATS_RECYCLE_CACHE_FULL,
+	NETDEV_A_PAGE_POOL_STATS_RECYCLE_RING,
+	NETDEV_A_PAGE_POOL_STATS_RECYCLE_RING_FULL,
+	NETDEV_A_PAGE_POOL_STATS_RECYCLE_RELEASED_REFCNT,
+
+	__NETDEV_A_PAGE_POOL_STATS_MAX,
+	NETDEV_A_PAGE_POOL_STATS_MAX = (__NETDEV_A_PAGE_POOL_STATS_MAX - 1)
+};
+
 enum {
 	NETDEV_CMD_DEV_GET = 1,
 	NETDEV_CMD_DEV_ADD_NTF,
@@ -69,6 +88,7 @@ enum {
 	NETDEV_CMD_PAGE_POOL_ADD_NTF,
 	NETDEV_CMD_PAGE_POOL_DEL_NTF,
 	NETDEV_CMD_PAGE_POOL_CHANGE_NTF,
+	NETDEV_CMD_PAGE_POOL_STATS_GET,
 
 	__NETDEV_CMD_MAX,
 	NETDEV_CMD_MAX = (__NETDEV_CMD_MAX - 1)
net/core/netdev-genl-gen.c +23 -0
index 23d5bd111ddd..5dab064988c1 100644
@@ -10,6 +10,12 @@
 
 #include <uapi/linux/netdev.h>
 
+/* Common nested types */
+const struct nla_policy netdev_page_pool_info_nl_policy[NETDEV_A_PAGE_POOL_IFINDEX + 1] = {
+	[NETDEV_A_PAGE_POOL_ID] = NLA_POLICY_MIN(NLA_U32, 1),
+	[NETDEV_A_PAGE_POOL_IFINDEX] = { .type = NLA_U32, },
+};
+
 /* NETDEV_CMD_DEV_GET - do */
 static const struct nla_policy netdev_dev_get_nl_policy[NETDEV_A_DEV_IFINDEX + 1] = {
 	[NETDEV_A_DEV_IFINDEX] = NLA_POLICY_MIN(NLA_U32, 1),
@@ -20,6 +26,11 @@ static const struct nla_policy netdev_page_pool_get_nl_policy[NETDEV_A_PAGE_POOL
 	[NETDEV_A_PAGE_POOL_ID] = NLA_POLICY_MIN(NLA_U32, 1),
 };
 
+/* NETDEV_CMD_PAGE_POOL_STATS_GET - do */
+static const struct nla_policy netdev_page_pool_stats_get_nl_policy[NETDEV_A_PAGE_POOL_STATS_INFO + 1] = {
+	[NETDEV_A_PAGE_POOL_STATS_INFO] = NLA_POLICY_NESTED(netdev_page_pool_info_nl_policy),
+};
+
 /* Ops table for netdev */
 static const struct genl_split_ops netdev_nl_ops[] = {
 	{
@@ -46,6 +57,18 @@ static const struct genl_split_ops netdev_nl_ops[] = {
 		.dumpit	= netdev_nl_page_pool_get_dumpit,
 		.flags	= GENL_CMD_CAP_DUMP,
 	},
+	{
+		.cmd		= NETDEV_CMD_PAGE_POOL_STATS_GET,
+		.doit		= netdev_nl_page_pool_stats_get_doit,
+		.policy		= netdev_page_pool_stats_get_nl_policy,
+		.maxattr	= NETDEV_A_PAGE_POOL_STATS_INFO,
+		.flags		= GENL_CMD_CAP_DO,
+	},
+	{
+		.cmd	= NETDEV_CMD_PAGE_POOL_STATS_GET,
+		.dumpit	= netdev_nl_page_pool_stats_get_dumpit,
+		.flags	= GENL_CMD_CAP_DUMP,
+	},
 };
 
 static const struct genl_multicast_group netdev_nl_mcgrps[] = {
net/core/netdev-genl-gen.h +7 -0
index 738097847100..649e4b46eccf 100644
@@ -11,11 +11,18 @@
 
 #include <uapi/linux/netdev.h>
 
+/* Common nested types */
+extern const struct nla_policy netdev_page_pool_info_nl_policy[NETDEV_A_PAGE_POOL_IFINDEX + 1];
+
 int netdev_nl_dev_get_doit(struct sk_buff *skb, struct genl_info *info);
 int netdev_nl_dev_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb);
 int netdev_nl_page_pool_get_doit(struct sk_buff *skb, struct genl_info *info);
 int netdev_nl_page_pool_get_dumpit(struct sk_buff *skb,
 				   struct netlink_callback *cb);
+int netdev_nl_page_pool_stats_get_doit(struct sk_buff *skb,
+				       struct genl_info *info);
+int netdev_nl_page_pool_stats_get_dumpit(struct sk_buff *skb,
+					 struct netlink_callback *cb);
 
 enum {
 	NETDEV_NLGRP_MGMT,
net/core/page_pool.c +1 -1
index 733ca2198d94..b7dac08709ea 100644
@@ -71,7 +71,7 @@ static const char pp_stats[][ETH_GSTRING_LEN] = {
  * is passed to this API which is filled in. The caller can then report
  * those stats to the user (perhaps via ethtool, debugfs, etc.).
  */
-bool page_pool_get_stats(struct page_pool *pool,
+bool page_pool_get_stats(const struct page_pool *pool,
 			 struct page_pool_stats *stats)
 {
 	int cpu = 0;
net/core/page_pool_user.c +106 -0

118 lines not on this page · raw message

Signature · 1 line
2.41.0
author maintainer
#26 lore raw

RFC 13/13 tools: netdev: regen after page pool changes

In reply to Jakub Kicinski

Regenerate the code in tools/ after page pool addition to the netdev netlink family.

---
 tools/include/uapi/linux/netdev.h     |  37 +++
 tools/net/ynl/generated/netdev-user.c | 415 ++++++++++++++++++++++++++
 tools/net/ynl/generated/netdev-user.h | 169 +++++++++++
 3 files changed, 621 insertions(+)
tools/include/uapi/linux/netdev.h +37 -0
index c1634b95c223..25754237eb2f 100644
@@ -48,16 +48,53 @@ enum {
 	NETDEV_A_DEV_MAX = (__NETDEV_A_DEV_MAX - 1)
 };
 
+enum {
+	NETDEV_A_PAGE_POOL_ID = 1,
+	NETDEV_A_PAGE_POOL_PAD,
+	NETDEV_A_PAGE_POOL_IFINDEX,
+	NETDEV_A_PAGE_POOL_NAPI_ID,
+	NETDEV_A_PAGE_POOL_DESTROYED,
+	NETDEV_A_PAGE_POOL_INFLIGHT,
+
+	__NETDEV_A_PAGE_POOL_MAX,
+	NETDEV_A_PAGE_POOL_MAX = (__NETDEV_A_PAGE_POOL_MAX - 1)
+};
+
+enum {
+	NETDEV_A_PAGE_POOL_STATS_INFO = 1,
+	NETDEV_A_PAGE_POOL_STATS_PAD,
+	NETDEV_A_PAGE_POOL_STATS_ALLOC_FAST = 8,
+	NETDEV_A_PAGE_POOL_STATS_ALLOC_SLOW,
+	NETDEV_A_PAGE_POOL_STATS_ALLOC_SLOW_HIGH_ORDER,
+	NETDEV_A_PAGE_POOL_STATS_ALLOC_EMPTY,
+	NETDEV_A_PAGE_POOL_STATS_ALLOC_REFILL,
+	NETDEV_A_PAGE_POOL_STATS_ALLOC_WAIVE,
+	NETDEV_A_PAGE_POOL_STATS_RECYCLE_CACHED,
+	NETDEV_A_PAGE_POOL_STATS_RECYCLE_CACHE_FULL,
+	NETDEV_A_PAGE_POOL_STATS_RECYCLE_RING,
+	NETDEV_A_PAGE_POOL_STATS_RECYCLE_RING_FULL,
+	NETDEV_A_PAGE_POOL_STATS_RECYCLE_RELEASED_REFCNT,
+
+	__NETDEV_A_PAGE_POOL_STATS_MAX,
+	NETDEV_A_PAGE_POOL_STATS_MAX = (__NETDEV_A_PAGE_POOL_STATS_MAX - 1)
+};
+
 enum {
 	NETDEV_CMD_DEV_GET = 1,
 	NETDEV_CMD_DEV_ADD_NTF,
 	NETDEV_CMD_DEV_DEL_NTF,
 	NETDEV_CMD_DEV_CHANGE_NTF,
+	NETDEV_CMD_PAGE_POOL_GET,
+	NETDEV_CMD_PAGE_POOL_ADD_NTF,
+	NETDEV_CMD_PAGE_POOL_DEL_NTF,
+	NETDEV_CMD_PAGE_POOL_CHANGE_NTF,
+	NETDEV_CMD_PAGE_POOL_STATS_GET,
 
 	__NETDEV_CMD_MAX,
 	NETDEV_CMD_MAX = (__NETDEV_CMD_MAX - 1)
 };
 
 #define NETDEV_MCGRP_MGMT	"mgmt"
+#define NETDEV_MCGRP_PAGE_POOL	"page-pool"
 
 #endif /* _UAPI_LINUX_NETDEV_H */
tools/net/ynl/generated/netdev-user.c +415 -0

446 lines not on this page · raw message

tools/net/ynl/generated/netdev-user.h +169 -0

179 lines not on this page · raw message

Signature · 1 line
2.41.0
#27 lore raw
In reply to Jakub Kicinski
Quoted 9 lines · Jakub Kicinski

As the page pool functionality grows we will be increasingly constrained by the lack of any uAPI. This series is a first step towards such a uAPI, it creates a way for users to query information about page pools and associated statistics.

I'm purposefully exposing only the information which I'd find useful when monitoring production workloads.

Quoted 10 lines

For the SET part (which to be clear isn't addressed by this series at all) I think we'll need to turn to something more along the lines of "netdev-level policy". Instead of configuring page pools, which are somewhat ephemeral, and hard to change at runtime, we should set the "expected page pool parameters" at the netdev level, and have the driver consult those when instantiating pools. My ramblings from yesterday about Queue API may make this more or less clear...
https://lore.kernel.org/all/[email protected]/

The patches themselves look good to me, and I'll provide Reviewed-by for the ones I feel I understand well enough to review, but I'm a bit unsure about exposing specifically page_pool stats to the user. Isn't it better to expose rx-queue stats (slightly more generic) to the user? In my mind, the advantages:

- we could implement better SET apis that allocate, delete, or reconfigure rx-queues. APIs that allocate or delete page-pool make less sense semantically maybe? The page-pool doesn't decide to instantiate itself.

- The api can be extended for non-page-pool stats (per rx-queue dropped packets maybe, or something like that).

- The api maybe can be extended to non-page-pool drivers. The driver may be able to implement their own function to provide equivalent stats (although this one may not be that important).

- rx-queue GET API fits in nicely with what you described yesterday [1]. At the moment I'm a bit unsure because the SET api you described yesterday sounded per-rx-queue to me. But the GET api here is per-page-pool based. Maybe the distinction doesn't matter? Maybe you're thinking they're unrelated APIs?

[1] https://lore.kernel.org/all/[email protected]/

Quoted 41 lines

Jakub Kicinski (13):
net: page_pool: split the page_pool_params into fast and slow
net: page_pool: avoid touching slow on the fastpath
net: page_pool: factor out uninit
net: page_pool: id the page pools
net: page_pool: record pools per netdev
net: page_pool: stash the NAPI ID for easier access
eth: link netdev to pp
net: page_pool: add nlspec for basic access to page pools
net: page_pool: implement GET in the netlink API
net: page_pool: add netlink notifications for state changes
net: page_pool: report when page pool was destroyed
net: page_pool: expose page pool stats via netlink
tools: netdev: regen after page pool changes

 Documentation/netlink/specs/netdev.yaml       | 158 +++++++
 Documentation/networking/page_pool.rst        |   5 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt.c     |   1 +
 .../net/ethernet/mellanox/mlx5/core/en_main.c |   1 +
 drivers/net/ethernet/microsoft/mana/mana_en.c |   1 +
 include/linux/list.h                          |  20 +
 include/linux/netdevice.h                     |   4 +
 include/net/page_pool/helpers.h               |   8 +-
 include/net/page_pool/types.h                 |  43 +-
 include/uapi/linux/netdev.h                   |  37 ++
 net/core/Makefile                             |   2 +-
 net/core/netdev-genl-gen.c                    |  41 ++
 net/core/netdev-genl-gen.h                    |  11 +
 net/core/page_pool.c                          |  56 ++-
 net/core/page_pool_priv.h                     |  12 +
 net/core/page_pool_user.c                     | 409 +++++++++++++++++
 tools/include/uapi/linux/netdev.h             |  37 ++
 tools/net/ynl/generated/netdev-user.c         | 415 ++++++++++++++++++
 tools/net/ynl/generated/netdev-user.h         | 169 +++++++
 19 files changed, 1391 insertions(+), 39 deletions(-)
 create mode 100644 net/core/page_pool_priv.h
 create mode 100644 net/core/page_pool_user.c

--
2.41.0

Signature · 2 lines
Thanks,
Mina
author maintainer
#28 lore raw
Quoted 5 lines · On Thu, 17 Aug 2023 14:21:26 -0700 Mina Almasry wrote:

- rx-queue GET API fits in nicely with what you described yesterday [1]. At the moment I'm a bit unsure because the SET api you described yesterday sounded per-rx-queue to me. But the GET api here is per-page-pool based. Maybe the distinction doesn't matter? Maybe you're thinking they're unrelated APIs?

Right, they aren't unrelated but I do somehow prefer the more flexible model where each object type has its own API and the objects can refer to each other.

I think there's too much of a risk that the mapping between page pools to queues will become n:m and then having them tied together will be a problem.