Skip to contentSkip to conversation

net: page_pool: remove page_pool_release_page()

PATCH Applied Jakub Kicinski 5 participants 13 messages Jul 20, 2023 – Jul 24, 2023

Subsystems (6)
lore
Share & tools
lore

Jakub Kicinski

author maintainer
#0 lore raw

cover net: page_pool: remove page_pool_release_page()

page_pool_return_page() is a historic artefact from before recycling of pages attached to skbs was supported. Theoretical uses for it may be thought up but in practice all existing users can be converted to use skb_mark_for_recycle() instead.

This code was previously posted as part of the memory provider RFC.

Jakub Kicinski (4):
eth: tsnep: let page recycling happen with skbs
eth: stmmac: let page recycling happen with skbs
net: page_pool: hide page_pool_release_page()
net: page_pool: merge page_pool_release_page() with
page_pool_return_page()

 Documentation/networking/page_pool.rst            | 11 ++++-------
 drivers/net/ethernet/engleder/tsnep_main.c        |  2 +-
 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c |  4 ++--
 include/net/page_pool.h                           | 10 ++--------
 net/core/page_pool.c                              | 13 ++-----------
 5 files changed, 11 insertions(+), 29 deletions(-)
Signature · 1 line
2.41.0
author maintainer
#1 lore raw

PATCH 1/4 eth: tsnep: let page recycling happen with skbs

tsnep builds an skb with napi_build_skb() and then calls page_pool_release_page() for the page in which that skb's head sits. Use recycling instead, recycling of heads works just fine.

---
CC: [email protected]

---
 drivers/net/ethernet/engleder/tsnep_main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/engleder/tsnep_main.c +1 -1
index 84751bb303a6..079f9f6ae21a 100644
@@ -1333,7 +1333,7 @@ static void tsnep_rx_page(struct tsnep_rx *rx, struct napi_struct *napi,
 
 	skb = tsnep_build_skb(rx, page, length);
 	if (skb) {
-		page_pool_release_page(rx->page_pool, page);
+		skb_mark_for_recycle(skb);
 
 		rx->packets++;
 		rx->bytes += length;
Signature · 1 line
2.41.0
author maintainer
#3 lore raw

PATCH 2/4 eth: stmmac: let page recycling happen with skbs

In reply to Jakub Kicinski

stmmac removes pages from the page pool after attaching them to skbs. Use page recycling instead.

skb heads are always copied, and pages are always from page pool in this driver. We could as well mark all allocated skbs for recycling.

---
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]

---
 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +2 -2
index efe85b086abe..40b17f94d21b 100644
@@ -5416,7 +5416,7 @@ static int stmmac_rx(struct stmmac_priv *priv, int limit, u32 queue)
 					priv->dma_conf.dma_buf_sz);
 
 			/* Data payload appended into SKB */
-			page_pool_release_page(rx_q->page_pool, buf->page);
+			skb_mark_for_recycle(skb);
 			buf->page = NULL;
 		}
 
@@ -5428,7 +5428,7 @@ static int stmmac_rx(struct stmmac_priv *priv, int limit, u32 queue)
 					priv->dma_conf.dma_buf_sz);
 
 			/* Data payload appended into SKB */
-			page_pool_release_page(rx_q->page_pool, buf->sec_page);
+			skb_mark_for_recycle(skb);
 			buf->sec_page = NULL;
 		}
 
Signature · 1 line
2.41.0
author maintainer
#5 lore raw

PATCH 3/4 net: page_pool: hide page_pool_release_page()

In reply to Jakub Kicinski

There seems to be no user calling page_pool_release_page() for legit reasons, all the users simply haven't been converted to skb-based recycling, yet. Previous changes converted them. Update the docs, and unexport the function.

---
CC: [email protected]
CC: [email protected]

---
 Documentation/networking/page_pool.rst | 11 ++++-------
 include/net/page_pool.h                | 10 ++--------
 net/core/page_pool.c                   |  3 +--
 3 files changed, 7 insertions(+), 17 deletions(-)
Documentation/networking/page_pool.rst +4 -7
index 873efd97f822..0aa850cf4447 100644
@@ -13,9 +13,9 @@ replacing dev_alloc_pages().
 
 API keeps track of in-flight pages, in order to let API user know
 when it is safe to free a page_pool object.  Thus, API users
-must run page_pool_release_page() when a page is leaving the page_pool or
-call page_pool_put_page() where appropriate in order to maintain correct
-accounting.
+must call page_pool_put_page() to free the page, or attach
+the page to a page_pool-aware objects like skbs marked with
+skb_mark_for_recycle().
 
 API user must call page_pool_put_page() once on a page, as it
 will either recycle the page, or in case of refcnt > 1, it will
@@ -87,9 +87,6 @@ a page will cause no race conditions is enough.
   must guarantee safe context (e.g NAPI), since it will recycle the page
   directly into the pool fast cache.
 
-* page_pool_release_page(): Unmap the page (if mapped) and account for it on
-  in-flight counters.
-
 * page_pool_dev_alloc_pages(): Get a page from the page allocator or page_pool
   caches.
 
@@ -194,7 +191,7 @@ NAPI poller
             if XDP_DROP:
                 page_pool_recycle_direct(page_pool, page);
         } else (packet_is_skb) {
-            page_pool_release_page(page_pool, page);
+            skb_mark_for_recycle(skb);
             new_page = page_pool_dev_alloc_pages(page_pool);
         }
     }
include/net/page_pool.h +2 -8
index 126f9e294389..f1d5cc1fa13b 100644
@@ -18,9 +18,8 @@
  *
  * API keeps track of in-flight pages, in-order to let API user know
  * when it is safe to dealloactor page_pool object.  Thus, API users
- * must make sure to call page_pool_release_page() when a page is
- * "leaving" the page_pool.  Or call page_pool_put_page() where
- * appropiate.  For maintaining correct accounting.
+ * must call page_pool_put_page() where appropriate and only attach
+ * the page to a page_pool-aware objects, like skbs marked for recycling.
  *
  * API user must only call page_pool_put_page() once on a page, as it
  * will either recycle the page, or in case of elevated refcnt, it
@@ -251,7 +250,6 @@ void page_pool_unlink_napi(struct page_pool *pool);
 void page_pool_destroy(struct page_pool *pool);
 void page_pool_use_xdp_mem(struct page_pool *pool, void (*disconnect)(void *),
 			   struct xdp_mem_info *mem);
-void page_pool_release_page(struct page_pool *pool, struct page *page);
 void page_pool_put_page_bulk(struct page_pool *pool, void **data,
 			     int count);
 #else
@@ -268,10 +266,6 @@ static inline void page_pool_use_xdp_mem(struct page_pool *pool,
 					 struct xdp_mem_info *mem)
 {
 }
-static inline void page_pool_release_page(struct page_pool *pool,
-					  struct page *page)
-{
-}
 
 static inline void page_pool_put_page_bulk(struct page_pool *pool, void **data,
 					   int count)
net/core/page_pool.c +1 -2
index a3e12a61d456..2c7cf5f2bcb8 100644
@@ -492,7 +492,7 @@ static s32 page_pool_inflight(struct page_pool *pool)
  * a regular page (that will eventually be returned to the normal
  * page-allocator via put_page).
  */
-void page_pool_release_page(struct page_pool *pool, struct page *page)
+static void page_pool_release_page(struct page_pool *pool, struct page *page)
 {
 	dma_addr_t dma;
 	int count;
@@ -519,7 +519,6 @@ void page_pool_release_page(struct page_pool *pool, struct page *page)
 	count = atomic_inc_return_relaxed(&pool->pages_state_release_cnt);
 	trace_page_pool_state_release(pool, page, count);
 }
-EXPORT_SYMBOL(page_pool_release_page);
 
 /* Return a page to the page allocator, cleaning up our state */
 static void page_pool_return_page(struct page_pool *pool, struct page *page)
Signature · 1 line
2.41.0
author maintainer
#8 lore raw

PATCH 4/4 net: page_pool: merge page_pool_release_page() with page_pool_return_page()

In reply to Jakub Kicinski

Now that page_pool_release_page() is not exported we can merge it with page_pool_return_page(). I believe that the "Do not replace this with page_pool_return_page()" comment was there in case page_pool_return_page() was not inlined, to avoid two function calls.

---
CC: [email protected]
CC: [email protected]

---
 net/core/page_pool.c | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)
net/core/page_pool.c +2 -10
index 2c7cf5f2bcb8..7ca456bfab71 100644
@@ -492,7 +492,7 @@ static s32 page_pool_inflight(struct page_pool *pool)
  * a regular page (that will eventually be returned to the normal
  * page-allocator via put_page).
  */
-static void page_pool_release_page(struct page_pool *pool, struct page *page)
+static void page_pool_return_page(struct page_pool *pool, struct page *page)
 {
 	dma_addr_t dma;
 	int count;
@@ -518,12 +518,6 @@ static void page_pool_release_page(struct page_pool *pool, struct page *page)
 	 */
 	count = atomic_inc_return_relaxed(&pool->pages_state_release_cnt);
 	trace_page_pool_state_release(pool, page, count);
-}
-
-/* Return a page to the page allocator, cleaning up our state */
-static void page_pool_return_page(struct page_pool *pool, struct page *page)
-{
-	page_pool_release_page(pool, page);
 
 	put_page(page);
 	/* An optimization would be to call __free_pages(page, pool->p.order)
@@ -615,9 +609,7 @@ __page_pool_put_page(struct page_pool *pool, struct page *page,
 	 * will be invoking put_page.
 	 */
 	recycle_stat_inc(pool, released_refcnt);
-	/* Do not replace this with page_pool_return_page() */
-	page_pool_release_page(pool, page);
-	put_page(page);
+	page_pool_return_page(pool, page);
 
 	return NULL;
 }
Signature · 1 line
2.41.0
Reviewed-by on v1 1/4
#2 lore raw

Re: [PATCH net-next 1/4] eth: tsnep: let page recycling happen with skbs

In reply to Jakub Kicinski
Quoted 1 line · Jakub Kicinski

tsnep builds an skb with napi_build_skb() and then calls

nit: an -> a ?

Othewise, LGTM.

Quoted 25 lines

page_pool_release_page() for the page in which that skb's head sits. Use recycling instead, recycling of heads works just fine.

---
CC: [email protected]

---
 drivers/net/ethernet/engleder/tsnep_main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/engleder/tsnep_main.c +1 -1
index 84751bb303a6..079f9f6ae21a 100644
@@ -1333,7 +1333,7 @@ static void tsnep_rx_page(struct tsnep_rx *rx, struct napi_struct *napi,
 
 	skb = tsnep_build_skb(rx, page, length);
 	if (skb) {
-		page_pool_release_page(rx->page_pool, page);
+		skb_mark_for_recycle(skb);
 
 		rx->packets++;
 		rx->bytes += length;
Reviewed-by on v1 2/4
#4 lore raw

Re: [PATCH net-next 2/4] eth: stmmac: let page recycling happen with skbs

In reply to Jakub Kicinski
Quoted 6 lines · Jakub Kicinski

stmmac removes pages from the page pool after attaching them to skbs. Use page recycling instead.

skb heads are always copied, and pages are always from page pool in this driver. We could as well mark all allocated skbs for recycling.

LGTM,

#6 lore raw

Re: [PATCH net-next 3/4] net: page_pool: hide page_pool_release_page()

In reply to Jakub Kicinski
Quoted 22 lines · Jakub Kicinski
 Documentation/networking/page_pool.rst | 11 ++++-------
 include/net/page_pool.h                | 10 ++--------
 net/core/page_pool.c                   |  3 +--
 3 files changed, 7 insertions(+), 17 deletions(-)
Documentation/networking/page_pool.rst +3 -3
index 873efd97f822..0aa850cf4447 100644
@@ -13,9 +13,9 @@ replacing dev_alloc_pages().
 
 API keeps track of in-flight pages, in order to let API user know
 when it is safe to free a page_pool object.  Thus, API users
-must run page_pool_release_page() when a page is leaving the page_pool or
-call page_pool_put_page() where appropriate in order to maintain correct
-accounting.
+must call page_pool_put_page() to free the page, or attach
+the page to a page_pool-aware objects like skbs marked with
+skb_mark_for_recycle().
 
 API user must call page_pool_put_page() once on a page, as it
 will either recycle the page, or in case of refcnt > 1, it will

...

Quoted 6 lines
include/net/page_pool.h +0 -0
index 126f9e294389..f1d5cc1fa13b 100644
@@ -18,9 +18,8 @@
  *

include/net/page_pool.h line 25

Quoted 10 lines
include/net/page_pool.h +2 -3
  * API keeps track of in-flight pages, in-order to let API user know
  * when it is safe to dealloactor page_pool object.  Thus, API users
- * must make sure to call page_pool_release_page() when a page is
- * "leaving" the page_pool.  Or call page_pool_put_page() where
- * appropiate.  For maintaining correct accounting.
+ * must call page_pool_put_page() where appropriate and only attach
+ * the page to a page_pool-aware objects, like skbs marked for recycling.
  *
  * API user must only call page_pool_put_page() once on a page, as it
  * will either recycle the page, or in case of elevated refcnt, it

It seems the above comment is almost the same as the one in page_pool.rst, Is there a reason not removing the above to remove the duplicatation?

Reviewed-by on v1 4/4
#9 lore raw

Re: [PATCH net-next 4/4] net: page_pool: merge page_pool_release_page() with page_pool_return_page()

In reply to Jakub Kicinski
Quoted 6 lines · Jakub Kicinski

Now that page_pool_release_page() is not exported we can merge it with page_pool_return_page(). I believe that the "Do not replace this with page_pool_return_page()" comment was there in case page_pool_return_page() was not inlined, to avoid two function calls.

LGTM,

Reviewed-by on the v1 series
#11 lore raw
In reply to Jakub Kicinski

From: Jakub Kicinski <[email protected]>
Date: Wed, 19 Jul 2023 18:04:05 -0700

Quoted 4 lines

page_pool_return_page() is a historic artefact from before recycling of pages attached to skbs was supported. Theoretical uses for it may be thought up but in practice all existing users can be converted to use skb_mark_for_recycle() instead.

For the series:

Good to see that now all PP users use skb page recycling. Apart from benefits for drivers obv, this also increases coverage for the related PP code.

Quoted 10 lines

This code was previously posted as part of the memory provider RFC.

Jakub Kicinski (4):
eth: tsnep: let page recycling happen with skbs
eth: stmmac: let page recycling happen with skbs
net: page_pool: hide page_pool_release_page()
net: page_pool: merge page_pool_release_page() with
page_pool_return_page()

Quoted 7 lines
 Documentation/networking/page_pool.rst            | 11 ++++-------
 drivers/net/ethernet/engleder/tsnep_main.c        |  2 +-
 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c |  4 ++--
 include/net/page_pool.h                           | 10 ++--------
 net/core/page_pool.c                              | 13 ++-----------
 5 files changed, 11 insertions(+), 29 deletions(-)

Thanks,
Olek

#12 lore raw
In reply to Jakub Kicinski
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Quoted 8 lines · On Wed, 19 Jul 2023 18:04:05 -0700 you wrote:

page_pool_return_page() is a historic artefact from before recycling of pages attached to skbs was supported. Theoretical uses for it may be thought up but in practice all existing users can be converted to use skb_mark_for_recycle() instead.

This code was previously posted as part of the memory provider RFC.

[...]

Here is the summary with links:
- [net-next,1/4] eth: tsnep: let page recycling happen with skbs

    https://git.kernel.org/netdev/net-next/c/b03f68ba26c8
  - [net-next,2/4] eth: stmmac: let page recycling happen with skbs
    https://git.kernel.org/netdev/net-next/c/98e2727c79d0
  - [net-next,3/4] net: page_pool: hide page_pool_release_page()
    https://git.kernel.org/netdev/net-next/c/535b9c61bdef
  - [net-next,4/4] net: page_pool: merge page_pool_release_page() with page_pool_return_page()
    https://git.kernel.org/netdev/net-next/c/07e0c7d3179d

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
maintainer Reviewed-by on v1 3/4
#7 lore raw

Re: [PATCH net-next 3/4] net: page_pool: hide page_pool_release_page()

In reply to Jakub Kicinski

Hi Jakub,

Apologies for being late, I was on vacation and will now slowly go through the pile of patches

Quoted 108 lines · Jakub Kicinski

There seems to be no user calling page_pool_release_page() for legit reasons, all the users simply haven't been converted to skb-based recycling, yet. Previous changes converted them. Update the docs, and unexport the function.

---
CC: [email protected]
CC: [email protected]

---
 Documentation/networking/page_pool.rst | 11 ++++-------
 include/net/page_pool.h                | 10 ++--------
 net/core/page_pool.c                   |  3 +--
 3 files changed, 7 insertions(+), 17 deletions(-)
Documentation/networking/page_pool.rst +4 -7
index 873efd97f822..0aa850cf4447 100644
@@ -13,9 +13,9 @@ replacing dev_alloc_pages().
 
 API keeps track of in-flight pages, in order to let API user know
 when it is safe to free a page_pool object.  Thus, API users
-must run page_pool_release_page() when a page is leaving the page_pool or
-call page_pool_put_page() where appropriate in order to maintain correct
-accounting.
+must call page_pool_put_page() to free the page, or attach
+the page to a page_pool-aware objects like skbs marked with
+skb_mark_for_recycle().
 
 API user must call page_pool_put_page() once on a page, as it
 will either recycle the page, or in case of refcnt > 1, it will
@@ -87,9 +87,6 @@ a page will cause no race conditions is enough.
   must guarantee safe context (e.g NAPI), since it will recycle the page
   directly into the pool fast cache.
 
-* page_pool_release_page(): Unmap the page (if mapped) and account for it on
-  in-flight counters.
-
 * page_pool_dev_alloc_pages(): Get a page from the page allocator or page_pool
   caches.
 
@@ -194,7 +191,7 @@ NAPI poller
             if XDP_DROP:
                 page_pool_recycle_direct(page_pool, page);
         } else (packet_is_skb) {
-            page_pool_release_page(page_pool, page);
+            skb_mark_for_recycle(skb);
             new_page = page_pool_dev_alloc_pages(page_pool);
         }
     }
include/net/page_pool.h +2 -8
index 126f9e294389..f1d5cc1fa13b 100644
@@ -18,9 +18,8 @@
  *
  * API keeps track of in-flight pages, in-order to let API user know
  * when it is safe to dealloactor page_pool object.  Thus, API users
- * must make sure to call page_pool_release_page() when a page is
- * "leaving" the page_pool.  Or call page_pool_put_page() where
- * appropiate.  For maintaining correct accounting.
+ * must call page_pool_put_page() where appropriate and only attach
+ * the page to a page_pool-aware objects, like skbs marked for recycling.
  *
  * API user must only call page_pool_put_page() once on a page, as it
  * will either recycle the page, or in case of elevated refcnt, it
@@ -251,7 +250,6 @@ void page_pool_unlink_napi(struct page_pool *pool);
 void page_pool_destroy(struct page_pool *pool);
 void page_pool_use_xdp_mem(struct page_pool *pool, void (*disconnect)(void *),
 			   struct xdp_mem_info *mem);
-void page_pool_release_page(struct page_pool *pool, struct page *page);
 void page_pool_put_page_bulk(struct page_pool *pool, void **data,
 			     int count);
 #else
@@ -268,10 +266,6 @@ static inline void page_pool_use_xdp_mem(struct page_pool *pool,
 					 struct xdp_mem_info *mem)
 {
 }
-static inline void page_pool_release_page(struct page_pool *pool,
-					  struct page *page)
-{
-}
 
 static inline void page_pool_put_page_bulk(struct page_pool *pool, void **data,
 					   int count)
net/core/page_pool.c +1 -3
index a3e12a61d456..2c7cf5f2bcb8 100644
@@ -492,7 +492,7 @@ static s32 page_pool_inflight(struct page_pool *pool)
  * a regular page (that will eventually be returned to the normal
  * page-allocator via put_page).
  */
-void page_pool_release_page(struct page_pool *pool, struct page *page)
+static void page_pool_release_page(struct page_pool *pool, struct page *page)
 {
 	dma_addr_t dma;
 	int count;
@@ -519,7 +519,6 @@ void page_pool_release_page(struct page_pool *pool, struct page *page)
 	count = atomic_inc_return_relaxed(&pool->pages_state_release_cnt);
 	trace_page_pool_state_release(pool, page, count);
 }
-EXPORT_SYMBOL(page_pool_release_page);
 
 /* Return a page to the page allocator, cleaning up our state */
 static void page_pool_return_page(struct page_pool *pool, struct page *page)
--
Quoted 2 lines

2.41.0

maintainer Reviewed-by on v1 4/4
#10 lore raw

Re: [PATCH net-next 4/4] net: page_pool: merge page_pool_release_page() with page_pool_return_page()

In reply to Jakub Kicinski
Quoted 41 lines · Jakub Kicinski

Now that page_pool_release_page() is not exported we can merge it with page_pool_return_page(). I believe that the "Do not replace this with page_pool_return_page()" comment was there in case page_pool_return_page() was not inlined, to avoid two function calls.

---
CC: [email protected]
CC: [email protected]

---
 net/core/page_pool.c | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)
net/core/page_pool.c +1 -7
index 2c7cf5f2bcb8..7ca456bfab71 100644
@@ -492,7 +492,7 @@ static s32 page_pool_inflight(struct page_pool *pool)
  * a regular page (that will eventually be returned to the normal
  * page-allocator via put_page).
  */
-static void page_pool_release_page(struct page_pool *pool, struct page *page)
+static void page_pool_return_page(struct page_pool *pool, struct page *page)
 {
 	dma_addr_t dma;
 	int count;
@@ -518,12 +518,6 @@ static void page_pool_release_page(struct page_pool *pool, struct page *page)
 	 */
 	count = atomic_inc_return_relaxed(&pool->pages_state_release_cnt);
 	trace_page_pool_state_release(pool, page, count);
-}
-
-/* Return a page to the page allocator, cleaning up our state */
-static void page_pool_return_page(struct page_pool *pool, struct page *page)
-{
-	page_pool_release_page(pool, page);
 
 	put_page(page);
 	/* An optimization would be to call __free_pages(page, pool->p.order)

net/core/page_pool.c __page_pool_put_page(struct page_pool *pool, struct page *page, · line 612

Quoted 7 lines
net/core/page_pool.c +1 -3
@@ -615,9 +609,7 @@ __page_pool_put_page(struct page_pool *pool, struct page *page,
 	 * will be invoking put_page.
 	 */
 	recycle_stat_inc(pool, released_refcnt);
-	/* Do not replace this with page_pool_return_page() */
-	page_pool_release_page(pool, page);
-	put_page(page);
+	page_pool_return_page(pool, page);

That comment is my fault. In hindsight, it wasn't very helpful ... IIRC Jesper wanted to keep the calls discrete because we could optimize the page pool internal and eventually not call put_page(). But I am fine with the change regardless, it makes the code easier to read.

Quoted 6 lines
net/core/page_pool.c +0 -1
 	return NULL;
 }
--

2.41.0