Change page_pool timeout handling
Subsystems (8)
cover Change page_pool timeout handling
It isn't safe to remove the page pool until all in-flight pages are returned. If the pages are handed up the stack, it might be a while before they are returned.
The page pool can also be used independently of xdp, so it shouldn't be depending on timeout handling from xdp. Change things around so the pool handles its own timeout.
I wanted to send this out to clarify how I see things working. This is currently being tested with the mlx4/mlx5 drivers for attaching DMA-mapped pages to skbs, sending them up the stack, and then putting them back in the page pool afterwards.
Jonathan Lemon (1):
page_pool: do not release pool until inflight == 0.
.../net/ethernet/stmicro/stmmac/stmmac_main.c | 4 +- include/net/page_pool.h | 55 +++----- include/net/xdp_priv.h | 4 - include/trace/events/xdp.h | 19 +-- net/core/page_pool.c | 115 ++++++++++------ net/core/xdp.c | 130 +++++++----------- 6 files changed, 138 insertions(+), 189 deletions(-)
Signature · 1 line
2.17.1
RFC PATCH 1/1 page_pool: do not release pool until inflight == 0.
The page pool keeps track of the number of pages in flight, and it isn't safe to remove the pool until all pages are returned.
Disallow removing the pool until all pages are back, so the pool is always available for page producers.
Make the page pool responsible for its own delayed destruction instead of relying on XDP, so the page pool can be used without xdp.
When all pages are returned, free the pool and notify xdp if the pool is being being used by xdp. Perform a table walk since some drivers (cpsw) may share the pool among multiple xdp_rxq_info.
--- .../net/ethernet/stmicro/stmmac/stmmac_main.c | 4 +- include/net/page_pool.h | 55 +++----- include/net/xdp_priv.h | 4 - include/trace/events/xdp.h | 19 +-- net/core/page_pool.c | 115 ++++++++++------ net/core/xdp.c | 130 +++++++----------- 6 files changed, 138 insertions(+), 189 deletions(-)
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +1 -3
@@ -1502,10 +1502,8 @@ static void free_dma_rx_desc_resources(struct stmmac_priv *priv) rx_q->dma_erx, rx_q->dma_rx_phy);kfree (rx_q->buf_pool); -if (rx_q->page_pool) { -page_pool_request_shutdown (rx_q->page_pool); +if (rx_q->page_pool)page_pool_destroy (rx_q->page_pool); - } } }
include/net/page_pool.h +16 -39
90 lines not on this page · raw message
include/net/xdp_priv.h +0 -4
@@ -12,12 +12,8 @@ struct xdp_mem_allocator {struct page_pool *page_pool; struct zero_copy_allocator *zc_alloc; }; -int disconnect_cnt; -unsigned long defer_start;struct rhash_head node;struct rcu_head rcu; -struct delayed_work defer_wq; -unsigned long defer_warn; };#endif /* __LINUX_NET_XDP_PRIV_H__ */
include/trace/events/xdp.h +4 -15
@@ -317,19 +317,15 @@ __MEM_TYPE_MAP(__MEM_TYPE_TP_FN) TRACE_EVENT(mem_disconnect, - TP_PROTO(const struct xdp_mem_allocator *xa, - bool safe_to_remove, bool force), + TP_PROTO(const struct xdp_mem_allocator *xa), - TP_ARGS(xa, safe_to_remove, force), + TP_ARGS(xa), TP_STRUCT__entry( __field(const struct xdp_mem_allocator *, xa) __field(u32, mem_id) __field(u32, mem_type) __field(const void *, allocator) - __field(bool, safe_to_remove) - __field(bool, force) - __field(int, disconnect_cnt) ), TP_fast_assign(
@@ -337,19 +333,12 @@ TRACE_EVENT(mem_disconnect, __entry->mem_id = xa->mem.id; __entry->mem_type = xa->mem.type; __entry->allocator = xa->allocator; - __entry->safe_to_remove = safe_to_remove; - __entry->force = force; - __entry->disconnect_cnt = xa->disconnect_cnt; ), -TP_printk ("mem_id=%d mem_type=%s allocator=%p" -" safe_to_remove=%s force=%s disconnect_cnt=%d" , +TP_printk ("mem_id=%d mem_type=%s allocator=%p" , __entry->mem_id,__print_symbolic (__entry->mem_type, __MEM_TYPE_SYM_TAB), - __entry->allocator, - __entry->safe_to_remove ?"true" :"false" , - __entry->force ?"true" :"false" , - __entry->disconnect_cnt + __entry->allocator ) );
net/core/page_pool.c +71 -44
149 lines not on this page · raw message
net/core/xdp.c +46 -84
182 lines not on this page · raw message
Signature · 1 line
2.17.1
Re: [RFC PATCH 1/1] page_pool: do not release pool until inflight == 0.
Quoted 12 lines · On Sun, 10 Nov 2019 22:20:38 -0800 Jonathan Lemon <[email protected]>…
net/core/xdp.c +0
-4@@ -70,25 +70,47 @@ static void __xdp_mem_allocator_rcu_free(struct rcu_head *rcu) xa = container_of(rcu, struct xdp_mem_allocator, rcu); -/* Allocator have indicated safe to remove before this is called */ -if (xa->mem.type ==MEM_TYPE_PAGE_POOL ) -page_pool_free (xa->page_pool); -
net/core/xdp.c static void __xdp_mem_allocator_rcu_free(struct rcu_head *rcu) · line 77
Quoted 10 lines
net/core/xdp.c +0
-5/* Allow this ID to be reused */ ida_simple_remove (&mem_id_pool, xa->mem.id); -/* Poison memory */ - xa->mem.id =0xFFFF ; - xa->mem.type =0xF0F0 ; - xa->allocator = (void *)0xDEAD9001 ; -kfree (xa); }
Can you PLEASE leave the memory poisonings that I have added alone. Removing these are irrelevant for current patch. You clearly don't like this approach, but I've also clearly told that I disagree. I'm the maintainer of this code and I prefer letting them stay. I'm the one that signed up for dealing with hard to find bugs in the code.
I'll try to explain again, hopefully one last time. You argue that the memory subsystem already have use-after-free detection e.g via kmemleak. I argue that these facilities change the timing so much, that race condition will not be provoked when enabled. This is not theoretical, I've seen bugzilla cases consume a huge amount of support and engineering resources, trying to track down bugs that would disappear once the debug facility is enabled.
Signature · 4 lines
Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer
Re: [RFC PATCH 1/1] page_pool: do not release pool until inflight == 0.
Quoted 19 lines · Jesper Dangaard Brouer
Quoted 16 lines · On Sun, 10 Nov 2019 22:20:38 -0800 Jonathan Lemon <[email protected]>…
net/core/xdp.c +0
-4@@ -70,25 +70,47 @@ static void __xdp_mem_allocator_rcu_free(struct rcu_head *rcu) xa = container_of(rcu, struct xdp_mem_allocator, rcu); -/* Allocator have indicated safe to remove before this is called */ -if (xa->mem.type ==MEM_TYPE_PAGE_POOL ) -page_pool_free (xa->page_pool); -/* Allow this ID to be reused */ ida_simple_remove (&mem_id_pool, xa->mem.id);
Quoted 13 lines
net/core/xdp.c static void __xdp_mem_allocator_rcu_free(struct rcu_head *rcu) · line 77
Quoted 7 lines
net/core/xdp.c +0
-5-/* Poison memory */ - xa->mem.id =0xFFFF ; - xa->mem.type =0xF0F0 ; - xa->allocator = (void *)0xDEAD9001 ; -kfree (xa); }Can you PLEASE leave the memory poisonings that I have added alone. Removing these are irrelevant for current patch. You clearly don't like this approach, but I've also clearly told that I disagree. I'm the maintainer of this code and I prefer letting them stay. I'm the one that signed up for dealing with hard to find bugs in the code.
As you're the maintainer, I'll put this back. However, I strongly feel that this is poor practice; detecting use-after-free in cases like this should be the job of the page and memory allocator; not kmemleak (which detects lost memory), but page poisoning and redzone support.
If this is really useful, it should specifically be under DEBUG support, rather than in the mainline code. It's not a performance issue - just more techdebt, from my POV.
Signature · 1 line
Jonathan
Re: [RFC PATCH 1/1] page_pool: do not release pool until inflight == 0.
In reply to Jonathan LemonQuoted 7 lines · On Sun, 10 Nov 2019 22:20:38 -0800 Jonathan Lemon <[email protected]>…
The page pool keeps track of the number of pages in flight, and it isn't safe to remove the pool until all pages are returned.
Disallow removing the pool until all pages are back, so the pool is always available for page producers.
Make the page pool responsible for its own delayed destruction
I like this part, making page_pool responsible for its own delayed destruction. I originally also wanted to do this, but got stuck on mem.id getting removed prematurely from rhashtable. You actually solved this, via introducing a disconnect callback, from page_pool into mem_allocator_disconnect(). I like it.
Quoted 2 lines
instead of relying on XDP, so the page pool can be used without xdp.
This is a misconception, the xdp_rxq_info_reg_mem_model API does not imply driver is using XDP. Yes, I know the naming is sort of wrong, contains "xdp". Also the xdp_mem_info name. Ilias and I have discussed to rename this several times.
The longer term plan is/was to use this (xdp_)mem_info as generic return path for SKBs, creating a more flexible memory model for networking. This patch is fine and in itself does not disrupt/change that, but your offlist changes does. As your offlist changes does imply a performance gain, I will likely accept this (and then find another plan for more flexible memory model for networking).
Quoted 3 lines
When all pages are returned, free the pool and notify xdp if the pool is being being used by xdp. Perform a table walk since some drivers (cpsw) may share the pool among multiple
xdp_rxq_info.
I misunderstood this description, first after reading the code in details, I realized that this describe your disconnect callback. And how the mem.id removal is safe, by being delayed until after all pages
are returned. The notes below is the code, was just for me to follow
this disconnect callback system, which I think is fine... left it if others also want to double check the correctness.
Quoted 2 lines
Fixes d956a048cd3f ("xdp: force mem allocator removal and periodic warning")
No newline between "Fixes" line and :Signed-off-by:
Quoted 8 lines
Signed-off-by Jonathan Lemon <[email protected]>--- .../net/ethernet/stmicro/stmmac/stmmac_main.c | 4 +- include/net/page_pool.h | 55 +++----- include/net/xdp_priv.h | 4 - include/trace/events/xdp.h | 19 +-- net/core/page_pool.c | 115 ++++++++++------ net/core/xdp.c | 130 +++++++-----------
[...]
Quoted 4 lines
net/core/page_pool.c +0
-0
[...]
Quoted 23 lines
/* Cleanup
page_poolstate from page */net/core/page_pool.c +1
-16@@ -338,31 +333,10 @@ static void __page_pool_empty_ring(struct page_pool *pool) } } -static void __warn_in_flight (struct page_pool *pool) +static void page_pool_free (struct page_pool *pool) { -u32 release_cnt =atomic_read (&pool->pages_state_release_cnt); -u32 hold_cnt =READ_ONCE (pool->pages_state_hold_cnt); -s32 distance; - - distance =_distance (hold_cnt, release_cnt); - -/* Drivers should fix this, but only problematic when DMA is used */ -WARN (1 ,"Still in-flight pages:%d hold:%u released:%u" , - distance, hold_cnt, release_cnt); -} - -void __page_pool_free (struct page_pool *pool) -{ -/* Only last user actually free/release resources */ -if (!page_pool_put (pool))
net/core/page_pool.c static void __page_pool_empty_ring(struct page_pool *pool) · line 339
Quoted 10 lines
net/core/page_pool.c +2
-8-return ; - -WARN (pool->alloc.count,"API usage violation" ); -WARN (!ptr_ring_empty (&pool->ring),"ptr_ring is not empty" ); - -/* Can happen due to forced shutdown */ -if (!__page_pool_safe_to_destroy (pool)) -__warn_in_flight (pool); +if (pool->disconnect) + pool->disconnect (pool);
Callback to mem reg system.
Quoted 56 lines
net/core/page_pool.c +32
-8ptr_ring_cleanup (&pool->ring,NULL );@@ -371,12 +345,8 @@ void __page_pool_free(struct page_pool *pool)kfree (pool); } -EXPORT_SYMBOL (__page_pool_free); -/* Request to shutdown: release pages cached by page_pool, and check -* for in-flight pages -*/ -bool __page_pool_request_shutdown (struct page_pool *pool) +static void page_pool_scrub (struct page_pool *pool) {struct page *page;@@ -393,7 +363,64 @@ bool __page_pool_request_shutdown(struct page_pool *pool)* be in-flight. */ __page_pool_empty_ring (pool); - -return __page_pool_safe_to_destroy (pool); } -EXPORT_SYMBOL (__page_pool_request_shutdown); + +static int page_pool_release (struct page_pool *pool) +{ +int inflight; + +page_pool_scrub (pool); + inflight =page_pool_inflight (pool); +if (!inflight) +page_pool_free (pool); + +return inflight; +} + +static void page_pool_release_retry (struct work_struct *wq) +{ +struct delayed_work *dwq =to_delayed_work (wq); +struct page_pool *pool =container_of (dwq,typeof (*pool), release_dw); +int inflight; + + inflight =page_pool_release (pool); +if (!inflight) +return ; + +/* Periodic warning */ +if (time_after_eq (jiffies, pool->defer_warn)) { +int sec = (s32)((u32 )jiffies - (u32 )pool->defer_start) /HZ ; + +pr_warn ("%s() stalled pool shutdown %d inflight %d sec\n" , + __func__, inflight, sec); + pool->defer_warn = jiffies +DEFER_WARN_INTERVAL ; + }
net/core/page_pool.c bool __page_pool_request_shutdown(struct page_pool *pool) · line 407
Quoted 10 lines
net/core/page_pool.c +10
-0+ +/* Still not ready to be disconnected, retry later */ +schedule_delayed_work (&pool->release_dw,DEFER_TIME ); +} + +void page_pool_use_xdp_mem (struct page_pool *pool,void (*disconnect)(void *)) +{ +refcount_inc (&pool->user_cnt); + pool->disconnect = disconnect; +}
Function page_pool_use_xdp_mem is used by xdp.c to register the callback.
Quoted 25 lines
net/core/page_pool.c +18
-0+void page_pool_destroy (struct page_pool *pool) +{ +if (!pool) +return ; + +if (!page_pool_put (pool)) +return ; + +if (!page_pool_release (pool)) +return ; + + pool->defer_start = jiffies; + pool->defer_warn = jiffies +DEFER_WARN_INTERVAL ; + +INIT_DELAYED_WORK (&pool->release_dw, page_pool_release_retry); +schedule_delayed_work (&pool->release_dw,DEFER_TIME ); +} +EXPORT_SYMBOL (page_pool_destroy);net/core/xdp.c +0
-0void xdp_rxq_info_unreg_mem_model (struct xdp_rxq_info *xdp_rxq)@@ -153,38 +139,21 @@ void xdp_rxq_info_unreg_mem_model(struct xdp_rxq_info *xdp_rxq)
[...]
net/core/xdp.c void xdp_rxq_info_unreg_mem_model(struct xdp_rxq_info *xdp_rxq) · line 153
Quoted 6 lines
net/core/xdp.c +5
-0+if (xdp_rxq->mem.type ==MEM_TYPE_PAGE_POOL ) { +rcu_read_lock (); + xa =rhashtable_lookup (mem_id_ht, &id, mem_id_rht_params); +page_pool_destroy (xa->page_pool); +rcu_read_unlock (); }
[...]
Calling page_pool_destroy() instead of mem_allocator_disconnect().
net/core/xdp.c int xdp_rxq_info_reg_mem_model(struct xdp_rxq_info *xdp_rxq, · line 343
Quoted 6 lines
net/core/xdp.c +1
-1@@ -371,7 +340,7 @@ int xdp_rxq_info_reg_mem_model(struct xdp_rxq_info *xdp_rxq, }if (type ==MEM_TYPE_PAGE_POOL ) -page_pool_get (xdp_alloc->page_pool); +page_pool_use_xdp_mem (allocator, mem_allocator_disconnect);
Register callback to mem_allocator_disconnect().
Quoted 11 lines
net/core/xdp.c +0
-4mutex_unlock (&mem_id_lock);@@ -402,15 +371,8 @@ static void __xdp_return(void *data, struct xdp_mem_info *mem, bool napi_direct,/* mem->id is valid, checked in xdp_rxq_info_reg_mem_model() */ xa =rhashtable_lookup (mem_id_ht, &mem->id, mem_id_rht_params); page =virt_to_head_page (data); -if (likely (xa)) { - napi_direct &= !xdp_return_frame_no_direct (); -page_pool_put_page (xa->page_pool, page, napi_direct); - }else {
net/core/xdp.c static void __xdp_return(void *data, struct xdp_mem_info *mem, bool napi_direct, · line 378
Quoted 10 lines
net/core/xdp.c +2
-5-/* Hopefully stack show who to blame for late return */ -WARN_ONCE (1 ,"page_pool gone mem.id=%d" , mem->id); -trace_mem_return_failed (mem, page); -put_page (page); - } + napi_direct &= !xdp_return_frame_no_direct (); +page_pool_put_page (xa->page_pool, page, napi_direct);rcu_read_unlock ();break ;case MEM_TYPE_PAGE_SHARED :
This should be correct.
Signature · 4 lines
Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer
Ilias Apalodimas
maintainerRe: [RFC PATCH 1/1] page_pool: do not release pool until inflight == 0.
Hi Jesper, Jonathan,
Quoted 9 lines · Jesper Dangaard Brouer
Quoted 6 lines · On Sun, 10 Nov 2019 22:20:38 -0800 Jonathan Lemon <[email protected]>…
The page pool keeps track of the number of pages in flight, and it isn't safe to remove the pool until all pages are returned.
Disallow removing the pool until all pages are back, so the pool is always available for page producers.
Quoted 7 lines
Quoted 1 line
Make the page pool responsible for its own delayed destruction
I like this part, making
page_poolresponsible for its own delayed destruction. I originally also wanted to do this, but got stuck on mem.id getting removed prematurely from rhashtable. You actually solved this, via introducing a disconnect callback, frompage_poolintomem_allocator_disconnect(). I like it.
+1 here. It seems cleaner since we'll keep the pool alive until all pages have been dealt with
Quoted 8 lines
Quoted 3 lines
instead of relying on XDP, so the page pool can be used without xdp.
This is a misconception, the
xdp_rxq_info_reg_mem_modelAPI does not imply driver is using XDP. Yes, I know the naming is sort of wrong, contains "xdp". Also thexdp_mem_infoname. Ilias and I have discussed to rename this several times.
Yes. We'll try to document that since it's a bit confusing :(
Quoted 8 lines
The longer term plan is/was to use this (xdp_)mem_info as generic return path for SKBs, creating a more flexible memory model for networking. This patch is fine and in itself does not disrupt/change that, but your offlist changes does. As your offlist changes does imply a performance gain, I will likely accept this (and then find another plan for more flexible memory model for networking).
Keep in mind the stmmac driver is using the pool differently.
They figured out mapping and unmapping was more expensive that copying memory in that hardware.
So the NAPI Rx handler copies the page pool data into an skb and recycles the buffer immediately. Tehy should have 0 inflight packets during the teardown
Quoted 12 lines
Quoted 4 lines
When all pages are returned, free the pool and notify xdp if the pool is being being used by xdp. Perform a table walk since some drivers (cpsw) may share the pool among multiple
xdp_rxq_info.I misunderstood this description, first after reading the code in details, I realized that this describe your disconnect callback. And how the mem.id removal is safe, by being delayed until after all pages
are returned. The notes below is the code, was just for me to follow
this disconnect callback system, which I think is fine... left it if others also want to double check the correctness.
Quoted 4 lines
Quoted 2 lines
Fixes d956a048cd3f ("xdp: force mem allocator removal and periodic warning")No newline between "Fixes" line and :Signed-off-by:
Thanks for working on this
/Ilias
Quoted 197 lines
Quoted 8 lines
Signed-off-by Jonathan Lemon <[email protected]>--- .../net/ethernet/stmicro/stmmac/stmmac_main.c | 4 +- include/net/page_pool.h | 55 +++----- include/net/xdp_priv.h | 4 - include/trace/events/xdp.h | 19 +-- net/core/page_pool.c | 115 ++++++++++------ net/core/xdp.c | 130 +++++++-----------[...]
Quoted 4 lines
net/core/page_pool.c +0
-0[...]
Quoted 23 lines
/* Cleanup
page_poolstate from page */net/core/page_pool.c +1
-16@@ -338,31 +333,10 @@ static void __page_pool_empty_ring(struct page_pool *pool) } } -static void __warn_in_flight (struct page_pool *pool) +static void page_pool_free (struct page_pool *pool) { -u32 release_cnt =atomic_read (&pool->pages_state_release_cnt); -u32 hold_cnt =READ_ONCE (pool->pages_state_hold_cnt); -s32 distance; - - distance =_distance (hold_cnt, release_cnt); - -/* Drivers should fix this, but only problematic when DMA is used */ -WARN (1 ,"Still in-flight pages:%d hold:%u released:%u" , - distance, hold_cnt, release_cnt); -} - -void __page_pool_free (struct page_pool *pool) -{ -/* Only last user actually free/release resources */ -if (!page_pool_put (pool))net/core/page_pool.c static void __page_pool_empty_ring(struct page_pool *pool) · line 339
Quoted 10 lines
net/core/page_pool.c +2
-8-return ; - -WARN (pool->alloc.count,"API usage violation" ); -WARN (!ptr_ring_empty (&pool->ring),"ptr_ring is not empty" ); - -/* Can happen due to forced shutdown */ -if (!__page_pool_safe_to_destroy (pool)) -__warn_in_flight (pool); +if (pool->disconnect) + pool->disconnect (pool);Callback to mem reg system.
Quoted 56 lines
net/core/page_pool.c +32
-8ptr_ring_cleanup (&pool->ring,NULL );@@ -371,12 +345,8 @@ void __page_pool_free(struct page_pool *pool)kfree (pool); } -EXPORT_SYMBOL (__page_pool_free); -/* Request to shutdown: release pages cached by page_pool, and check -* for in-flight pages -*/ -bool __page_pool_request_shutdown (struct page_pool *pool) +static void page_pool_scrub (struct page_pool *pool) {struct page *page;@@ -393,7 +363,64 @@ bool __page_pool_request_shutdown(struct page_pool *pool)* be in-flight. */ __page_pool_empty_ring (pool); - -return __page_pool_safe_to_destroy (pool); } -EXPORT_SYMBOL (__page_pool_request_shutdown); + +static int page_pool_release (struct page_pool *pool) +{ +int inflight; + +page_pool_scrub (pool); + inflight =page_pool_inflight (pool); +if (!inflight) +page_pool_free (pool); + +return inflight; +} + +static void page_pool_release_retry (struct work_struct *wq) +{ +struct delayed_work *dwq =to_delayed_work (wq); +struct page_pool *pool =container_of (dwq,typeof (*pool), release_dw); +int inflight; + + inflight =page_pool_release (pool); +if (!inflight) +return ; + +/* Periodic warning */ +if (time_after_eq (jiffies, pool->defer_warn)) { +int sec = (s32)((u32 )jiffies - (u32 )pool->defer_start) /HZ ; + +pr_warn ("%s() stalled pool shutdown %d inflight %d sec\n" , + __func__, inflight, sec); + pool->defer_warn = jiffies +DEFER_WARN_INTERVAL ; + }net/core/page_pool.c bool __page_pool_request_shutdown(struct page_pool *pool) · line 407
Quoted 10 lines
net/core/page_pool.c +10
-0+ +/* Still not ready to be disconnected, retry later */ +schedule_delayed_work (&pool->release_dw,DEFER_TIME ); +} + +void page_pool_use_xdp_mem (struct page_pool *pool,void (*disconnect)(void *)) +{ +refcount_inc (&pool->user_cnt); + pool->disconnect = disconnect; +}Function
page_pool_use_xdp_memis used byxdp.cto register the callback.Quoted 25 lines
net/core/page_pool.c +18
-0+void page_pool_destroy (struct page_pool *pool) +{ +if (!pool) +return ; + +if (!page_pool_put (pool)) +return ; + +if (!page_pool_release (pool)) +return ; + + pool->defer_start = jiffies; + pool->defer_warn = jiffies +DEFER_WARN_INTERVAL ; + +INIT_DELAYED_WORK (&pool->release_dw, page_pool_release_retry); +schedule_delayed_work (&pool->release_dw,DEFER_TIME ); +} +EXPORT_SYMBOL (page_pool_destroy);net/core/xdp.c +0
-0void xdp_rxq_info_unreg_mem_model (struct xdp_rxq_info *xdp_rxq)@@ -153,38 +139,21 @@ void xdp_rxq_info_unreg_mem_model(struct xdp_rxq_info *xdp_rxq)[...]
net/core/xdp.c void xdp_rxq_info_unreg_mem_model(struct xdp_rxq_info *xdp_rxq) · line 153
Quoted 6 lines
net/core/xdp.c +5
-0+if (xdp_rxq->mem.type ==MEM_TYPE_PAGE_POOL ) { +rcu_read_lock (); + xa =rhashtable_lookup (mem_id_ht, &id, mem_id_rht_params); +page_pool_destroy (xa->page_pool); +rcu_read_unlock (); }[...]
Calling
page_pool_destroy()instead ofmem_allocator_disconnect().net/core/xdp.c int xdp_rxq_info_reg_mem_model(struct xdp_rxq_info *xdp_rxq, · line 343
Quoted 6 lines
net/core/xdp.c +1
-1@@ -371,7 +340,7 @@ int xdp_rxq_info_reg_mem_model(struct xdp_rxq_info *xdp_rxq, }if (type ==MEM_TYPE_PAGE_POOL ) -page_pool_get (xdp_alloc->page_pool); +page_pool_use_xdp_mem (allocator, mem_allocator_disconnect);Register callback to
mem_allocator_disconnect().Quoted 11 lines
net/core/xdp.c +0
-4mutex_unlock (&mem_id_lock);@@ -402,15 +371,8 @@ static void __xdp_return(void *data, struct xdp_mem_info *mem, bool napi_direct,/* mem->id is valid, checked in xdp_rxq_info_reg_mem_model() */ xa =rhashtable_lookup (mem_id_ht, &mem->id, mem_id_rht_params); page =virt_to_head_page (data); -if (likely (xa)) { - napi_direct &= !xdp_return_frame_no_direct (); -page_pool_put_page (xa->page_pool, page, napi_direct); - }else {net/core/xdp.c static void __xdp_return(void *data, struct xdp_mem_info *mem, bool napi_direct, · line 378
Quoted 10 lines
net/core/xdp.c +2
-5-/* Hopefully stack show who to blame for late return */ -WARN_ONCE (1 ,"page_pool gone mem.id=%d" , mem->id); -trace_mem_return_failed (mem, page); -put_page (page); - } + napi_direct &= !xdp_return_frame_no_direct (); +page_pool_put_page (xa->page_pool, page, napi_direct);rcu_read_unlock ();break ;case MEM_TYPE_PAGE_SHARED :This should be correct.
Signature · 4 lines
Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer
Re: [RFC PATCH 1/1] page_pool: do not release pool until inflight == 0.
In reply to Jesper Dangaard BrouerQuoted 25 lines · Jesper Dangaard Brouer
Quoted 7 lines · On Sun, 10 Nov 2019 22:20:38 -0800 Jonathan Lemon <[email protected]>…
The page pool keeps track of the number of pages in flight, and it isn't safe to remove the pool until all pages are returned.
Disallow removing the pool until all pages are back, so the pool is always available for page producers.
Make the page pool responsible for its own delayed destruction
I like this part, making
page_poolresponsible for its own delayed destruction. I originally also wanted to do this, but got stuck on mem.id getting removed prematurely from rhashtable. You actually solved this, via introducing a disconnect callback, frompage_poolintomem_allocator_disconnect(). I like it.Quoted 2 lines
instead of relying on XDP, so the page pool can be used without xdp.
This is a misconception, the
xdp_rxq_info_reg_mem_modelAPI does not imply driver is using XDP. Yes, I know the naming is sort of wrong, contains "xdp". Also thexdp_mem_infoname. Ilias and I have discussed to rename this several times.
Quoted 6 lines
The longer term plan is/was to use this (xdp_)mem_info as generic return path for SKBs, creating a more flexible memory model for networking. This patch is fine and in itself does not disrupt/change that, but your offlist changes does. As your offlist changes does imply a performance gain, I will likely accept this (and then find another plan for more flexible memory model for networking).
Are you referring to the patch which encodes the page pool pointer in the page, and then sends it directly to the pool on skb free instead of performing a mem id lookup and indirection through the memory model?
It could be done either way. I'm not seeing any advantages of the additional indirection, as the pool lifetime is guaranteed.
All that is needed is:
1) A way to differentiate this page as coming from the page pool.
The current plan of setting a bit on the skb which indicates that the pages should be returned via the page pool is workable, but there will be some pages returned which came from the system page allocator, and these need to be filtered out.
There must be some type of signature the page permits filtering and returning non-matching pages back to the page allocator.
2) Identifying up exactly which page pool the page belongs to.
This could be done by just placing the pool pointer on the page, or putting the mem info there and indirecting through the lookup.
Signature · 1 line
Jonathan
kbuild test robot
build botRe: [RFC PATCH 1/1] page_pool: do not release pool until inflight == 0.
In reply to Jonathan Lemon[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please…
Hi Jonathan,
[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on net-next/master]
[also build test ERROR on v5.4-rc7 next-20191112]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Jonathan-Lemon/page_pool-do-not-release-pool-until-inflight-0/20191112-204355 base: https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git ca22d6977b9b4ab0fd2e7909b57e32ba5b95046f config: parisc-c3000_defconfig (attached as .config) compiler: hppa-linux-gcc (GCC) 7.4.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=7.4.0 make.cross ARCH=parisc
If you fix the issue, kindly add following tag
All error/warnings (new ones prefixed by >>):
In file included from net/core/xdp.c:14:0:
>> include/net/page_pool.h:148:1: error: expected identifier or '(' before '{' token
{
^
>> include/net/page_pool.h:146:20: warning: 'page_pool_use_xdp_mem' used but never defined static inline void page_pool_use_xdp_mem(struct page_pool *pool,
^~~~~~~~~~~~~~~~~~~~~
vim +148 include/net/page_pool.h
145
> 146 static inline void page_pool_use_xdp_mem(struct page_pool *pool,
147 void (*disconnect)(void *));
> 148 { 149 }
150 #endif
151
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org Intel Corporation