Skip to contentSkip to conversation

mvpp2: page_pool support

RFC In review Matteo Croce 3 participants 9 messages Dec 24, 2019 – Dec 27, 2019

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover mvpp2: page_pool support

This patches change the memory allocator of mvpp2 from the frag allocator to the page_pool API. This change is needed to add later XDP support to mvpp2.

The reason I send it as RFC is that with this changeset, mvpp2 performs much more slower. This is the tc drop rate measured with a single flow:

stock net-next with frag allocator:
rx: 900.7 Mbps 1877 Kpps

this patchset with page_pool:
rx: 423.5 Mbps 882.3 Kpps

This is the perf top when receiving traffic:

  27.68%  [kernel]            [k] __page_pool_clean_page
   9.79%  [kernel]            [k] get_page_from_freelist
   7.18%  [kernel]            [k] free_unref_page
   4.64%  [kernel]            [k] build_skb
   4.63%  [kernel]            [k] __netif_receive_skb_core
   3.83%  [mvpp2]             [k] mvpp2_poll
   3.64%  [kernel]            [k] eth_type_trans
   3.61%  [kernel]            [k] kmem_cache_free
   3.03%  [kernel]            [k] kmem_cache_alloc
   2.76%  [kernel]            [k] dev_gro_receive
   2.69%  [mvpp2]             [k] mvpp2_bm_pool_put
   2.68%  [kernel]            [k] page_frag_free
   1.83%  [kernel]            [k] inet_gro_receive
   1.74%  [kernel]            [k] page_pool_alloc_pages
   1.70%  [kernel]            [k] __build_skb
   1.47%  [kernel]            [k] __alloc_pages_nodemask
   1.36%  [mvpp2]             [k] mvpp2_buf_alloc.isra.0
   1.29%  [kernel]            [k] tcf_action_exec

I tried Ilias patches for page_pool recycling, I get an improvement to ~1100, but I'm still far than the original allocator.

Any idea on why I get such bad numbers?

Another reason to send it as RFC is that I'm not fully convinced on how to use the page_pool given the HW limitation of the BM.

The driver currently uses, for every CPU, a page_pool for short packets and another for long ones. The driver also has 4 rx queue per port, so every RXQ #1 will share the short and long page pools of CPU #1.

This means that for every RX queue I call xdp_rxq_info_reg_mem_model() twice, on two different page_pool, can this be a problem?

As usual, ideas are welcome.

Matteo Croce (2):
mvpp2: use page_pool allocator
mvpp2: memory accounting

 drivers/net/ethernet/marvell/Kconfig          |   1 +
 drivers/net/ethernet/marvell/mvpp2/mvpp2.h    |   7 +
 .../net/ethernet/marvell/mvpp2/mvpp2_main.c   | 142 +++++++++++++++---
 3 files changed, 125 insertions(+), 25 deletions(-)
Signature · 1 line
2.24.1
#1 lore raw

RFC 1/2 mvpp2: use page_pool allocator

Use the page_pool API for memory management. This is a prerequisite for native XDP support.

---
 drivers/net/ethernet/marvell/Kconfig          |   1 +
 drivers/net/ethernet/marvell/mvpp2/mvpp2.h    |   4 +
 .../net/ethernet/marvell/mvpp2/mvpp2_main.c   | 111 ++++++++++++++----
 3 files changed, 92 insertions(+), 24 deletions(-)
drivers/net/ethernet/marvell/Kconfig +1 -0
index 3d5caea096fb..e2612cc4920d 100644
@@ -87,6 +87,7 @@ config MVPP2
 	depends on ARCH_MVEBU || COMPILE_TEST
 	select MVMDIO
 	select PHYLINK
+	select PAGE_POOL
 	---help---
 	  This driver supports the network interface units in the
 	  Marvell ARMADA 375, 7K and 8K SoCs.
drivers/net/ethernet/marvell/mvpp2/mvpp2.h +4 -0
index 543a310ec102..67b3bf0d3c8b 100644
@@ -15,6 +15,7 @@
 #include <linux/phy.h>
 #include <linux/phylink.h>
 #include <net/flow_offload.h>
+#include <net/page_pool.h>
 
 /* Fifo Registers */
 #define MVPP2_RX_DATA_FIFO_SIZE_REG(port)	(0x00 + 4 * (port))
@@ -820,6 +821,9 @@ struct mvpp2 {
 
 	/* RSS Indirection tables */
 	struct mvpp2_rss_table *rss_tables[MVPP22_N_RSS_TABLES];
+
+	/* page_pool allocator */
+	struct page_pool *page_pool[MVPP2_PORT_MAX_RXQ];
 };
 
 struct mvpp2_pcpu_stats {
drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +87 -24

222 lines not on this page · raw message

Signature · 1 line
2.24.1
#2 lore raw

RFC 2/2 mvpp2: memory accounting

In reply to Matteo Croce

Use the XDP API for memory accounting.

---
 drivers/net/ethernet/marvell/mvpp2/mvpp2.h    |  3 ++
 .../net/ethernet/marvell/mvpp2/mvpp2_main.c   | 31 ++++++++++++++++++-
 2 files changed, 33 insertions(+), 1 deletion(-)
drivers/net/ethernet/marvell/mvpp2/mvpp2.h +3 -0
index 67b3bf0d3c8b..ffd633e0a3be 100644
@@ -1165,6 +1165,9 @@ struct mvpp2_rx_queue {
 
 	/* Port's logic RXQ number to which physical RXQ is mapped */
 	int logic_rxq;
+
+	/* XDP memory accounting */
+	struct xdp_rxq_info xdp_rxq;
 };
 
 struct mvpp2_bm_pool {
drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +30 -1
index 4edb81c8941f..3b0aac66ac52 100644
@@ -2405,10 +2405,11 @@ static int mvpp2_aggr_txq_init(struct platform_device *pdev,
 /* Create a specified Rx queue */
 static int mvpp2_rxq_init(struct mvpp2_port *port,
 			  struct mvpp2_rx_queue *rxq)
-
 {
+	struct mvpp2 *priv = port->priv;
 	unsigned int thread;
 	u32 rxq_dma;
+	int err;
 
 	rxq->size = port->rx_ring_size;
 
@@ -2446,7 +2447,35 @@ static int mvpp2_rxq_init(struct mvpp2_port *port,
 	/* Add number of descriptors ready for receiving packets */
 	mvpp2_rxq_status_update(port, rxq->id, 0, rxq->size);
 
+	err = xdp_rxq_info_reg(&rxq->xdp_rxq, port->dev, rxq->id);
+	if (err < 0)
+		goto err_free_dma;
+
+	if (priv->percpu_pools) {
+		/* Every RXQ has a pool for short and another for long packets */
+		err = xdp_rxq_info_reg_mem_model(&rxq->xdp_rxq,
+						 MEM_TYPE_PAGE_POOL,
+						 priv->page_pool[rxq->logic_rxq]);
+		if (err < 0)
+			goto err_unregister_rxq;
+
+		err = xdp_rxq_info_reg_mem_model(&rxq->xdp_rxq,
+						 MEM_TYPE_PAGE_POOL,
+						 priv->page_pool[rxq->logic_rxq +
+								 port->nrxqs]);
+		if (err < 0)
+			goto err_unregister_rxq;
+	}
+
 	return 0;
+
+err_unregister_rxq:
+	xdp_rxq_info_unreg(&rxq->xdp_rxq);
+err_free_dma:
+	dma_free_coherent(port->dev->dev.parent,
+			  rxq->size * MVPP2_DESC_ALIGNED_SIZE,
+			  rxq->descs, rxq->descs_dma);
+	return err;
 }
 
 /* Push packets received by the RXQ to BM pool */
Signature · 1 line
2.24.1
#3 lore raw
In reply to Matteo Croce
Quoted 9 lines · Matteo Croce

This patches change the memory allocator of mvpp2 from the frag allocator to the page_pool API. This change is needed to add later XDP support to mvpp2.

The reason I send it as RFC is that with this changeset, mvpp2 performs much more slower. This is the tc drop rate measured with a single flow:

stock net-next with frag allocator:
rx: 900.7 Mbps 1877 Kpps

Quoted 6 lines

this patchset with page_pool:
rx: 423.5 Mbps 882.3 Kpps

This is the perf top when receiving traffic:

27.68% [kernel] [k] __page_pool_clean_page

This seems extremly high on the list.

Quoted 18 lines
   9.79%  [kernel]            [k] get_page_from_freelist
   7.18%  [kernel]            [k] free_unref_page
   4.64%  [kernel]            [k] build_skb
   4.63%  [kernel]            [k] __netif_receive_skb_core
   3.83%  [mvpp2]             [k] mvpp2_poll
   3.64%  [kernel]            [k] eth_type_trans
   3.61%  [kernel]            [k] kmem_cache_free
   3.03%  [kernel]            [k] kmem_cache_alloc
   2.76%  [kernel]            [k] dev_gro_receive
   2.69%  [mvpp2]             [k] mvpp2_bm_pool_put
   2.68%  [kernel]            [k] page_frag_free
   1.83%  [kernel]            [k] inet_gro_receive
   1.74%  [kernel]            [k] page_pool_alloc_pages
   1.70%  [kernel]            [k] __build_skb
   1.47%  [kernel]            [k] __alloc_pages_nodemask
   1.36%  [mvpp2]             [k] mvpp2_buf_alloc.isra.0
   1.29%  [kernel]            [k] tcf_action_exec
Quoted 2 lines

I tried Ilias patches for page_pool recycling, I get an improvement to ~1100, but I'm still far than the original allocator.

Can you post the recycling perf for comparison?

Quoted 2 lines

Any idea on why I get such bad numbers?

Nop but it's indeed strange

Quoted 3 lines

Another reason to send it as RFC is that I'm not fully convinced on how to use the page_pool given the HW limitation of the BM.

I'll have a look right after holidays

Quoted 5 lines

The driver currently uses, for every CPU, a page_pool for short packets and another for long ones. The driver also has 4 rx queue per port, so every RXQ #1 will share the short and long page pools of CPU #1.

I am not sure i am following the hardware config here

Quoted 5 lines

This means that for every RX queue I call xdp_rxq_info_reg_mem_model() twice, on two different page_pool, can this be a problem?

As usual, ideas are welcome.

Quoted 12 lines

Matteo Croce (2):
mvpp2: use page_pool allocator
mvpp2: memory accounting

 drivers/net/ethernet/marvell/Kconfig          |   1 +
 drivers/net/ethernet/marvell/mvpp2/mvpp2.h    |   7 +
 .../net/ethernet/marvell/mvpp2/mvpp2_main.c   | 142 +++++++++++++++---
 3 files changed, 125 insertions(+), 25 deletions(-)
Signature · 1 line
2.24.1

Cheers
/Ilias

#4 lore raw
Quoted 38 lines · Ilias Apalodimas
Quoted 9 lines · Matteo Croce

This patches change the memory allocator of mvpp2 from the frag allocator to the page_pool API. This change is needed to add later XDP support to mvpp2.

The reason I send it as RFC is that with this changeset, mvpp2 performs much more slower. This is the tc drop rate measured with a single flow:

stock net-next with frag allocator:
rx: 900.7 Mbps 1877 Kpps

Quoted 6 lines

this patchset with page_pool:
rx: 423.5 Mbps 882.3 Kpps

This is the perf top when receiving traffic:

27.68% [kernel] [k] __page_pool_clean_page

This seems extremly high on the list.

Quoted 18 lines
   9.79%  [kernel]            [k] get_page_from_freelist
   7.18%  [kernel]            [k] free_unref_page
   4.64%  [kernel]            [k] build_skb
   4.63%  [kernel]            [k] __netif_receive_skb_core
   3.83%  [mvpp2]             [k] mvpp2_poll
   3.64%  [kernel]            [k] eth_type_trans
   3.61%  [kernel]            [k] kmem_cache_free
   3.03%  [kernel]            [k] kmem_cache_alloc
   2.76%  [kernel]            [k] dev_gro_receive
   2.69%  [mvpp2]             [k] mvpp2_bm_pool_put
   2.68%  [kernel]            [k] page_frag_free
   1.83%  [kernel]            [k] inet_gro_receive
   1.74%  [kernel]            [k] page_pool_alloc_pages
   1.70%  [kernel]            [k] __build_skb
   1.47%  [kernel]            [k] __alloc_pages_nodemask
   1.36%  [mvpp2]             [k] mvpp2_buf_alloc.isra.0
   1.29%  [kernel]            [k] tcf_action_exec
Quoted 5 lines
Quoted 2 lines

I tried Ilias patches for page_pool recycling, I get an improvement to ~1100, but I'm still far than the original allocator.

Can you post the recycling perf for comparison?

  12.00%  [kernel]                  [k] get_page_from_freelist
   9.25%  [kernel]                  [k] free_unref_page
   6.83%  [kernel]                  [k] eth_type_trans
   5.33%  [kernel]                  [k] __netif_receive_skb_core
   4.96%  [mvpp2]                   [k] mvpp2_poll
   4.64%  [kernel]                  [k] kmem_cache_free
   4.06%  [kernel]                  [k] __xdp_return
   3.60%  [kernel]                  [k] kmem_cache_alloc
   3.31%  [kernel]                  [k] dev_gro_receive
   3.29%  [kernel]                  [k] __page_pool_clean_page
   3.25%  [mvpp2]                   [k] mvpp2_bm_pool_put
   2.73%  [kernel]                  [k] __page_pool_put_page
   2.33%  [kernel]                  [k] __alloc_pages_nodemask
   2.33%  [kernel]                  [k] inet_gro_receive
   2.05%  [kernel]                  [k] __build_skb
   1.95%  [kernel]                  [k] build_skb
   1.89%  [cls_matchall]            [k] mall_classify
   1.83%  [kernel]                  [k] page_pool_alloc_pages
   1.80%  [kernel]                  [k] tcf_action_exec
   1.70%  [mvpp2]                   [k] mvpp2_buf_alloc.isra.0
   1.63%  [kernel]                  [k] free_unref_page_prepare.part.0
   1.45%  [kernel]                  [k] page_pool_return_skb_page
   1.42%  [act_gact]                [k] tcf_gact_act
   1.16%  [kernel]                  [k] netif_receive_skb_list_internal
   1.08%  [kernel]                  [k] kfree_skb
   1.07%  [kernel]                  [k] skb_release_data
Quoted 11 lines
Quoted 2 lines

Any idea on why I get such bad numbers?

Nop but it's indeed strange

Quoted 3 lines

Another reason to send it as RFC is that I'm not fully convinced on how to use the page_pool given the HW limitation of the BM.

I'll have a look right after holidays

Thanks

Quoted 8 lines
Quoted 5 lines

The driver currently uses, for every CPU, a page_pool for short packets and another for long ones. The driver also has 4 rx queue per port, so every RXQ #1 will share the short and long page pools of CPU #1.

I am not sure i am following the hardware config here

Never mind, it's quite a mess, I needed a lot of time to get it :)

The HW put the packets in different buffer pools depending on the size:
short: 64..128
long: 128..1664
jumbo: 1664..9856

Let's skip the jumbo buffer for now and assume we have 4 CPU, the driver allocates 4 short and 4 long buffers.
Each port has 4 RX queues, and each one uses a short and a long buffer. With the page_pool api, we have 8 struct page_pool, 4 for the short and 4 for the long buffers.

Quoted 14 lines
Quoted 14 lines

This means that for every RX queue I call xdp_rxq_info_reg_mem_model() twice, on two different page_pool, can this be a problem?

As usual, ideas are welcome.

Matteo Croce (2):
mvpp2: use page_pool allocator
mvpp2: memory accounting

 drivers/net/ethernet/marvell/Kconfig          |   1 +
 drivers/net/ethernet/marvell/mvpp2/mvpp2.h    |   7 +
 .../net/ethernet/marvell/mvpp2/mvpp2_main.c   | 142 +++++++++++++++---
 3 files changed, 125 insertions(+), 25 deletions(-)
Quoted 6 lines
Quoted 3 lines

--
2.24.1

Cheers
/Ilias

Bye,

Signature · 2 lines
Matteo Croce
per aspera ad upstream
#5 lore raw
Quoted 40 lines · On Tue, 24 Dec 2019 14:34:07 +0100 Matteo Croce <[email protected]> wrote:
Quoted 38 lines · Ilias Apalodimas
Quoted 9 lines · Matteo Croce

This patches change the memory allocator of mvpp2 from the frag allocator to the page_pool API. This change is needed to add later XDP support to mvpp2.

The reason I send it as RFC is that with this changeset, mvpp2 performs much more slower. This is the tc drop rate measured with a single flow:

stock net-next with frag allocator:
rx: 900.7 Mbps 1877 Kpps

Quoted 6 lines

this patchset with page_pool:
rx: 423.5 Mbps 882.3 Kpps

This is the perf top when receiving traffic:

27.68% [kernel] [k] __page_pool_clean_page

This seems extremly high on the list.

Quoted 18 lines
   9.79%  [kernel]            [k] get_page_from_freelist
   7.18%  [kernel]            [k] free_unref_page
   4.64%  [kernel]            [k] build_skb
   4.63%  [kernel]            [k] __netif_receive_skb_core
   3.83%  [mvpp2]             [k] mvpp2_poll
   3.64%  [kernel]            [k] eth_type_trans
   3.61%  [kernel]            [k] kmem_cache_free
   3.03%  [kernel]            [k] kmem_cache_alloc
   2.76%  [kernel]            [k] dev_gro_receive
   2.69%  [mvpp2]             [k] mvpp2_bm_pool_put
   2.68%  [kernel]            [k] page_frag_free
   1.83%  [kernel]            [k] inet_gro_receive
   1.74%  [kernel]            [k] page_pool_alloc_pages
   1.70%  [kernel]            [k] __build_skb
   1.47%  [kernel]            [k] __alloc_pages_nodemask
   1.36%  [mvpp2]             [k] mvpp2_buf_alloc.isra.0
   1.29%  [kernel]            [k] tcf_action_exec
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

I tried Ilias patches for page_pool recycling, I get an improvement to ~1100, but I'm still far than the original allocator.

Can you post the recycling perf for comparison?

  12.00%  [kernel]                  [k] get_page_from_freelist
   9.25%  [kernel]                  [k] free_unref_page

Hmm, this indicate pages are not getting recycled.

Quoted 5 lines
   6.83%  [kernel]                  [k] eth_type_trans
   5.33%  [kernel]                  [k] __netif_receive_skb_core
   4.96%  [mvpp2]                   [k] mvpp2_poll
   4.64%  [kernel]                  [k] kmem_cache_free
   4.06%  [kernel]                  [k] __xdp_return

You do invoke __xdp_return() code, but it might find that the page cannot be recycled...

Quoted 19 lines
   3.60%  [kernel]                  [k] kmem_cache_alloc
   3.31%  [kernel]                  [k] dev_gro_receive
   3.29%  [kernel]                  [k] __page_pool_clean_page
   3.25%  [mvpp2]                   [k] mvpp2_bm_pool_put
   2.73%  [kernel]                  [k] __page_pool_put_page
   2.33%  [kernel]                  [k] __alloc_pages_nodemask
   2.33%  [kernel]                  [k] inet_gro_receive
   2.05%  [kernel]                  [k] __build_skb
   1.95%  [kernel]                  [k] build_skb
   1.89%  [cls_matchall]            [k] mall_classify
   1.83%  [kernel]                  [k] page_pool_alloc_pages
   1.80%  [kernel]                  [k] tcf_action_exec
   1.70%  [mvpp2]                   [k] mvpp2_buf_alloc.isra.0
   1.63%  [kernel]                  [k] free_unref_page_prepare.part.0
   1.45%  [kernel]                  [k] page_pool_return_skb_page
   1.42%  [act_gact]                [k] tcf_gact_act
   1.16%  [kernel]                  [k] netif_receive_skb_list_internal
   1.08%  [kernel]                  [k] kfree_skb
   1.07%  [kernel]                  [k] skb_release_data
Signature · 4 lines
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer
#6 lore raw
In reply to Ilias Apalodimas
Quoted 10 lines · On Tue, 24 Dec 2019 11:52:29 +0200 Ilias Apalodimas…
Quoted 9 lines · Matteo Croce

This patches change the memory allocator of mvpp2 from the frag allocator to the page_pool API. This change is needed to add later XDP support to mvpp2.

The reason I send it as RFC is that with this changeset, mvpp2 performs much more slower. This is the tc drop rate measured with a single flow:

stock net-next with frag allocator:
rx: 900.7 Mbps 1877 Kpps

Quoted 8 lines
Quoted 6 lines

this patchset with page_pool:
rx: 423.5 Mbps 882.3 Kpps

This is the perf top when receiving traffic:

27.68% [kernel] [k] __page_pool_clean_page

This seems extremly high on the list.

This looks related to the cost of dma unmap, as page_pool have PP_FLAG_DMA_MAP. (It is a little strange, as page_pool have flag DMA_ATTR_SKIP_CPU_SYNC, which should make it less expensive).

Quoted 1 line
Quoted 1 line

9.79% [kernel] [k] get_page_from_freelist

You are clearly hitting page-allocator every time, because you are not using page_pool recycle facility.

Quoted 19 lines
Quoted 19 lines
   7.18%  [kernel]            [k] free_unref_page
   4.64%  [kernel]            [k] build_skb
   4.63%  [kernel]            [k] __netif_receive_skb_core
   3.83%  [mvpp2]             [k] mvpp2_poll
   3.64%  [kernel]            [k] eth_type_trans
   3.61%  [kernel]            [k] kmem_cache_free
   3.03%  [kernel]            [k] kmem_cache_alloc
   2.76%  [kernel]            [k] dev_gro_receive
   2.69%  [mvpp2]             [k] mvpp2_bm_pool_put
   2.68%  [kernel]            [k] page_frag_free
   1.83%  [kernel]            [k] inet_gro_receive
   1.74%  [kernel]            [k] page_pool_alloc_pages
   1.70%  [kernel]            [k] __build_skb
   1.47%  [kernel]            [k] __alloc_pages_nodemask
   1.36%  [mvpp2]             [k] mvpp2_buf_alloc.isra.0
   1.29%  [kernel]            [k] tcf_action_exec

I tried Ilias patches for page_pool recycling, I get an improvement to ~1100, but I'm still far than the original allocator.

Signature · 4 lines
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer
#7 lore raw
Quoted 51 lines · Jesper Dangaard Brouer
Quoted 10 lines · On Tue, 24 Dec 2019 11:52:29 +0200 Ilias Apalodimas…
Quoted 9 lines · Matteo Croce

This patches change the memory allocator of mvpp2 from the frag allocator to the page_pool API. This change is needed to add later XDP support to mvpp2.

The reason I send it as RFC is that with this changeset, mvpp2 performs much more slower. This is the tc drop rate measured with a single flow:

stock net-next with frag allocator:
rx: 900.7 Mbps 1877 Kpps

Quoted 8 lines
Quoted 6 lines

this patchset with page_pool:
rx: 423.5 Mbps 882.3 Kpps

This is the perf top when receiving traffic:

27.68% [kernel] [k] __page_pool_clean_page

This seems extremly high on the list.

This looks related to the cost of dma unmap, as page_pool have PP_FLAG_DMA_MAP. (It is a little strange, as page_pool have flag DMA_ATTR_SKIP_CPU_SYNC, which should make it less expensive).

Quoted 1 line
Quoted 1 line

9.79% [kernel] [k] get_page_from_freelist

You are clearly hitting page-allocator every time, because you are not using page_pool recycle facility.

Quoted 17 lines
Quoted 17 lines
   7.18%  [kernel]            [k] free_unref_page
   4.64%  [kernel]            [k] build_skb
   4.63%  [kernel]            [k] __netif_receive_skb_core
   3.83%  [mvpp2]             [k] mvpp2_poll
   3.64%  [kernel]            [k] eth_type_trans
   3.61%  [kernel]            [k] kmem_cache_free
   3.03%  [kernel]            [k] kmem_cache_alloc
   2.76%  [kernel]            [k] dev_gro_receive
   2.69%  [mvpp2]             [k] mvpp2_bm_pool_put
   2.68%  [kernel]            [k] page_frag_free
   1.83%  [kernel]            [k] inet_gro_receive
   1.74%  [kernel]            [k] page_pool_alloc_pages
   1.70%  [kernel]            [k] __build_skb
   1.47%  [kernel]            [k] __alloc_pages_nodemask
   1.36%  [mvpp2]             [k] mvpp2_buf_alloc.isra.0
   1.29%  [kernel]            [k] tcf_action_exec
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

I tried Ilias patches for page_pool recycling, I get an improvement to ~1100, but I'm still far than the original allocator.

--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
LinkedIn: http://www.linkedin.com/in/brouer

The change I did to use the recycling is the following:

drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +0 -0
@@ -3071,7 +3071,7 @@ static int mvpp2_rx(struct mvpp2_port *port,

struct napi_struct *napi,

    if (pp)
-       page_pool_release_page(pp, virt_to_page(data));
+      skb_mark_for_recycle(skb, virt_to_page(data), &rxq->xdp_rxq.mem);
    else
         dma_unmap_single_attrs(dev->dev.parent, dma_addr,

--
Matteo Croce
per aspera ad upstream

#8 lore raw
Quoted 67 lines · Matteo Croce
Quoted 51 lines · Jesper Dangaard Brouer
Quoted 10 lines · On Tue, 24 Dec 2019 11:52:29 +0200 Ilias Apalodimas…
Quoted 9 lines · Matteo Croce

This patches change the memory allocator of mvpp2 from the frag allocator to the page_pool API. This change is needed to add later XDP support to mvpp2.

The reason I send it as RFC is that with this changeset, mvpp2 performs much more slower. This is the tc drop rate measured with a single flow:

stock net-next with frag allocator:
rx: 900.7 Mbps 1877 Kpps

Quoted 8 lines
Quoted 6 lines

this patchset with page_pool:
rx: 423.5 Mbps 882.3 Kpps

This is the perf top when receiving traffic:

27.68% [kernel] [k] __page_pool_clean_page

This seems extremly high on the list.

This looks related to the cost of dma unmap, as page_pool have PP_FLAG_DMA_MAP. (It is a little strange, as page_pool have flag DMA_ATTR_SKIP_CPU_SYNC, which should make it less expensive).

Quoted 1 line
Quoted 1 line

9.79% [kernel] [k] get_page_from_freelist

You are clearly hitting page-allocator every time, because you are not using page_pool recycle facility.

Quoted 17 lines
Quoted 17 lines
   7.18%  [kernel]            [k] free_unref_page
   4.64%  [kernel]            [k] build_skb
   4.63%  [kernel]            [k] __netif_receive_skb_core
   3.83%  [mvpp2]             [k] mvpp2_poll
   3.64%  [kernel]            [k] eth_type_trans
   3.61%  [kernel]            [k] kmem_cache_free
   3.03%  [kernel]            [k] kmem_cache_alloc
   2.76%  [kernel]            [k] dev_gro_receive
   2.69%  [mvpp2]             [k] mvpp2_bm_pool_put
   2.68%  [kernel]            [k] page_frag_free
   1.83%  [kernel]            [k] inet_gro_receive
   1.74%  [kernel]            [k] page_pool_alloc_pages
   1.70%  [kernel]            [k] __build_skb
   1.47%  [kernel]            [k] __alloc_pages_nodemask
   1.36%  [mvpp2]             [k] mvpp2_buf_alloc.isra.0
   1.29%  [kernel]            [k] tcf_action_exec
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

I tried Ilias patches for page_pool recycling, I get an improvement to ~1100, but I'm still far than the original allocator.

--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
LinkedIn: http://www.linkedin.com/in/brouer

The change I did to use the recycling is the following:

drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c static int mvpp2_rx(struct mvpp2_port *port, · line 3077

Quoted 8 lines
drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +1 -1
@@ -3071,7 +3071,7 @@ static int mvpp2_rx(struct mvpp2_port *port,
 struct napi_struct *napi,
    if (pp)
-       page_pool_release_page(pp, virt_to_page(data));
+      skb_mark_for_recycle(skb, virt_to_page(data), &rxq->xdp_rxq.mem);
    else
         dma_unmap_single_attrs(dev->dev.parent, dma_addr,
 
 

Jesper is rightm you aren't recycling anything.

The mark skb_mark_for_recycle() usage seems correct.
There are a few more places that we refuse to recycle (for example coalescing page pool and slub allocated pages is forbidden). I wonder if you hit any of those cases and recycling doesn't take place.
We'll hopefully release updated code shortly. I'll ping you and we can test on that

Thanks
/Ilias

Quoted 6 lines

--
Matteo Croce
per aspera ad upstream