ixgbe: Update adaptive ITR algorithm
Subsystems (3)
PATCH 1/1 ixgbe: Update adaptive ITR algorithm
From: Alexander Duyck <[email protected]>
The following change is meant to update the adaptive ITR algorithm to better support the needs of the network. Specifically with this change what I have done is make it so that our ITR algorithm will try to prevent either starving a socket buffer for memory in the case of Tx, or overruing an Rx socket buffer on receive.
In addition a side effect of the calculations used is that we should function better with new features such as XDP which can handle small packets at high rates without needing to lock us into NAPI polling mode.
---
So I am putting this out to a wider distribution list than normal for a patch like this in order to get feedback on if there are any areas I may have overlooked. With this patch is should address many of the performance limitations seen with pktgen and XDP in terms of workloads that the old adaptive scheme wasn't handling.
drivers/net/ethernet/intel/ixgbe/ixgbe.h | 7 + drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c | 11 + drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 215 +++++++++++++++++++------ 3 files changed, 178 insertions(+), 55 deletions(-)
drivers/net/ethernet/intel/ixgbe/ixgbe.h +7 -0
@@ -435,8 +435,15 @@ static inline unsigned int ixgbe_rx_pg_order(struct ixgbe_ring *ring) }#define ixgbe_rx_pg_size (_ring) (PAGE_SIZE << ixgbe_rx_pg_order(_ring)) +#define IXGBE_ITR_ADAPTIVE_MIN_INC 2 +#define IXGBE_ITR_ADAPTIVE_MIN_USECS 10 +#define IXGBE_ITR_ADAPTIVE_MAX_USECS 126 +#define IXGBE_ITR_ADAPTIVE_LATENCY 0x80 +#define IXGBE_ITR_ADAPTIVE_BULK 0x00 +struct ixgbe_ring_container {struct ixgbe_ring *ring;/* pointer to linked list of rings */ +unsigned long next_update;/* jiffies value of last update */ unsigned int total_bytes;/* total bytes processed this int */ unsigned int total_packets;/* total packets processed this int */ u16 work_limit;/* total work allowed per interrupt */
drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c +9 -2
@@ -806,6 +806,7 @@ static void ixgbe_add_ring(struct ixgbe_ring *ring, ring->next = head->ring; head->ring = ring; head->count++; + head->next_update = jiffies +1 ; }/**
@@ -879,8 +880,11 @@ static int ixgbe_alloc_q_vector(struct ixgbe_adapter *adapter,/* initialize work limits */ q_vector->tx.work_limit = adapter->tx_work_limit; -/* initialize pointer to rings */ - ring = q_vector->ring; +/* Initialize setting for adaptive ITR */ + q_vector->tx.itr =IXGBE_ITR_ADAPTIVE_MAX_USECS | +IXGBE_ITR_ADAPTIVE_LATENCY ; + q_vector->rx.itr =IXGBE_ITR_ADAPTIVE_MAX_USECS | +IXGBE_ITR_ADAPTIVE_LATENCY ;/* intialize ITR */ if (txr_count && !rxr_count) {
@@ -897,6 +901,9 @@ static int ixgbe_alloc_q_vector(struct ixgbe_adapter *adapter, q_vector->itr = adapter->rx_itr_setting; } +/* initialize pointer to rings */ + ring = q_vector->ring; +while (txr_count) {/* assign generic ring traits */ ring->dev = &adapter->pdev->dev;
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +162 -53
246 lines not on this page · raw message
Quoted 12 lines · On Mon, 25 Sep 2017 14:55:36 -0700 Alexander Duyck <[email protected]>…
From: Alexander Duyck <[email protected]>
The following change is meant to update the adaptive ITR algorithm to better support the needs of the network. Specifically with this change what I have done is make it so that our ITR algorithm will try to prevent either starving a socket buffer for memory in the case of Tx, or overruing an Rx socket buffer on receive.
In addition a side effect of the calculations used is that we should function better with new features such as XDP which can handle small packets at high rates without needing to lock us into NAPI polling mode.
Quoted 8 lines
Signed-off-by Alexander Duyck <[email protected]>---
So I am putting this out to a wider distribution list than normal for a patch like this in order to get feedback on if there are any areas I may have overlooked. With this patch is should address many of the performance limitations seen with pktgen and XDP in terms of workloads that the old adaptive scheme wasn't handling.
Thanks a lot Alex!
I've tested the patch with XDP redirect (map), and the issue I reported in [1] is solved with this patch.
[1] Subject: "XDP redirect measurements, gotchas and tracepoints"
http://lkml.kernel.org/r/20170821212506.1cb0d5d6 at redhat.com
Signature · 4 lines
Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer
Bowers
PATCH 1/1 ixgbe: Update adaptive ITR algorithm
In reply to Alexander DuyckQuoted 30 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at osuosl.org] On Behalf Of Alexander Duyck
Sent: Monday, September 25, 2017 2:56 PM
To: netdev at vger.kernel.org; intel-wired-lan at lists.osuosl.org
Cc: brouer at redhat.com
Subject: [Intel-wired-lan] [jkirsher/next-queue PATCH] ixgbe: Update adaptive ITR algorithmFrom: Alexander Duyck <[email protected]>
The following change is meant to update the adaptive ITR algorithm to better support the needs of the network. Specifically with this change what I have done is make it so that our ITR algorithm will try to prevent either starving a socket buffer for memory in the case of Tx, or overruing an Rx socket buffer on receive.
In addition a side effect of the calculations used is that we should function better with new features such as XDP which can handle small packets at high rates without needing to lock us into NAPI polling mode.
Signed-off-by Alexander Duyck <[email protected]>---
So I am putting this out to a wider distribution list than normal for a patch like this in order to get feedback on if there are any areas I may have overlooked. With this patch is should address many of the performance limitations seen with pktgen and XDP in terms of workloads that the old adaptive scheme wasn't handling.
Quoted 5 lines
drivers/net/ethernet/intel/ixgbe/ixgbe.h | 7 + drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c | 11 + drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 215 +++++++++++++++++++------ 3 files changed, 178 insertions(+), 55 deletions(-)