sfc: reduce the number of requested xdp ev queues
Subsystems (4)
PATCH 1/1 sfc: reduce the number of requested xdp ev queues
Without this change the driver tries to allocate too many queues, breaching the number of available msi-x interrupts on machines with many logical cpus and default adapter settings:
Insufficient resources for 12 XDP event queues (24 other channels, max 32)
Which in turn triggers EINVAL on XDP processing:
sfc 0000:86:00.0 ext0: XDP TX failed (-22)
--- drivers/net/ethernet/sfc/efx_channels.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/net/ethernet/sfc/efx_channels.c +4 -2
@@ -17,6 +17,7 @@#include "rx_common.h" #include "nic.h" #include "sriov.h" +#include "workarounds.h" /* This is the first interrupt mode to try out of: * 0 => MSI-X
@@ -137,6 +138,7 @@ static int efx_allocate_msix_channels(struct efx_nic *efx, {unsigned int n_channels = parallelism;int vec_count; +int tx_per_ev;int n_xdp_tx;int n_xdp_ev;
@@ -149,9 +151,9 @@ static int efx_allocate_msix_channels(struct efx_nic *efx,* multiple tx queues, assuming tx and ev queues are both * maximum size. */ - + tx_per_ev =EFX_MAX_EVQ_SIZE /EFX_TXQ_MAX_ENT (efx); n_xdp_tx =num_possible_cpus (); - n_xdp_ev =DIV_ROUND_UP (n_xdp_tx,EFX_MAX_TXQ_PER_CHANNEL ); + n_xdp_ev =DIV_ROUND_UP (n_xdp_tx, tx_per_ev); vec_count =pci_msix_vec_count (efx->pci_dev);if (vec_count <0 )
Signature · 1 line
2.29.2
Quoted 11 lines · Ivan Babrou
Without this change the driver tries to allocate too many queues, breaching the number of available msi-x interrupts on machines with many logical cpus and default adapter settings:
Insufficient resources for 12 XDP event queues (24 other channels, max 32)
Which in turn triggers EINVAL on XDP processing:
sfc 0000:86:00.0 ext0: XDP TX failed (-22)
Signed-off-by Ivan Babrou <[email protected]>
Quoted 4 lines · On Wed, 20 Jan 2021 13:27:59 -0800 Ivan Babrou <[email protected]> wrote:
Without this change the driver tries to allocate too many queues, breaching the number of available msi-x interrupts on machines with many logical cpus and default adapter settings:
Quoted 8 lines
Insufficient resources for 12 XDP event queues (24 other channels, max 32)
Which in turn triggers EINVAL on XDP processing:
sfc 0000:86:00.0 ext0: XDP TX failed (-22)
Signed-off-by Ivan Babrou <[email protected]>---
I guess the patch is good in itself due to available msi-x interrupts.
Per earlier discussion: What will happen if a CPU with an ID higher than available XDP TX-queues redirect a packet out this driver?
Quoted 35 lines
drivers/net/ethernet/sfc/efx_channels.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)drivers/net/ethernet/sfc/efx_channels.c +4
-2@@ -17,6 +17,7 @@#include "rx_common.h" #include "nic.h" #include "sriov.h" +#include "workarounds.h" /* This is the first interrupt mode to try out of: * 0 => MSI-X @@ -137,6 +138,7 @@ static int efx_allocate_msix_channels(struct efx_nic *efx, {unsigned int n_channels = parallelism;int vec_count; +int tx_per_ev;int n_xdp_tx;int n_xdp_ev;@@ -149,9 +151,9 @@ static int efx_allocate_msix_channels(struct efx_nic *efx,* multiple tx queues, assuming tx and ev queues are both * maximum size. */ - + tx_per_ev =EFX_MAX_EVQ_SIZE /EFX_TXQ_MAX_ENT (efx); n_xdp_tx =num_possible_cpus (); - n_xdp_ev =DIV_ROUND_UP (n_xdp_tx,EFX_MAX_TXQ_PER_CHANNEL ); + n_xdp_ev =DIV_ROUND_UP (n_xdp_tx, tx_per_ev); vec_count =pci_msix_vec_count (efx->pci_dev);if (vec_count <0 )
Signature · 4 lines
Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer
Quoted 7 lines · Jesper Dangaard Brouer
Quoted 4 lines · On Wed, 20 Jan 2021 13:27:59 -0800 Ivan Babrou <[email protected]> wrote:
Without this change the driver tries to allocate too many queues, breaching the number of available msi-x interrupts on machines with many logical cpus and default adapter settings:
Quoted 5 lines
Quoted 5 lines
Insufficient resources for 12 XDP event queues (24 other channels, max 32)
Which in turn triggers EINVAL on XDP processing:
sfc 0000:86:00.0 ext0: XDP TX failed (-22)
Please mention in commit message *how* you are addressing/fixing this issue.
Quoted 8 lines
Quoted 3 lines
Signed-off-by Ivan Babrou <[email protected]>---
I guess the patch is good in itself due to available msi-x interrupts.
Per earlier discussion: What will happen if a CPU with an ID higher than available XDP TX-queues redirect a packet out this driver?
+1 on that question
Quoted 46 lines
Quoted 37 lines
drivers/net/ethernet/sfc/efx_channels.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)drivers/net/ethernet/sfc/efx_channels.c +4
-2@@ -17,6 +17,7 @@#include "rx_common.h" #include "nic.h" #include "sriov.h" +#include "workarounds.h" /* This is the first interrupt mode to try out of: * 0 => MSI-X @@ -137,6 +138,7 @@ static int efx_allocate_msix_channels(struct efx_nic *efx, {unsigned int n_channels = parallelism;int vec_count; +int tx_per_ev;int n_xdp_tx;int n_xdp_ev;@@ -149,9 +151,9 @@ static int efx_allocate_msix_channels(struct efx_nic *efx,* multiple tx queues, assuming tx and ev queues are both * maximum size. */ - + tx_per_ev =EFX_MAX_EVQ_SIZE /EFX_TXQ_MAX_ENT (efx); n_xdp_tx =num_possible_cpus (); - n_xdp_ev =DIV_ROUND_UP (n_xdp_tx,EFX_MAX_TXQ_PER_CHANNEL ); + n_xdp_ev =DIV_ROUND_UP (n_xdp_tx, tx_per_ev); vec_count =pci_msix_vec_count (efx->pci_dev);if (vec_count <0 )Signature · 4 lines
Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer
This patch was applied to netdev/net-next.git (refs/heads/master):
Hello:
This patch was applied to netdev/net-next.git (refs/heads/master):
Quoted 9 lines · On Wed, 20 Jan 2021 13:27:59 -0800 you wrote:
Without this change the driver tries to allocate too many queues, breaching the number of available msi-x interrupts on machines with many logical cpus and default adapter settings:
Insufficient resources for 12 XDP event queues (24 other channels, max 32)
Which in turn triggers EINVAL on XDP processing:
[...]
Here is the summary with links:
- [net-next] sfc: reduce the number of requested xdp ev queues
https://git.kernel.org/netdev/net-next/c/e26ca4b53582
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
Quoted 11 lines · Ivan Babrou
Without this change the driver tries to allocate too many queues, breaching the number of available msi-x interrupts on machines with many logical cpus and default adapter settings:
Insufficient resources for 12 XDP event queues (24 other channels, max 32)
Which in turn triggers EINVAL on XDP processing:
sfc 0000:86:00.0 ext0: XDP TX failed (-22)
Signed-off-by Ivan Babrou <[email protected]>
Quoted 38 lines
--- drivers/net/ethernet/sfc/efx_channels.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)drivers/net/ethernet/sfc/efx_channels.c +4
-2@@ -17,6 +17,7 @@#include "rx_common.h" #include "nic.h" #include "sriov.h" +#include "workarounds.h" /* This is the first interrupt mode to try out of: * 0 => MSI-X @@ -137,6 +138,7 @@ static int efx_allocate_msix_channels(struct efx_nic *efx, {unsigned int n_channels = parallelism;int vec_count; +int tx_per_ev;int n_xdp_tx;int n_xdp_ev;@@ -149,9 +151,9 @@ static int efx_allocate_msix_channels(struct efx_nic *efx,* multiple tx queues, assuming tx and ev queues are both * maximum size. */ - + tx_per_ev =EFX_MAX_EVQ_SIZE /EFX_TXQ_MAX_ENT (efx); n_xdp_tx =num_possible_cpus (); - n_xdp_ev =DIV_ROUND_UP (n_xdp_tx,EFX_MAX_TXQ_PER_CHANNEL ); + n_xdp_ev =DIV_ROUND_UP (n_xdp_tx, tx_per_ev); vec_count =pci_msix_vec_count (efx->pci_dev);if (vec_count <0 )Signature · 1 line
2.29.2