XSK related fixes
Subsystems (5)
cover XSK related fixes
Cameron reported [0] that on fresh bpf-next he could not run multiple xdpsock instances in Tx-only mode on single network interface with i40e driver.
Turns out that Maxim's series [1] which was adding RCU protection around ndo_xsk_wakeup added check against the __I40E_CONFIG_BUSY being set on pf->state within i40e_xsk_wakeup() - if it's set, return -ENETDOWN. Since this bit is set per PF when UMEM is being enabled/disabled, the situation Cameron stumbled upon was that when he launched second xdpsock instance, second UMEM was being registered, hence set __I40E_CONFIG_BUSY which is now observed by first xdpsock and therefore xdpsock's kick_tx() gets -ENETDOWN as errno.
-ENETDOWN currently is not allowed in kick_tx(), so we were exiting the first application. Such exit means also XDP program being unloaded and its dedicated resources, which caused an -ENXIO being return in the second xdpsock instance.
Let's fix the issue from both sides - protect ourselves from future xdpsock crashes by allowing for -ENETDOWN errno being set in kick_tx() (patch 3) and from driver side, return -EAGAIN for the case where PF is busy (patch 1).
Remove also doubled variable from xdpsock_user.c (patch 2).
Note that ixgbe seems not to be affected since UMEM registration sets the busy/disable bit per ring, not per PF.
Thanks!
Maciej
[0]: https://www.spinics.net/lists/xdp-newbies/msg01558.html
[1]: https://lore.kernel.org/netdev/[email protected]/
Maciej Fijalkowski (3):
i40e: Relax i40e_xsk_wakeup's return value when PF is busy
samples: bpf: drop doubled variable declaration in xdpsock
samples: bpf: allow for -ENETDOWN in xdpsock
drivers/net/ethernet/intel/i40e/i40e_xsk.c | 2 +- samples/bpf/xdpsock_user.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
Signature · 1 line
2.20.1
PATCH 1/3 i40e: Relax i40e_xsk_wakeup's return value when PF is busy
Return -EAGAIN instead of -ENETDOWN to provide a slightly milder information to user space so that an application will know to retry the syscall when __I40E_CONFIG_BUSY bit is set on pf->state.
--- drivers/net/ethernet/intel/i40e/i40e_xsk.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/intel/i40e/i40e_xsk.c +1 -1
@@ -791,7 +791,7 @@ int i40e_xsk_wakeup(struct net_device *dev, u32 queue_id, u32 flags)struct i40e_ring *ring;if (test_bit (__I40E_CONFIG_BUSY, pf->state)) -return -ENETDOWN ; +return -EAGAIN ;if (test_bit (__I40E_VSI_DOWN, vsi->state))return -ENETDOWN ;
Signature · 1 line
2.20.1
PATCH 2/3 samples: bpf: drop doubled variable declaration in xdpsock
In reply to Maciej FijalkowskiSeems that by accident there is a doubled declaration of global variable opt_xdp_bind_flags in xdpsock_user.c. The second one is uninitialized so compiler was simply ignoring it.
To keep things clean, drop the doubled variable.
--- samples/bpf/xdpsock_user.c | 1 - 1 file changed, 1 deletion(-)
samples/bpf/xdpsock_user.c +0 -1
@@ -83,7 +83,6 @@ static u32 opt_xdp_bind_flags = XDP_USE_NEED_WAKEUP;static u32 opt_umem_flags;static int opt_unaligned_chunks;static int opt_mmap_flags; -static u32 opt_xdp_bind_flags;static int opt_xsk_frame_size =XSK_UMEM__DEFAULT_FRAME_SIZE ;static int opt_timeout =1000 ;static bool opt_need_wakeup = true;
Signature · 1 line
2.20.1
PATCH 3/3 samples: bpf: allow for -ENETDOWN in xdpsock
In reply to Maciej Fijalkowskindo_xsk_wakeup() can return -ENETDOWN and there's no particular reason to bail the whole application out on that case. Let's check in kick_tx() whether errno was set to mentioned value and basically allow application to further process frames.
--- samples/bpf/xdpsock_user.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
samples/bpf/xdpsock_user.c +2 -1
@@ -788,7 +788,8 @@ static void kick_tx(struct xsk_socket_info *xsk)int ret; ret =sendto (xsk_socket__fd (xsk->xsk),NULL ,0 ,MSG_DONTWAIT ,NULL ,0 ); -if (ret >=0 || errno ==ENOBUFS || errno ==EAGAIN || errno ==EBUSY ) +if (ret >=0 || errno ==ENOBUFS || errno ==EAGAIN || + errno ==EBUSY || errno ==ENETDOWN )return ;exit_with_error (errno); }
Signature · 1 line
2.20.1
Quoted 25 lines · Maciej Fijalkowski
Cameron reported [0] that on fresh bpf-next he could not run multiple xdpsock instances in Tx-only mode on single network interface with i40e driver.
Turns out that Maxim's series [1] which was adding RCU protection around
ndo_xsk_wakeupadded check against the__I40E_CONFIG_BUSYbeing set onpf->statewithini40e_xsk_wakeup()- if it's set, return-ENETDOWN. Since this bit is set per PF when UMEM is being enabled/disabled, the situation Cameron stumbled upon was that when he launched second xdpsock instance, second UMEM was being registered, hence set__I40E_CONFIG_BUSYwhich is now observed by first xdpsock and therefore xdpsock'skick_tx()gets-ENETDOWNas errno.
-ENETDOWNcurrently is not allowed inkick_tx(), so we were exiting the first application. Such exit means also XDP program being unloaded and its dedicated resources, which caused an-ENXIObeing return in the second xdpsock instance.Let's fix the issue from both sides - protect ourselves from future xdpsock crashes by allowing for
-ENETDOWNerrno being set inkick_tx()(patch 3) and from driver side, return-EAGAINfor the case where PF is busy (patch 1).Remove also doubled variable from
xdpsock_user.c(patch 2).
Quoted 8 lines
Note that ixgbe seems not to be affected since UMEM registration sets the busy/disable bit per ring, not per PF.
Thanks!
Maciej[0]: https://www.spinics.net/lists/xdp-newbies/msg01558.html
[1]: https://lore.kernel.org/netdev/[email protected]/
Applied, thanks!
Quoted 34 lines · Maciej Fijalkowski
Cameron reported [0] that on fresh bpf-next he could not run multiple xdpsock instances in Tx-only mode on single network interface with i40e driver.
Turns out that Maxim's series [1] which was adding RCU protection around
ndo_xsk_wakeupadded check against the__I40E_CONFIG_BUSYbeing set onpf->statewithini40e_xsk_wakeup()- if it's set, return-ENETDOWN. Since this bit is set per PF when UMEM is being enabled/disabled, the situation Cameron stumbled upon was that when he launched second xdpsock instance, second UMEM was being registered, hence set__I40E_CONFIG_BUSYwhich is now observed by first xdpsock and therefore xdpsock'skick_tx()gets-ENETDOWNas errno.
-ENETDOWNcurrently is not allowed inkick_tx(), so we were exiting the first application. Such exit means also XDP program being unloaded and its dedicated resources, which caused an-ENXIObeing return in the second xdpsock instance.Let's fix the issue from both sides - protect ourselves from future xdpsock crashes by allowing for
-ENETDOWNerrno being set inkick_tx()(patch 3) and from driver side, return-EAGAINfor the case where PF is busy (patch 1).Remove also doubled variable from
xdpsock_user.c(patch 2).Note that ixgbe seems not to be affected since UMEM registration sets the busy/disable bit per ring, not per PF.
Thanks!
Maciej[0]: https://www.spinics.net/lists/xdp-newbies/msg01558.html
[1]: https://lore.kernel.org/netdev/[email protected]/
Quoted 9 lines
Maciej Fijalkowski (3):
i40e: Relax i40e_xsk_wakeup's return value when PF is busy
samples: bpf: drop doubled variable declaration in xdpsock
samples: bpf: allow for-ENETDOWNin xdpsockdrivers/net/ethernet/intel/i40e/i40e_xsk.c | 2 +- samples/bpf/xdpsock_user.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
Though it's already merged (that was too fast).