Skip to contentSkip to conversation

XSK related fixes

PATCH Applied Maciej Fijalkowski 3 participants 6 messages Feb 5, 2020 – Feb 6, 2020

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover XSK related fixes

Cameron reported [0] that on fresh bpf-next he could not run multiple xdpsock instances in Tx-only mode on single network interface with i40e driver.

Turns out that Maxim's series [1] which was adding RCU protection around ndo_xsk_wakeup added check against the __I40E_CONFIG_BUSY being set on pf->state within i40e_xsk_wakeup() - if it's set, return -ENETDOWN. Since this bit is set per PF when UMEM is being enabled/disabled, the situation Cameron stumbled upon was that when he launched second xdpsock instance, second UMEM was being registered, hence set __I40E_CONFIG_BUSY which is now observed by first xdpsock and therefore xdpsock's kick_tx() gets -ENETDOWN as errno.

-ENETDOWN currently is not allowed in kick_tx(), so we were exiting the first application. Such exit means also XDP program being unloaded and its dedicated resources, which caused an -ENXIO being return in the second xdpsock instance.

Let's fix the issue from both sides - protect ourselves from future xdpsock crashes by allowing for -ENETDOWN errno being set in kick_tx() (patch 3) and from driver side, return -EAGAIN for the case where PF is busy (patch 1).

Remove also doubled variable from xdpsock_user.c (patch 2).

Note that ixgbe seems not to be affected since UMEM registration sets the busy/disable bit per ring, not per PF.

Thanks!
Maciej

[0]: https://www.spinics.net/lists/xdp-newbies/msg01558.html
[1]: https://lore.kernel.org/netdev/[email protected]/

Maciej Fijalkowski (3):
i40e: Relax i40e_xsk_wakeup's return value when PF is busy
samples: bpf: drop doubled variable declaration in xdpsock
samples: bpf: allow for -ENETDOWN in xdpsock

 drivers/net/ethernet/intel/i40e/i40e_xsk.c | 2 +-
 samples/bpf/xdpsock_user.c                 | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/3 i40e: Relax i40e_xsk_wakeup's return value when PF is busy

Return -EAGAIN instead of -ENETDOWN to provide a slightly milder information to user space so that an application will know to retry the syscall when __I40E_CONFIG_BUSY bit is set on pf->state.

Fixes b3873a5be757 ("net/i40e: Fix concurrency issues between config flow and XSK")
---
 drivers/net/ethernet/intel/i40e/i40e_xsk.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/intel/i40e/i40e_xsk.c +1 -1
index 42058fad6..0b7d29192 100644
@@ -791,7 +791,7 @@ int i40e_xsk_wakeup(struct net_device *dev, u32 queue_id, u32 flags)
 	struct i40e_ring *ring;
 
 	if (test_bit(__I40E_CONFIG_BUSY, pf->state))
-		return -ENETDOWN;
+		return -EAGAIN;
 
 	if (test_bit(__I40E_VSI_DOWN, vsi->state))
 		return -ENETDOWN;
Signature · 1 line
2.20.1
#2 lore raw

PATCH 2/3 samples: bpf: drop doubled variable declaration in xdpsock

In reply to Maciej Fijalkowski

Seems that by accident there is a doubled declaration of global variable opt_xdp_bind_flags in xdpsock_user.c. The second one is uninitialized so compiler was simply ignoring it.

To keep things clean, drop the doubled variable.

Fixes c543f5469822 ("samples/bpf: add unaligned chunks mode support to xdpsock")
---
 samples/bpf/xdpsock_user.c | 1 -
 1 file changed, 1 deletion(-)
samples/bpf/xdpsock_user.c +0 -1
index 0b5acd722..bab7a850e 100644
@@ -83,7 +83,6 @@ static u32 opt_xdp_bind_flags = XDP_USE_NEED_WAKEUP;
 static u32 opt_umem_flags;
 static int opt_unaligned_chunks;
 static int opt_mmap_flags;
-static u32 opt_xdp_bind_flags;
 static int opt_xsk_frame_size = XSK_UMEM__DEFAULT_FRAME_SIZE;
 static int opt_timeout = 1000;
 static bool opt_need_wakeup = true;
Signature · 1 line
2.20.1
#3 lore raw

PATCH 3/3 samples: bpf: allow for -ENETDOWN in xdpsock

In reply to Maciej Fijalkowski

ndo_xsk_wakeup() can return -ENETDOWN and there's no particular reason to bail the whole application out on that case. Let's check in kick_tx() whether errno was set to mentioned value and basically allow application to further process frames.

Fixes 248c7f9c0e21 ("samples/bpf: convert xdpsock to use libbpf for AF_XDP access")
---
 samples/bpf/xdpsock_user.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
samples/bpf/xdpsock_user.c +2 -1
index bab7a850e..c91e91362 100644
@@ -788,7 +788,8 @@ static void kick_tx(struct xsk_socket_info *xsk)
 	int ret;
 
 	ret = sendto(xsk_socket__fd(xsk->xsk), NULL, 0, MSG_DONTWAIT, NULL, 0);
-	if (ret >= 0 || errno == ENOBUFS || errno == EAGAIN || errno == EBUSY)
+	if (ret >= 0 || errno == ENOBUFS || errno == EAGAIN ||
+	    errno == EBUSY || errno == ENETDOWN)
 		return;
 	exit_with_error(errno);
 }
Signature · 1 line
2.20.1
maintainer Applied
#4 lore raw
In reply to Maciej Fijalkowski
Quoted 25 lines · Maciej Fijalkowski

Cameron reported [0] that on fresh bpf-next he could not run multiple xdpsock instances in Tx-only mode on single network interface with i40e driver.

Turns out that Maxim's series [1] which was adding RCU protection around ndo_xsk_wakeup added check against the __I40E_CONFIG_BUSY being set on pf->state within i40e_xsk_wakeup() - if it's set, return -ENETDOWN. Since this bit is set per PF when UMEM is being enabled/disabled, the situation Cameron stumbled upon was that when he launched second xdpsock instance, second UMEM was being registered, hence set __I40E_CONFIG_BUSY which is now observed by first xdpsock and therefore xdpsock's kick_tx() gets -ENETDOWN as errno.

-ENETDOWN currently is not allowed in kick_tx(), so we were exiting the first application. Such exit means also XDP program being unloaded and its dedicated resources, which caused an -ENXIO being return in the second xdpsock instance.

Let's fix the issue from both sides - protect ourselves from future xdpsock crashes by allowing for -ENETDOWN errno being set in kick_tx() (patch 3) and from driver side, return -EAGAIN for the case where PF is busy (patch 1).

Remove also doubled variable from xdpsock_user.c (patch 2).

Quoted 8 lines

Note that ixgbe seems not to be affected since UMEM registration sets the busy/disable bit per ring, not per PF.

Thanks!
Maciej

[0]: https://www.spinics.net/lists/xdp-newbies/msg01558.html
[1]: https://lore.kernel.org/netdev/[email protected]/

Applied, thanks!

Acked-by on the v1 series
#5 lore raw
In reply to Maciej Fijalkowski
Quoted 34 lines · Maciej Fijalkowski

Cameron reported [0] that on fresh bpf-next he could not run multiple xdpsock instances in Tx-only mode on single network interface with i40e driver.

Turns out that Maxim's series [1] which was adding RCU protection around ndo_xsk_wakeup added check against the __I40E_CONFIG_BUSY being set on pf->state within i40e_xsk_wakeup() - if it's set, return -ENETDOWN. Since this bit is set per PF when UMEM is being enabled/disabled, the situation Cameron stumbled upon was that when he launched second xdpsock instance, second UMEM was being registered, hence set __I40E_CONFIG_BUSY which is now observed by first xdpsock and therefore xdpsock's kick_tx() gets -ENETDOWN as errno.

-ENETDOWN currently is not allowed in kick_tx(), so we were exiting the first application. Such exit means also XDP program being unloaded and its dedicated resources, which caused an -ENXIO being return in the second xdpsock instance.

Let's fix the issue from both sides - protect ourselves from future xdpsock crashes by allowing for -ENETDOWN errno being set in kick_tx() (patch 3) and from driver side, return -EAGAIN for the case where PF is busy (patch 1).

Remove also doubled variable from xdpsock_user.c (patch 2).

Note that ixgbe seems not to be affected since UMEM registration sets the busy/disable bit per ring, not per PF.

Thanks!
Maciej

[0]: https://www.spinics.net/lists/xdp-newbies/msg01558.html
[1]: https://lore.kernel.org/netdev/[email protected]/

Quoted 9 lines

Maciej Fijalkowski (3):
i40e: Relax i40e_xsk_wakeup's return value when PF is busy
samples: bpf: drop doubled variable declaration in xdpsock
samples: bpf: allow for -ENETDOWN in xdpsock

  drivers/net/ethernet/intel/i40e/i40e_xsk.c | 2 +-
  samples/bpf/xdpsock_user.c                 | 4 ++--
  2 files changed, 3 insertions(+), 3 deletions(-)

Though it's already merged (that was too fast).