AF_PACKET PMD loops back packets on veth with tc
https://bugs.dpdk.org/show_bug.cgi?id=1592
Bug ID: 1592
Summary: AF_PACKET PMD loops back packets on veth with tc
Product: DPDK
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: Normal
Component: ethdev
Assignee: [email protected]
Reporter: [email protected]
Target Milestone: ---
Found using AF_PACKET PMD on veth with Linux TC via Mininet. Ubuntu 24.04.1 LTS.
High-level, I have a basic Mininet topology with h1 <-> r1 <-> h2. (Each link represents a pair of veths.) r1 is a transparent middlebox running the DPDK skeleton/basicfwd.c example, which should forward packets through r1 (eth0 -> eth1 and vice verse). I ping h1 <-> h2, expecting r1 to perform this bridging.
The DPDK basicfwd program was started with the vdev configuration:--vdev=eth_af_packet0,iface=r1-eth0,qdisc_bypass=0 --vdev=eth_af_packet1,iface=r1-eth1,qdisc_bypass=0 &.
(I turn off qdisc_bypass because I'd like to use TC.)
When one of r1's veths have TC params (e.g., delay), this all works fine.
However, when both of r1's veths (r1-eth0 and r1-eth1) have TC params set up, traffic is looped forever. Here's an example output from h1:
``` ... 64 bytes from 10.0.1.9: icmp_seq=1 ttl=64 time=161 ms (DUP!) 64 bytes from 10.0.1.9: icmp_seq=1 ttl=64 time=161 ms (DUP!) 64 bytes from 10.0.1.9: icmp_seq=1 ttl=64 time=161 ms (DUP!) 64 bytes from 10.0.1.9: icmp_seq=1 ttl=64 time=161 ms (DUP!) 64 bytes from 10.0.1.9: icmp_seq=1 ttl=64 time=161 ms (DUP!) 64 bytes from 10.0.1.9: icmp_seq=1 ttl=64 time=181 ms (DUP!) ... (continues perpetually until stopped) ```
Note that this issue only occurs if TC parameters (e.g., delay) are configured on both egress veths. If qdisc_bypass is disabled on both, but no actual TC parameters are set, the duplication does not happen.
I took packet captures (tcpdump), and I also printed packets that the application was actually seeing. It appears that when the middlebox writes a packet to a socket, it immediately reads the same packet out from the same socket.
I also tried making small modifications in the DPDK driver to investigate whether this could be a Linux or Mininet issue (e.g., packet mmap + TC?) vs. a DPDK driver issue. I reduced the # of RX/TX queues to 1, and I also turned of packet mmap (by removing the setsockopt calls in rte_eth_af_packet.c for PACKET_RX_RING and PACKET_TX_RING -- not sure if there's anything else I would need to do here?).
A basic python script for repro is below. This works fine when only one of the r1.cmd("tc qdisc add dev r1-ethX root netem delay 10ms") commands is present. When both are added, the pings begin to loop.
```python
from mininet.net import Mininet
from mininet.link import TCLink
from mininet.cli import CLI
net = Mininet(controller=None, link=TCLink)
# Add hosts in same subnet
h1 = net.addHost('h1', ip='10.0.1.10/24', mac='00:00:00:00:00:01')
h2 = net.addHost('h2', ip='10.0.1.9/24', mac='00:00:00:00:00:02')
r1 = net.addHost('r1')
net.addLink(r1, h1)
net.addLink(r1, h2)
net.build()
# Configure TC on egress
# (could also be done by adding `delay` param to `addLink` above)
h2.cmd("tc qdisc add dev h2-eth0 root netem delay 10ms")
h1.cmd("tc qdisc add dev h1-eth0 root netem delay 10ms")
r1.cmd("tc qdisc add dev r1-eth0 root netem delay 10ms")
r1.cmd("tc qdisc add dev r1-eth1 root netem delay 10ms")
r1.cmd('sudo /path/to/dpdk-24.07/examples/skeleton/build/basicfwd 0 1
--vdev=eth_af_packet0,iface=r1-eth0,qdisc_bypass=0
--vdev=eth_af_packet1,iface=r1-eth1,qdisc_bypass=0 &')
CLI(net)
net.stop()
```I think there is a bug here, though I haven't been able to figure out for sure what might be going on or confirm 100% whether the issue is in the PMD.
Signature · 2 lines
You are receiving this mail because: You are the assignee for the bug.
https://bugs.dpdk.org/show_bug.cgi?id=1592
Stephen Hemminger ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |INVALID
Status|UNCONFIRMED |RESOLVED
CC| |[email protected]--- Comment #1 from Stephen Hemminger ([email protected]) --- This is not a DPDK bug.
The Linux kernel will interpret the packets it receives.
By default Linux has a weak host model, and it will look at the L3 header.
If you want to forward you need to either use hardware DPDK network devices (not packet, tun, ...); or put each eth device into a different network or namespace.
The DPDK part of forwarding has nothing wrong here.
Signature · 2 lines
You are receiving this mail because: You are the assignee for the bug.