packet: packet fanout rollover during socket overload
Subsystems (3)
PATCH v2 1/1 packet: packet fanout rollover during socket overload
Minimize packet drop in a fanout group. If one socket is full, roll over packets to another from the group. Maintain flow affinity during normal load using an rxhash fanout policy, while dispersing unexpected traffic storms that hit a single cpu, such as spoofed-source DoS flows. Rollover breaks affinity for flows arriving at saturated sockets during those conditions.
The patch adds a fanout policy ROLLOVER that rotates between sockets, filling each socket before moving to the next. It also adds a fanout flag ROLLOVER. If passed along with any other fanout policy, the primary policy is applied until the chosen socket is full. Then, rollover selects another socket, to delay packet drop until the entire system is saturated.
Probing sockets is not free. Selecting the last used socket, as rollover does, is a greedy approach that maximizes chance of success, at the cost of extreme load imbalance. In practice, with sufficiently long queues to absorb bursts, sockets are drained in parallel and load balance looks uniform in top.
To avoid contention, scales counters with number of sockets and accesses them lockfree. Values are bounds checked to ensure correctness.
Tested using an application with 9 threads pinned to CPUs, one socket per thread and sufficient busywork per packet operation to limits each thread to handling 32 Kpps. When sent 500 Kpps single UDP stream packets, a FANOUT_CPU setup processes 32 Kpps in total without this patch, 270 Kpps with the patch. Tested with read() and with a packet ring (V1).
--- include/linux/if_packet.h | 2 + net/packet/af_packet.c | 112 ++++++++++++++++++++++++++++++++++++---------- 2 files changed, 90 insertions(+), 24 deletions(-)
include/linux/if_packet.h +2 -0
@@ -54,6 +54,8 @@ struct sockaddr_ll {#define PACKET_FANOUT_HASH 0#define PACKET_FANOUT_LB 1#define PACKET_FANOUT_CPU 2 +#define PACKET_FANOUT_ROLLOVER 3 +#define PACKET_FANOUT_FLAG_ROLLOVER 0x1000#define PACKET_FANOUT_FLAG_DEFRAG 0x8000struct tpacket_stats {
net/packet/af_packet.c +88 -24
212 lines not on this page · raw message
Signature · 1 line
1.8.1.3
Quoted 31 lines · Willem de Bruijn
Minimize packet drop in a fanout group. If one socket is full, roll over packets to another from the group. Maintain flow affinity during normal load using an rxhash fanout policy, while dispersing unexpected traffic storms that hit a single cpu, such as spoofed-source DoS flows. Rollover breaks affinity for flows arriving at saturated sockets during those conditions.
The patch adds a fanout policy ROLLOVER that rotates between sockets, filling each socket before moving to the next. It also adds a fanout flag ROLLOVER. If passed along with any other fanout policy, the primary policy is applied until the chosen socket is full. Then, rollover selects another socket, to delay packet drop until the entire system is saturated.
Probing sockets is not free. Selecting the last used socket, as rollover does, is a greedy approach that maximizes chance of success, at the cost of extreme load imbalance. In practice, with sufficiently long queues to absorb bursts, sockets are drained in parallel and load balance looks uniform in
top.To avoid contention, scales counters with number of sockets and accesses them lockfree. Values are bounds checked to ensure correctness.
Tested using an application with 9 threads pinned to CPUs, one socket per thread and sufficient busywork per packet operation to limits each thread to handling 32 Kpps. When sent 500 Kpps single UDP stream packets, a
FANOUT_CPUsetup processes 32 Kpps in total without this patch, 270 Kpps with the patch. Tested withread()and with a packet ring (V1).
Quoted 2 lines
Signed-off-by Willem de Bruijn <[email protected]>---
Quoted 32 lines · Eric Dumazet
Quoted 31 lines · Willem de Bruijn
Minimize packet drop in a fanout group. If one socket is full, roll over packets to another from the group. Maintain flow affinity during normal load using an rxhash fanout policy, while dispersing unexpected traffic storms that hit a single cpu, such as spoofed-source DoS flows. Rollover breaks affinity for flows arriving at saturated sockets during those conditions.
The patch adds a fanout policy ROLLOVER that rotates between sockets, filling each socket before moving to the next. It also adds a fanout flag ROLLOVER. If passed along with any other fanout policy, the primary policy is applied until the chosen socket is full. Then, rollover selects another socket, to delay packet drop until the entire system is saturated.
Probing sockets is not free. Selecting the last used socket, as rollover does, is a greedy approach that maximizes chance of success, at the cost of extreme load imbalance. In practice, with sufficiently long queues to absorb bursts, sockets are drained in parallel and load balance looks uniform in
top.To avoid contention, scales counters with number of sockets and accesses them lockfree. Values are bounds checked to ensure correctness.
Tested using an application with 9 threads pinned to CPUs, one socket per thread and sufficient busywork per packet operation to limits each thread to handling 32 Kpps. When sent 500 Kpps single UDP stream packets, a
FANOUT_CPUsetup processes 32 Kpps in total without this patch, 270 Kpps with the patch. Tested withread()and with a packet ring (V1).
Quoted 4 lines
Quoted 2 lines
Signed-off-by Willem de Bruijn <[email protected]>---
Reviewed-by Eric Dumazet <[email protected]>
Thanks Eric.
I need to rebase the patch once more to have it apply cleanly.
Quoted 2 lines
Quoted 3 lines · Willem de Bruijn
Thanks Eric.
I need to rebase the patch once more to have it apply cleanly.
Oh right, include/linux/if_packet.h was moved
to include/uapi/linux/if_packet.h
Will resend after rebase + testing.
Quoted 8 lines · Eric Dumazet
Quoted 3 lines · Willem de Bruijn
Thanks Eric.
I need to rebase the patch once more to have it apply cleanly.
Oh right,
include/linux/if_packet.hwas moved
toinclude/uapi/linux/if_packet.h
Yes. I found another way to mess up a patch. Have rebased it already and it compiles fine, but I want to test it, just to be sure. Will send tomorrow with no other changes if nothing shows up.