Skip to content

udp: avoid false sharing on receive

Paolo Abeni mptcp netdev 2 versions – Applied

Status from this message: This series was applied to netdev/net-next.git (master)

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest2 patches 5 repliesApplied 2/2
  2. v12 patches 3 repliesSuperseded

Changes in v2 from the cover letter

- introduce and use a common helper to initialize the UDP v4/v6 sockets (Kuniyuki)

Patches in v2 All 2 applied

#Subject and filesSizeRepliesApplied
1/2net: introduce and use custom sockopt socket flaginclude/linux/net.h net/mptcp/protocol.c net/socket.c+6 -72 repliesApplied
2/2udp: track the forward memory release threshold in an hot cachelineinclude/linux/udp.h include/net/udp.h net/ipv4/udp.c +1 more+29 -53 repliesApplied

Reviews trailers per patch

All 2 patches have a Reviewed-by or Acked-by.

PatchMatthieu BaertsEric DumazetKuniyuki Iwashima
1/2R-b R-b A-b
2/2R-b A-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v2 thread

Cover letter [PATCH net-next v2 0/2] udp: avoid false sharing on receive

Under high UDP load, the BH processing and the user-space receiver can run on different cores.

The UDP implementation does a lot of effort to avoid false sharing in the receive path, but recent changes to the struct sock layout moved the sk_forward_alloc and the sk_rcvbuf fields on the same cacheline:

        /* --- cacheline 4 boundary (256 bytes) --- */
                struct sk_buff *   tail;
        } sk_backlog;
        int                        sk_forward_alloc;
        unsigned int               sk_reserved_mem;
        unsigned int               sk_ll_usec;
        unsigned int               sk_napi_id;
        int                        sk_rcvbuf;

sk_forward_alloc is updated by the BH, while sk_rcvbuf is accessed by udp_recvmsg(), causing false sharing.

A possible solution would be to re-order the struct sock fields to avoid the false sharing. Such change is subject to being invalidated by future changes and could have negative side effects on other workload.

Instead this series uses a different approach, touching only the UDP socket layout.

The first patch generalizes the custom setsockopt infrastructure, to allow UDP tracking the buffer size, and the second patch addresses the issue, copying the relevant buffer information into an already hot cacheline.

Overall the above gives a 10% peek throughput increase under UDP flood.

v1 -> v2:
- introduce and use a common helper to initialize the UDP v4/v6 sockets (Kuniyuki)

Paolo Abeni (2):
net: introduce and use custom sockopt socket flag
udp: track the forward memory release threshold in an hot cacheline

 include/linux/net.h  |  1 +
 include/linux/udp.h  |  3 +++
 include/net/udp.h    |  9 +++++++++
 net/ipv4/udp.c       | 18 +++++++++++++++---
 net/ipv6/udp.c       |  4 ++--
 net/mptcp/protocol.c |  4 ++++
 net/socket.c         |  8 +-------
 7 files changed, 35 insertions(+), 12 deletions(-)
Signature · 1 line
2.37.3