Skip to contentSkip to conversation

PF_PACKET RPS like using bpf

DISCUSSION Alexandre Cassen 3 participants 6 messages Mar 14, 2024 – Mar 18, 2024 BPF

lore
Share & tools
lore
#0 lore raw

Hello,

While implementing a large scale L2 processing soft, there is a need for high perf ingress handling.

In short: using BPF socket_filter prog with a hash func over ethernet address to distribute load across multiple PF_PACKET sockets: mainly filtering out non related hkey packets. It simplify userland app by using multiple PF_PACKET sockets in dedicated pthread.

Longer discussion + quick source code illustration here:

https://github.com/acassen/bpf-pfpacket-rps

My question would be related to sockmap. I tried to figure out how to perform the same policing design using sockmap. And cant find my way around it :/ If bpf verdict prog can get socket fd from __sk_buff then it could drives the same hash/policing decision.

Is it possible to get socket fd from __sk_buff from sockmap verdict prog ?

regs,
Alexandre

#1 lore raw
Quoted 15 lines · Alexandre Cassen

Hello,

While implementing a large scale L2 processing soft, there is a need for high perf ingress handling.

In short: using BPF socket_filter prog with a hash func over ethernet address to distribute load across multiple PF_PACKET sockets: mainly filtering out non related hkey packets. It simplify userland app by using multiple PF_PACKET sockets in dedicated pthread.

Longer discussion + quick source code illustration here:

https://github.com/acassen/bpf-pfpacket-rps

Quoted 6 lines

My question would be related to sockmap. I tried to figure out how to perform the same policing design using sockmap. And cant find my way around it :/ If bpf verdict prog can get socket fd from __sk_buff then it could drives the same hash/policing decision.

Is it possible to get socket fd from __sk_buff from sockmap verdict prog ?

Don't have an answer about sockmap, but I'd suggest you look at AF_XDP if you want the best perf out there with raw frames.
IIRC, it just recently got a mode where you can redirect into multiple sockets (== consumer threads):

https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41

#2 lore raw
Quoted 21 lines · Stanislav Fomichev
Quoted 20 lines · Alexandre Cassen

Hello,

While implementing a large scale L2 processing soft, there is a need for high perf ingress handling.

In short: using BPF socket_filter prog with a hash func over ethernet address to distribute load across multiple PF_PACKET sockets: mainly filtering out non related hkey packets. It simplify userland app by using multiple PF_PACKET sockets in dedicated pthread.

Longer discussion + quick source code illustration here:

https://github.com/acassen/bpf-pfpacket-rps

My question would be related to sockmap. I tried to figure out how to perform the same policing design using sockmap. And cant find my way around it :/ If bpf verdict prog can get socket fd from __sk_buff then it could drives the same hash/policing decision.

Quoted 8 lines
Quoted 1 line

Is it possible to get socket fd from __sk_buff from sockmap verdict prog ?

Don't have an answer about sockmap, but I'd suggest you look at AF_XDP if you want the best perf out there with raw frames.
IIRC, it just recently got a mode where you can redirect into multiple sockets (== consumer threads):

https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41

thanks for replying, __sk_buff bpf mirror doesnt export struct sock as present into kernel skbuff, so no options but extending. AF_XDP is an option. Will extend test code with it later on.

#3 lore raw
Quoted 31 lines · Alexandre Cassen
Quoted 28 lines · Stanislav Fomichev
Quoted 15 lines · Alexandre Cassen

Hello,

While implementing a large scale L2 processing soft, there is a need for high perf ingress handling.

In short: using BPF socket_filter prog with a hash func over ethernet address to distribute load across multiple PF_PACKET sockets: mainly filtering out non related hkey packets. It simplify userland app by using multiple PF_PACKET sockets in dedicated pthread.

Longer discussion + quick source code illustration here:

https://github.com/acassen/bpf-pfpacket-rps

Quoted 6 lines

My question would be related to sockmap. I tried to figure out how to perform the same policing design using sockmap. And cant find my way around it :/ If bpf verdict prog can get socket fd from __sk_buff then it could drives the same hash/policing decision.

Is it possible to get socket fd from __sk_buff from sockmap verdict prog ?

Don't have an answer about sockmap, but I'd suggest you look at AF_XDP if you want the best perf out there with raw frames.
IIRC, it just recently got a mode where you can redirect into multiple sockets (== consumer threads):

Quoted 5 lines
Quoted 1 line

https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41

thanks for replying, __sk_buff bpf mirror doesnt export struct sock as present into kernel skbuff, so no options but extending. AF_XDP is an option. Will extend test code with it later on.

We do export bpf_sock in __sk_buff:

	__bpf_md_ptr(struct bpf_sock *, sk);

But it has a very minimal set of 'struct sock' fields exported.

reviewer
#4 lore raw
Quoted 31 lines · Stanislav Fomichev
Quoted 30 lines · Alexandre Cassen
Quoted 28 lines · Stanislav Fomichev
Quoted 15 lines · Alexandre Cassen

Hello,

While implementing a large scale L2 processing soft, there is a need for high perf ingress handling.

In short: using BPF socket_filter prog with a hash func over ethernet address to distribute load across multiple PF_PACKET sockets: mainly filtering out non related hkey packets. It simplify userland app by using multiple PF_PACKET sockets in dedicated pthread.

Longer discussion + quick source code illustration here:

https://github.com/acassen/bpf-pfpacket-rps

Quoted 6 lines

My question would be related to sockmap. I tried to figure out how to perform the same policing design using sockmap. And cant find my way around it :/ If bpf verdict prog can get socket fd from __sk_buff then it could drives the same hash/policing decision.

Is it possible to get socket fd from __sk_buff from sockmap verdict prog ?

Don't have an answer about sockmap, but I'd suggest you look at AF_XDP if you want the best perf out there with raw frames.
IIRC, it just recently got a mode where you can redirect into multiple sockets (== consumer threads):

https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41

Quoted 8 lines
Quoted 3 lines

thanks for replying, __sk_buff bpf mirror doesnt export struct sock as present into kernel skbuff, so no options but extending. AF_XDP is an option. Will extend test code with it later on.

We do export bpf_sock in __sk_buff:

	__bpf_md_ptr(struct bpf_sock *, sk);

But it has a very minimal set of 'struct sock' fields exported.

You could use bpf_core_cast() in bpf_core_read.h to cast 'struct bpf_sock' to kernel 'struct sock'.

Quoted 1 line
#5 lore raw
Quoted 36 lines · Yonghong Song
Quoted 34 lines · Stanislav Fomichev
Quoted 33 lines · Alexandre Cassen
Quoted 21 lines · Stanislav Fomichev
Quoted 20 lines · Alexandre Cassen

Hello,

While implementing a large scale L2 processing soft, there is a need for high perf ingress handling.

In short: using BPF socket_filter prog with a hash func over ethernet address to distribute load across multiple PF_PACKET sockets: mainly filtering out non related hkey packets. It simplify userland app by using multiple PF_PACKET sockets in dedicated pthread.

Longer discussion + quick source code illustration here:

https://github.com/acassen/bpf-pfpacket-rps

My question would be related to sockmap. I tried to figure out how to perform the same policing design using sockmap. And cant find my way around it :/ If bpf verdict prog can get socket fd from __sk_buff then it could drives the same hash/policing decision.

Quoted 7 lines
Quoted 1 line

Is it possible to get socket fd from __sk_buff from sockmap verdict prog ?

Don't have an answer about sockmap, but I'd suggest you look at AF_XDP if you want the best perf out there with raw frames.
IIRC, it just recently got a mode where you can redirect into multiple sockets (== consumer threads):

https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41

thanks for replying, __sk_buff bpf mirror doesnt export struct sock as present into kernel skbuff, so no options but extending. AF_XDP is an option. Will extend test code with it later on.

Quoted 7 lines
Quoted 5 lines

We do export bpf_sock in __sk_buff:

    __bpf_md_ptr(struct bpf_sock *, sk);

But it has a very minimal set of 'struct sock' fields exported.

You could use bpf_core_cast() in bpf_core_read.h to cast 'struct bpf_sock' to kernel 'struct sock'.

good idea, other option could be to userland set SO_MARK one time and get it back from bpf prog via bpf_sock.mark to drive policing decision