PF_PACKET RPS like using bpf
Hello,
While implementing a large scale L2 processing soft, there is a need for high perf ingress handling.
In short: using BPF socket_filter prog with a hash func over ethernet address to distribute load across multiple PF_PACKET sockets: mainly filtering out non related hkey packets. It simplify userland app by using multiple PF_PACKET sockets in dedicated pthread.
Longer discussion + quick source code illustration here:
https://github.com/acassen/bpf-pfpacket-rps
My question would be related to sockmap. I tried to figure out how to perform the same policing design using sockmap. And cant find my way around it :/ If bpf verdict prog can get socket fd from __sk_buff then it could drives the same hash/policing decision.
Is it possible to get socket fd from __sk_buff from sockmap verdict prog ?
regs,
Alexandre
Quoted 15 lines · Alexandre Cassen
Hello,
While implementing a large scale L2 processing soft, there is a need for high perf ingress handling.
In short: using BPF
socket_filterprog with a hash func over ethernet address to distribute load across multiplePF_PACKETsockets: mainly filtering out non related hkey packets. It simplify userland app by using multiplePF_PACKETsockets in dedicated pthread.Longer discussion + quick source code illustration here:
Quoted 6 lines
My question would be related to sockmap. I tried to figure out how to perform the same policing design using sockmap. And cant find my way around it :/ If bpf verdict prog can get socket fd from
__sk_buffthen it could drives the same hash/policing decision.Is it possible to get socket fd from
__sk_bufffrom sockmap verdict prog ?
Don't have an answer about sockmap, but I'd suggest you look at AF_XDP if you want the best perf out there with raw frames.
IIRC, it just recently got a mode where you can redirect into multiple sockets (== consumer threads):
https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41
Quoted 21 lines · Stanislav Fomichev
Quoted 20 lines · Alexandre Cassen
Hello,
While implementing a large scale L2 processing soft, there is a need for high perf ingress handling.
In short: using BPF
socket_filterprog with a hash func over ethernet address to distribute load across multiplePF_PACKETsockets: mainly filtering out non related hkey packets. It simplify userland app by using multiplePF_PACKETsockets in dedicated pthread.Longer discussion + quick source code illustration here:
https://github.com/acassen/bpf-pfpacket-rps
My question would be related to sockmap. I tried to figure out how to perform the same policing design using sockmap. And cant find my way around it :/ If bpf verdict prog can get socket fd from
__sk_buffthen it could drives the same hash/policing decision.
Quoted 8 lines
Quoted 1 line
Is it possible to get socket fd from
__sk_bufffrom sockmap verdict prog ?Don't have an answer about sockmap, but I'd suggest you look at
AF_XDPif you want the best perf out there with raw frames.
IIRC, it just recently got a mode where you can redirect into multiple sockets (== consumer threads):https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41
thanks for replying, __sk_buff bpf mirror doesnt export struct sock as present into kernel skbuff, so no options but extending. AF_XDP is an option. Will extend test code with it later on.
Quoted 31 lines · Alexandre Cassen
Quoted 28 lines · Stanislav Fomichev
Quoted 15 lines · Alexandre Cassen
Hello,
While implementing a large scale L2 processing soft, there is a need for high perf ingress handling.
In short: using BPF
socket_filterprog with a hash func over ethernet address to distribute load across multiplePF_PACKETsockets: mainly filtering out non related hkey packets. It simplify userland app by using multiplePF_PACKETsockets in dedicated pthread.Longer discussion + quick source code illustration here:
Quoted 6 lines
My question would be related to sockmap. I tried to figure out how to perform the same policing design using sockmap. And cant find my way around it :/ If bpf verdict prog can get socket fd from
__sk_buffthen it could drives the same hash/policing decision.Is it possible to get socket fd from
__sk_bufffrom sockmap verdict prog ?Don't have an answer about sockmap, but I'd suggest you look at
AF_XDPif you want the best perf out there with raw frames.
IIRC, it just recently got a mode where you can redirect into multiple sockets (== consumer threads):
Quoted 5 lines
Quoted 1 line
https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41
thanks for replying,
__sk_buffbpf mirror doesnt export struct sock as present into kernel skbuff, so no options but extending.AF_XDPis an option. Will extend test code with it later on.
We do export bpf_sock in __sk_buff:
__bpf_md_ptr(struct bpf_sock *, sk);
But it has a very minimal set of 'struct sock' fields exported.
Yonghong Song
reviewerQuoted 31 lines · Stanislav Fomichev
Quoted 30 lines · Alexandre Cassen
Quoted 28 lines · Stanislav Fomichev
Quoted 15 lines · Alexandre Cassen
Hello,
While implementing a large scale L2 processing soft, there is a need for high perf ingress handling.
In short: using BPF
socket_filterprog with a hash func over ethernet address to distribute load across multiplePF_PACKETsockets: mainly filtering out non related hkey packets. It simplify userland app by using multiplePF_PACKETsockets in dedicated pthread.Longer discussion + quick source code illustration here:
Quoted 6 lines
My question would be related to sockmap. I tried to figure out how to perform the same policing design using sockmap. And cant find my way around it :/ If bpf verdict prog can get socket fd from
__sk_buffthen it could drives the same hash/policing decision.Is it possible to get socket fd from
__sk_bufffrom sockmap verdict prog ?Don't have an answer about sockmap, but I'd suggest you look at
AF_XDPif you want the best perf out there with raw frames.
IIRC, it just recently got a mode where you can redirect into multiple sockets (== consumer threads):https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41
Quoted 8 lines
Quoted 3 lines
thanks for replying,
__sk_buffbpf mirror doesnt export struct sock as present into kernel skbuff, so no options but extending.AF_XDPis an option. Will extend test code with it later on.We do export
bpf_sockin__sk_buff:__bpf_md_ptr(struct bpf_sock *, sk);But it has a very minimal set of 'struct sock' fields exported.
You could use bpf_core_cast() in bpf_core_read.h to cast 'struct bpf_sock' to kernel 'struct sock'.
Quoted 1 line
Quoted 36 lines · Yonghong Song
Quoted 34 lines · Stanislav Fomichev
Quoted 33 lines · Alexandre Cassen
Quoted 21 lines · Stanislav Fomichev
Quoted 20 lines · Alexandre Cassen
Hello,
While implementing a large scale L2 processing soft, there is a need for high perf ingress handling.
In short: using BPF
socket_filterprog with a hash func over ethernet address to distribute load across multiplePF_PACKETsockets: mainly filtering out non related hkey packets. It simplify userland app by using multiplePF_PACKETsockets in dedicated pthread.Longer discussion + quick source code illustration here:
https://github.com/acassen/bpf-pfpacket-rps
My question would be related to sockmap. I tried to figure out how to perform the same policing design using sockmap. And cant find my way around it :/ If bpf verdict prog can get socket fd from
__sk_buffthen it could drives the same hash/policing decision.Quoted 7 lines
Quoted 1 line
Is it possible to get socket fd from
__sk_bufffrom sockmap verdict prog ?Don't have an answer about sockmap, but I'd suggest you look at
AF_XDPif you want the best perf out there with raw frames.
IIRC, it just recently got a mode where you can redirect into multiple sockets (== consumer threads):https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41
thanks for replying,
__sk_buffbpf mirror doesnt export struct sock as present into kernel skbuff, so no options but extending.AF_XDPis an option. Will extend test code with it later on.
Quoted 7 lines
Quoted 5 lines
We do export
bpf_sockin__sk_buff:__bpf_md_ptr(struct
bpf_sock*, sk);But it has a very minimal set of 'struct sock' fields exported.
You could use
bpf_core_cast()inbpf_core_read.hto cast 'structbpf_sock' to kernel 'struct sock'.
good idea, other option could be to userland set SO_MARK one time and get it back from bpf prog via bpf_sock.mark to drive policing decision