vmlinux.h overlap/conflict with network protocol definitions
Subsystems (2)
I am fixing the use of TC BPF in DPDK to use libbpf and bpftool. Luca recommended using vmlinux.h to address possible build and CO:RE issues. But it won't work.
There are missing pieces such as definitions of IPV6 next header fields (in linux/ipv6.h) and TC actions.
Without major hack surgery, not possible to use vmlinux.h instead. Since vmlinux.h defines may things that overlap with other headers.
Using:
$ /usr/sbin/bpftool -V bpftool v7.3.0 using libbpf v1.3 features:
$ uname -r 6.6.15-amd64
The change to BPF program to use vmlinux is:
drivers/net/tap/bpf/tap_rss.c +1 -6
@@ -2,12 +2,7 @@* Copyright 2017 Mellanox Technologies, Ltd */ -#include <linux/in.h> -#include <linux/if_ether.h> -#include <linux/ip.h> -#include <linux/ipv6.h> -#include <linux/pkt_cls.h> -#include <linux/bpf.h> +#include "vmlinux.h" #include <bpf/bpf_helpers.h> #include <bpf/bpf_endian.h>
Resulting build failure is:
~/DPDK/tap $ ninja -C build
ninja: Entering directory `build'
[1/33] Generating drivers/net/tap/bpf/tap_rss.bpf.o with a custom command
FAILED: drivers/net/tap/bpf/tap_rss.o/usr/bin/clang -O2 -Wall -Wextra -DTAP_MAX_QUEUES=16 -target bpf -g -c -idirafter /usr/include -idirafter /usr/include/x86_64-linux-gnu ../drivers/net/tap/bpf/tap_rss.c -o drivers/net/tap/bpf/tap_rss.o ../drivers/net/tap/bpf/tap_rss.c:116:8: error: use of undeclared identifier 'IPPROTO_HOPOPTS'
case IPPROTO_HOPOPTS:
^
../drivers/net/tap/bpf/tap_rss.c:117:8: error: use of undeclared identifier 'IPPROTO_ROUTING'
case IPPROTO_ROUTING:
^
../drivers/net/tap/bpf/tap_rss.c:118:8: error: use of undeclared identifier 'IPPROTO_DSTOPTS'
case IPPROTO_DSTOPTS:
^
../drivers/net/tap/bpf/tap_rss.c:126:8: error: use of undeclared identifier 'IPPROTO_FRAGMENT'
case IPPROTO_FRAGMENT:
^
../drivers/net/tap/bpf/tap_rss.c:210:33: error: use of undeclared identifier 'ETH_P_IP'
if (skb->protocol == bpf_htons(ETH_P_IP))
^../drivers/net/tap/bpf/tap_rss.c:210:33: error: use of undeclared identifier 'ETH_P_IP' ../drivers/net/tap/bpf/tap_rss.c:210:33: error: use of undeclared identifier 'ETH_P_IP' ../drivers/net/tap/bpf/tap_rss.c:210:33: error: use of undeclared identifier 'ETH_P_IP' ../drivers/net/tap/bpf/tap_rss.c:212:38: error: use of undeclared identifier 'ETH_P_IPV6'
else if (skb->protocol == bpf_htons(ETH_P_IPV6))
^../drivers/net/tap/bpf/tap_rss.c:212:38: error: use of undeclared identifier 'ETH_P_IPV6' ../drivers/net/tap/bpf/tap_rss.c:212:38: error: use of undeclared identifier 'ETH_P_IPV6' ../drivers/net/tap/bpf/tap_rss.c:212:38: error: use of undeclared identifier 'ETH_P_IPV6' ../drivers/net/tap/bpf/tap_rss.c:248:10: error: use of undeclared identifier 'TC_ACT_OK'
return TC_ACT_OK;
^
../drivers/net/tap/bpf/tap_rss.c:252:10: error: use of undeclared identifier 'TC_ACT_OK'
return TC_ACT_OK;
^
../drivers/net/tap/bpf/tap_rss.c:256:9: error: use of undeclared identifier 'TC_ACT_PIPE'
return TC_ACT_PIPE;
^15 errors generated.
ninja: build stopped: subcommand failed.
Yonghong Song
reviewerQuoted 82 lines · Stephen Hemminger
I am fixing the use of TC BPF in DPDK to use libbpf and bpftool. Luca recommended using
vmlinux.hto address possible build and CO:RE issues. But it won't work.There are missing pieces such as definitions of IPV6 next header fields (in
linux/ipv6.h) and TC actions.Without major hack surgery, not possible to use
vmlinux.hinstead. Sincevmlinux.hdefines may things that overlap with other headers.Using:
$ /usr/sbin/bpftool -V bpftool v7.3.0 using libbpf v1.3 features:$ uname -r 6.6.15-amd64The change to BPF program to use vmlinux is:
drivers/net/tap/bpf/tap_rss.c +1
-6@@ -2,12 +2,7 @@* Copyright 2017 Mellanox Technologies, Ltd */ -#include <linux/in.h> -#include <linux/if_ether.h> -#include <linux/ip.h> -#include <linux/ipv6.h> -#include <linux/pkt_cls.h> -#include <linux/bpf.h> +#include "vmlinux.h" #include <bpf/bpf_helpers.h> #include <bpf/bpf_endian.h> Resulting build failure is:
~/DPDK/tap $ ninja -C build
ninja: Entering directory `build'
[1/33] Generatingdrivers/net/tap/bpf/tap_rss.bpf.owith a custom command
FAILED:drivers/net/tap/bpf/tap_rss.o/usr/bin/clang-O2 -Wall -Wextra -DTAP_MAX_QUEUES=16 -target bpf -g -c -idirafter/usr/include-idirafter/usr/include/x86_64-linux-gnu../drivers/net/tap/bpf/tap_rss.c-odrivers/net/tap/bpf/tap_rss.o../drivers/net/tap/bpf/tap_rss.c:116:8: error: use of undeclared identifier 'IPPROTO_HOPOPTS'case IPPROTO_HOPOPTS: ^ ../drivers/net/tap/bpf/tap_rss.c:117:8: error: use of undeclared identifier 'IPPROTO_ROUTING' case IPPROTO_ROUTING: ^ ../drivers/net/tap/bpf/tap_rss.c:118:8: error: use of undeclared identifier 'IPPROTO_DSTOPTS' case IPPROTO_DSTOPTS: ^ ../drivers/net/tap/bpf/tap_rss.c:126:8: error: use of undeclared identifier 'IPPROTO_FRAGMENT' case IPPROTO_FRAGMENT: ^ ../drivers/net/tap/bpf/tap_rss.c:210:33: error: use of undeclared identifier 'ETH_P_IP' if (skb->protocol == bpf_htons(ETH_P_IP)) ^
../drivers/net/tap/bpf/tap_rss.c:210:33: error: use of undeclared identifier 'ETH_P_IP'../drivers/net/tap/bpf/tap_rss.c:210:33: error: use of undeclared identifier 'ETH_P_IP'../drivers/net/tap/bpf/tap_rss.c:210:33: error: use of undeclared identifier 'ETH_P_IP'../drivers/net/tap/bpf/tap_rss.c:212:38: error: use of undeclared identifier 'ETH_P_IPV6'else if (skb->protocol == bpf_htons(ETH_P_IPV6)) ^
../drivers/net/tap/bpf/tap_rss.c:212:38: error: use of undeclared identifier 'ETH_P_IPV6'../drivers/net/tap/bpf/tap_rss.c:212:38: error: use of undeclared identifier 'ETH_P_IPV6'../drivers/net/tap/bpf/tap_rss.c:212:38: error: use of undeclared identifier 'ETH_P_IPV6'../drivers/net/tap/bpf/tap_rss.c:248:10: error: use of undeclared identifier 'TC_ACT_OK'return TC_ACT_OK; ^ ../drivers/net/tap/bpf/tap_rss.c:252:10: error: use of undeclared identifier 'TC_ACT_OK' return TC_ACT_OK; ^ ../drivers/net/tap/bpf/tap_rss.c:256:9: error: use of undeclared identifier 'TC_ACT_PIPE' return TC_ACT_PIPE; ^
Quoted 2 lines
15 errors generated.
ninja: build stopped: subcommand failed.
This is a known issue as currently vmlinux.h does not support macros. There are some efforts by Edward Zingerman to support this but this has not done yet. At the same time, you could have a trivial header file like https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/bpf_tracing_net.h to be used for bpf program and then your bpf program with vmlinux.h can have much easier CORE support.
Quoted 86 lines · On Thu, 4 Apr 2024 11:16:32 -0700 Yonghong Song <[email protected]> wrote:
Quoted 82 lines · Stephen Hemminger
I am fixing the use of TC BPF in DPDK to use libbpf and bpftool. Luca recommended using
vmlinux.hto address possible build and CO:RE issues. But it won't work.There are missing pieces such as definitions of IPV6 next header fields (in
linux/ipv6.h) and TC actions.Without major hack surgery, not possible to use
vmlinux.hinstead. Sincevmlinux.hdefines may things that overlap with other headers.Using:
$ /usr/sbin/bpftool -V bpftool v7.3.0 using libbpf v1.3 features:$ uname -r 6.6.15-amd64The change to BPF program to use vmlinux is:
drivers/net/tap/bpf/tap_rss.c +1
-6@@ -2,12 +2,7 @@* Copyright 2017 Mellanox Technologies, Ltd */ -#include <linux/in.h> -#include <linux/if_ether.h> -#include <linux/ip.h> -#include <linux/ipv6.h> -#include <linux/pkt_cls.h> -#include <linux/bpf.h> +#include "vmlinux.h" #include <bpf/bpf_helpers.h> #include <bpf/bpf_endian.h> Resulting build failure is:
~/DPDK/tap $ ninja -C build
ninja: Entering directory `build'
[1/33] Generatingdrivers/net/tap/bpf/tap_rss.bpf.owith a custom command
FAILED:drivers/net/tap/bpf/tap_rss.o/usr/bin/clang-O2 -Wall -Wextra -DTAP_MAX_QUEUES=16 -target bpf -g -c -idirafter/usr/include-idirafter/usr/include/x86_64-linux-gnu../drivers/net/tap/bpf/tap_rss.c-odrivers/net/tap/bpf/tap_rss.o../drivers/net/tap/bpf/tap_rss.c:116:8: error: use of undeclared identifier 'IPPROTO_HOPOPTS'case IPPROTO_HOPOPTS: ^ ../drivers/net/tap/bpf/tap_rss.c:117:8: error: use of undeclared identifier 'IPPROTO_ROUTING' case IPPROTO_ROUTING: ^ ../drivers/net/tap/bpf/tap_rss.c:118:8: error: use of undeclared identifier 'IPPROTO_DSTOPTS' case IPPROTO_DSTOPTS: ^ ../drivers/net/tap/bpf/tap_rss.c:126:8: error: use of undeclared identifier 'IPPROTO_FRAGMENT' case IPPROTO_FRAGMENT: ^ ../drivers/net/tap/bpf/tap_rss.c:210:33: error: use of undeclared identifier 'ETH_P_IP' if (skb->protocol == bpf_htons(ETH_P_IP)) ^
../drivers/net/tap/bpf/tap_rss.c:210:33: error: use of undeclared identifier 'ETH_P_IP'../drivers/net/tap/bpf/tap_rss.c:210:33: error: use of undeclared identifier 'ETH_P_IP'../drivers/net/tap/bpf/tap_rss.c:210:33: error: use of undeclared identifier 'ETH_P_IP'../drivers/net/tap/bpf/tap_rss.c:212:38: error: use of undeclared identifier 'ETH_P_IPV6'else if (skb->protocol == bpf_htons(ETH_P_IPV6)) ^
../drivers/net/tap/bpf/tap_rss.c:212:38: error: use of undeclared identifier 'ETH_P_IPV6'../drivers/net/tap/bpf/tap_rss.c:212:38: error: use of undeclared identifier 'ETH_P_IPV6'../drivers/net/tap/bpf/tap_rss.c:212:38: error: use of undeclared identifier 'ETH_P_IPV6'../drivers/net/tap/bpf/tap_rss.c:248:10: error: use of undeclared identifier 'TC_ACT_OK'return TC_ACT_OK; ^ ../drivers/net/tap/bpf/tap_rss.c:252:10: error: use of undeclared identifier 'TC_ACT_OK' return TC_ACT_OK; ^ ../drivers/net/tap/bpf/tap_rss.c:256:9: error: use of undeclared identifier 'TC_ACT_PIPE' return TC_ACT_PIPE; ^Quoted 2 lines
15 errors generated.
ninja: build stopped: subcommand failed.
Quoted 7 lines
This is a known issue as currently
vmlinux.hdoes not support macros. There are some efforts by Edward Zingerman to support this but this has not done yet. At the same time, you could have a trivial header file like
https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/bpf_tracing_net.h
to be used for bpf program and then your bpf program withvmlinux.hcan have much easier CORE support.
That is an example of header surgery which I would rather avoid having to carry as long term technical debt baggage.
Quoted 16 lines · Stephen Hemminger
Quoted 12 lines · On Thu, 4 Apr 2024 11:16:32 -0700 Yonghong Song <[email protected]> wrote:
Quoted 11 lines · Stephen Hemminger
I am fixing the use of TC BPF in DPDK to use libbpf and bpftool. Luca recommended using
vmlinux.hto address possible build and CO:RE issues. But it won't work.There are missing pieces such as definitions of IPV6 next header fields (in
linux/ipv6.h) and TC actions.Without major hack surgery, not possible to use
vmlinux.hinstead. Sincevmlinux.hdefines may things that overlap with other headers.Using:
[...]
Quoted 9 lines
Quoted 7 lines
This is a known issue as currently
vmlinux.hdoes not support macros. There are some efforts by Edward Zingerman to support this but this has not done yet. At the same time, you could have a trivial header file like
https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/bpf_tracing_net.h
to be used for bpf program and then your bpf program withvmlinux.hcan have much easier CORE support.
Quoted 3 lines
That is an example of header surgery which I would rather avoid having to carry as long term technical debt baggage.
What's your ultimate goal? As Yonghong said, vmlinux.h is not compatible with other headers. So you have to pick either using vmlinux.h as a base + adding missing #define's (because those are not recorded in types, so can't be put into vmlinux.h), or not use vmlinux.h, use linux UAPI/internal headers and then use explicit CO-RE helpers/attributes to make your application CO-RE-relocatable.
It's not clear from your original email why exactly you wanted to switch to vmlinux.h in the first place.
Quoted 20 lines · On Thu, 4 Apr 2024 14:45:04 -0700 Andrii Nakryiko <[email protected]>…
Quoted 9 lines
Quoted 7 lines
This is a known issue as currently
vmlinux.hdoes not support macros. There are some efforts by Edward Zingerman to support this but this has not done yet. At the same time, you could have a trivial header file like
https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/bpf_tracing_net.h
to be used for bpf program and then your bpf program withvmlinux.hcan have much easier CORE support.Quoted 3 lines
That is an example of header surgery which I would rather avoid having to carry as long term technical debt baggage.
What's your ultimate goal? As Yonghong said,
vmlinux.his not compatible with other headers. So you have to pick either usingvmlinux.has a base + adding missing #define's (because those are not recorded in types, so can't be put intovmlinux.h), or not usevmlinux.h, use linux UAPI/internal headers and then use explicit CO-RE helpers/attributes to make your application CO-RE-relocatable.
Quoted 2 lines
It's not clear from your original email why exactly you wanted to switch to
vmlinux.hin the first place.
Some backstory. There is not an existing TC filter for this, so the original developer had the idea of using BPF to do it.
The program is a small BPF program to implement a TC filter that looks at SKB and does mapping to queue based on L3 (or L3/L4) header. So not heavily dependent on kernel data structure, but sk_buff is not necessarily stable; actual layout depends on kernel config.
But the evolution of BPF has made the old code unusable. I am trying to get it working again, and cleanup to modern BPF by using BPF skeleton code.
Luca was one who had the suggestion about vmlinux.
Quoted 27 lines
Using bpftool to generate the header at build time is a bit icky, because it will look at sysfs on the build system, which is from the running kernel. But a build system's kernel might be some ancient LTS, and even be a completely different kconfig/build/distro from the actual runtime one.
We have ran in the same problem in systemd recently, and the solution is to have distros publish the
vmlinux.htogether with the kernel image/headers, that way we can rely on the fact that by build-depending on the right kernel package we get exactly the generatedvmlinux.hthat we want. This has already happened in Centos, Debian, Fedora and Arch, and I am trying to get Ubuntu onboard too.The annoying thing is that every distro packages differently, so the path needs to be configurable with a meson option.
Feel free to pilfer the systemd meson glue:
https://github.com/systemd/systemd/pull/26826/commits/d917079e7e320aa281fc4ad6f8073b0814b9cb13
It's of course file to go to the runtime kernel if no
vmlinux.his specified, as a fallback, which is going to be useful for developers machines.Signature · 2 lines
Kind regards, Luca Boccassi
Quoted 27 lines · Stephen Hemminger
Quoted 20 lines · On Thu, 4 Apr 2024 14:45:04 -0700 Andrii Nakryiko <[email protected]>…
Quoted 9 lines
Quoted 7 lines
This is a known issue as currently
vmlinux.hdoes not support macros. There are some efforts by Edward Zingerman to support this but this has not done yet. At the same time, you could have a trivial header file like
https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/bpf_tracing_net.h
to be used for bpf program and then your bpf program withvmlinux.hcan have much easier CORE support.Quoted 3 lines
That is an example of header surgery which I would rather avoid having to carry as long term technical debt baggage.
What's your ultimate goal? As Yonghong said,
vmlinux.his not compatible with other headers. So you have to pick either usingvmlinux.has a base + adding missing #define's (because those are not recorded in types, so can't be put intovmlinux.h), or not usevmlinux.h, use linux UAPI/internal headers and then use explicit CO-RE helpers/attributes to make your application CO-RE-relocatable.Quoted 2 lines
It's not clear from your original email why exactly you wanted to switch to
vmlinux.hin the first place.
Quoted 8 lines
Some backstory. There is not an existing TC filter for this, so the original developer had the idea of using BPF to do it.
The program is a small BPF program to implement a TC filter that looks at SKB and does mapping to queue based on L3 (or L3/L4) header. So not heavily dependent on kernel data structure, but
sk_buffis not necessarily stable; actual layout depends on kernel config.
If it's only a few fields from sk_buff that you need, you can define your own minimal sk_buff definition with __attribute__((preserve_access_index)) added to it. You don't have to declare fields in the right order, just make sure that field types match. E.g., something like:
struct sk_buff {
unsigned char *head;
unsigned char *data;
struct sock *sock;
} __attribute__((preserve_access_index));
You can call it struct sk_buff___mine if you already include sk_buff, to avoid the conflict. Libbpf will still understand that it should match it to struct sk_buff in the kernel.
Alternatively, you can just use BPF_CORE_READ() macro on types you get from kernel headers, even if they don't have that preserve_access_index attribute.
Or, you can just use __builtin_preserve_access_index(&skb->len) to access sk_buff's fields in CO-RE-relocatable way.
Or, you can have entire block within which all fields accesses will be CO-RE-relocatable:
int len;
__builtin_preserve_access_index(({
len = skb->len;
/* other skb accesses here as well */
}));Many ways to have bolted on CO-RE even without vmlinux.h. vmlinux.h is convenient (apart from lack of #defines, but that's an orthogonal problem), but by no means required for BPF CO-RE.
Quoted 3 lines
But the evolution of BPF has made the old code unusable. I am trying to get it working again, and cleanup to modern BPF by using BPF skeleton code.
That's a good idea independently from vmlinux.h, yep.
Quoted 32 lines
Luca was one who had the suggestion about vmlinux.
Quoted 27 lines
Using bpftool to generate the header at build time is a bit icky, because it will look at sysfs on the build system, which is from the running kernel. But a build system's kernel might be some ancient LTS, and even be a completely different kconfig/build/distro from the actual runtime one.
We have ran in the same problem in systemd recently, and the solution is to have distros publish the
vmlinux.htogether with the kernel image/headers, that way we can rely on the fact that by build-depending on the right kernel package we get exactly the generatedvmlinux.hthat we want. This has already happened in Centos, Debian, Fedora and Arch, and I am trying to get Ubuntu onboard too.The annoying thing is that every distro packages differently, so the path needs to be configurable with a meson option.
Feel free to pilfer the systemd meson glue:
https://github.com/systemd/systemd/pull/26826/commits/d917079e7e320aa281fc4ad6f8073b0814b9cb13
It's of course file to go to the runtime kernel if no
vmlinux.his specified, as a fallback, which is going to be useful for developers machines.--
Kind regards,
Luca Boccassi
Quoted 66 lines · On Fri, 5 Apr 2024 10:31:46 -0700 Andrii Nakryiko <[email protected]>…
Quoted 27 lines · Stephen Hemminger
Quoted 20 lines · On Thu, 4 Apr 2024 14:45:04 -0700 Andrii Nakryiko <[email protected]>…
Quoted 9 lines
Quoted 7 lines
This is a known issue as currently
vmlinux.hdoes not support macros. There are some efforts by Edward Zingerman to support this but this has not done yet. At the same time, you could have a trivial header file like
https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/bpf_tracing_net.h
to be used for bpf program and then your bpf program withvmlinux.hcan have much easier CORE support.Quoted 3 lines
That is an example of header surgery which I would rather avoid having to carry as long term technical debt baggage.
What's your ultimate goal? As Yonghong said,
vmlinux.his not compatible with other headers. So you have to pick either usingvmlinux.has a base + adding missing #define's (because those are not recorded in types, so can't be put intovmlinux.h), or not usevmlinux.h, use linux UAPI/internal headers and then use explicit CO-RE helpers/attributes to make your application CO-RE-relocatable.Quoted 2 lines
It's not clear from your original email why exactly you wanted to switch to
vmlinux.hin the first place.Quoted 8 lines
Some backstory. There is not an existing TC filter for this, so the original developer had the idea of using BPF to do it.
The program is a small BPF program to implement a TC filter that looks at SKB and does mapping to queue based on L3 (or L3/L4) header. So not heavily dependent on kernel data structure, but
sk_buffis not necessarily stable; actual layout depends on kernel config.If it's only a few fields from
sk_buffthat you need, you can define your own minimalsk_buffdefinition with __attribute__((preserve_access_index)) added to it. You don't have to declare fields in the right order, just make sure that field types match. E.g., something like:struct
sk_buff{
unsigned char *head;
unsigned char *data;
struct sock *sock;
} __attribute__((preserve_access_index));You can call it
struct sk_buff___mineif you already includesk_buff, to avoid the conflict. Libbpf will still understand that it should match it to structsk_buffin the kernel.Alternatively, you can just use
BPF_CORE_READ()macro on types you get from kernel headers, even if they don't have thatpreserve_access_indexattribute.Or, you can just use
__builtin_preserve_access_index(&skb->len)to access sk_buff's fields in CO-RE-relocatable way.Or, you can have entire block within which all fields accesses will be CO-RE-relocatable:
int len;
Quoted 9 lines
__builtin_preserve_access_index(({ len = skb->len; /* other skb accesses here as well */ }));Many ways to have bolted on CO-RE even without
vmlinux.h.vmlinux.his convenient (apart from lack of #defines, but that's an orthogonal problem), but by no means required for BPF CO-RE.
The skbuff pointer is passed to the TC program. So BPF_CORE_READ doesn't make sense here. The biggest concern is that kernel config of the build machine may not match the kernel config of the eventual target, and some of the fields of the sk_buff are hidden based on #ifdef and may change.
Quoted 81 lines · Stephen Hemminger
Quoted 66 lines · On Fri, 5 Apr 2024 10:31:46 -0700 Andrii Nakryiko <[email protected]>…
Quoted 27 lines · Stephen Hemminger
Quoted 20 lines · On Thu, 4 Apr 2024 14:45:04 -0700 Andrii Nakryiko <[email protected]>…
Quoted 9 lines
Quoted 7 lines
This is a known issue as currently
vmlinux.hdoes not support macros. There are some efforts by Edward Zingerman to support this but this has not done yet. At the same time, you could have a trivial header file like
https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/bpf_tracing_net.h
to be used for bpf program and then your bpf program withvmlinux.hcan have much easier CORE support.Quoted 3 lines
That is an example of header surgery which I would rather avoid having to carry as long term technical debt baggage.
What's your ultimate goal? As Yonghong said,
vmlinux.his not compatible with other headers. So you have to pick either usingvmlinux.has a base + adding missing #define's (because those are not recorded in types, so can't be put intovmlinux.h), or not usevmlinux.h, use linux UAPI/internal headers and then use explicit CO-RE helpers/attributes to make your application CO-RE-relocatable.Quoted 2 lines
It's not clear from your original email why exactly you wanted to switch to
vmlinux.hin the first place.Quoted 8 lines
Some backstory. There is not an existing TC filter for this, so the original developer had the idea of using BPF to do it.
The program is a small BPF program to implement a TC filter that looks at SKB and does mapping to queue based on L3 (or L3/L4) header. So not heavily dependent on kernel data structure, but
sk_buffis not necessarily stable; actual layout depends on kernel config.If it's only a few fields from
sk_buffthat you need, you can define your own minimalsk_buffdefinition with __attribute__((preserve_access_index)) added to it. You don't have to declare fields in the right order, just make sure that field types match. E.g., something like:struct
sk_buff{
unsigned char *head;
unsigned char *data;
struct sock *sock;
} __attribute__((preserve_access_index));You can call it
struct sk_buff___mineif you already includesk_buff, to avoid the conflict. Libbpf will still understand that it should match it to structsk_buffin the kernel.Alternatively, you can just use
BPF_CORE_READ()macro on types you get from kernel headers, even if they don't have thatpreserve_access_indexattribute.Or, you can just use
__builtin_preserve_access_index(&skb->len)to access sk_buff's fields in CO-RE-relocatable way.Or, you can have entire block within which all fields accesses will be CO-RE-relocatable:
int len;
Quoted 9 lines
__builtin_preserve_access_index(({ len = skb->len; /* other skb accesses here as well */ }));Many ways to have bolted on CO-RE even without
vmlinux.h.vmlinux.his convenient (apart from lack of #defines, but that's an orthogonal problem), but by no means required for BPF CO-RE.
Quoted 4 lines
The skbuff pointer is passed to the TC program. So
BPF_CORE_READdoesn't make sense here. The biggest concern is that kernel config of the build machine may not match the kernel config of the eventual target, and some of the fields of thesk_buffare hidden based on #ifdef and may change.
And everything else I wrote?..
Regardless, I think at this point if you want to get help, you probably should provide the source code.