Skip to contentSkip to conversation

Maintain selftest configuration in-tree

PATCH Superseded Daniel Müller 2 participants 7 messages Jul 26, 2022 – Jul 27, 2022

Subsystems (3)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover Maintain selftest configuration in-tree

BPF selftests mandate certain kernel configuration options to be present in order to pass. Currently the "reference" config files containing these options are hosted in a separate repository [0]. From there they are picked up by the BPF continuous integration system as well as the in-tree vmtest.sh helper script, which allows for running tests in a VM-based setup locally.

But it gets worse, as "BPF CI" is really two CI systems: one for libbpf (mentioned above) and one for the bpf-next kernel repository (or more precisely:
family of repositories, as bpf-rc is using the system). As such, we have an additional -- and slightly divergent -- copy of these configurations.

This patch set proposes the merging of said configurations into this repository. Doing so provides several benefits:
1) the vmtest.sh script is now self-contained, no longer requiring to pull configurations over the network
2) we can have a single copy of these configurations, eliminating the maintenance burden of keeping two versions in-sync
3) the kernel tree is the place where most development happens, so it is the most natural to adjust configurations as changes are proposed there, as opposed to out-of-tree, where they would always remain an afterthought

The patch set is structured in such a way that we first integrate the external configuration [0] and then adjust the vmtest.sh script to pick up the local configuration instead of reaching out to GitHub.

[0] https://github.com/libbpf/libbpf/tree/20f03302350a4143825cedcbd210c4d7112c1898/travis-ci/vmtest/configs

---
Changelog:
v2 -> v3:
- removed seven more options from s390x configuration that overlapped with pre-existing config
v1 -> v2:
- minimized imported kernel configs and made them build on top of existing tools/testing/selftests/bpf/config
- moved them directly into tools/testing/selftests/bpf/
- sorted and cleaned up tools/testing/selftests/bpf/config
- removed "selftests/bpf: Integrate vmtest configs" from patch set
- removed 4.9 & 5.5 configs

Daniel Müller (3):
selftests/bpf: Sort configuration
selftests/bpf: Copy over libbpf configs
selftests/bpf: Adjust vmtest.sh to use local kernel configuration

 tools/testing/selftests/bpf/DENYLIST       |   6 +
 tools/testing/selftests/bpf/DENYLIST.s390x |  67 ++++++
 tools/testing/selftests/bpf/config         | 101 ++++-----
 tools/testing/selftests/bpf/config.s390x   | 147 ++++++++++++
 tools/testing/selftests/bpf/config.x86_64  | 251 +++++++++++++++++++++
 tools/testing/selftests/bpf/vmtest.sh      |  51 +++--
 6 files changed, 554 insertions(+), 69 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/DENYLIST
 create mode 100644 tools/testing/selftests/bpf/DENYLIST.s390x
 create mode 100644 tools/testing/selftests/bpf/config.s390x
 create mode 100644 tools/testing/selftests/bpf/config.x86_64
Signature · 1 line
2.30.2
#1 lore raw

PATCH v3 1/3 selftests/bpf: Sort configuration

This change makes sure to sort the existing minimal kernel configuration containing options required for running BPF selftests alphabetically. Doing so will make it easier to diff it against other configurations, which in turn helps with maintaining disjunct config files that build on top of each other. It also helped identify the CONFIG_IPV6_GRE being set twice and removes one of the occurrences.
Lastly, we change NET_CLS_BPF from 'm' to 'y'. Having this option as 'm' will cause failures of the btf_skc_cls_ingress selftest.

---
 tools/testing/selftests/bpf/config | 101 ++++++++++++++---------------
 1 file changed, 50 insertions(+), 51 deletions(-)
tools/testing/selftests/bpf/config +50 -51
index c05904d..fabf0c 100644
@@ -1,65 +1,64 @@
+CONFIG_BLK_DEV_LOOP=y
 CONFIG_BPF=y
-CONFIG_BPF_SYSCALL=y
-CONFIG_NET_CLS_BPF=m
 CONFIG_BPF_EVENTS=y
-CONFIG_TEST_BPF=m
+CONFIG_BPF_JIT=y
+CONFIG_BPF_LIRC_MODE2=y
+CONFIG_BPF_LSM=y
+CONFIG_BPF_STREAM_PARSER=y
+CONFIG_BPF_SYSCALL=y
 CONFIG_CGROUP_BPF=y
-CONFIG_NETDEVSIM=m
-CONFIG_NET_CLS_ACT=y
-CONFIG_NET_SCHED=y
-CONFIG_NET_SCH_INGRESS=y
-CONFIG_NET_IPIP=y
-CONFIG_IPV6=y
-CONFIG_NET_IPGRE_DEMUX=y
-CONFIG_NET_IPGRE=y
-CONFIG_IPV6_GRE=y
-CONFIG_CRYPTO_USER_API_HASH=m
 CONFIG_CRYPTO_HMAC=m
 CONFIG_CRYPTO_SHA256=m
-CONFIG_VXLAN=y
-CONFIG_GENEVE=y
-CONFIG_NET_CLS_FLOWER=m
-CONFIG_LWTUNNEL=y
-CONFIG_BPF_STREAM_PARSER=y
-CONFIG_XDP_SOCKETS=y
+CONFIG_CRYPTO_USER_API_HASH=m
+CONFIG_DYNAMIC_FTRACE=y
+CONFIG_FPROBE=y
 CONFIG_FTRACE_SYSCALLS=y
-CONFIG_IPV6_TUNNEL=y
-CONFIG_IPV6_GRE=y
-CONFIG_IPV6_SEG6_BPF=y
-CONFIG_NET_FOU=m
-CONFIG_NET_FOU_IP_TUNNELS=y
+CONFIG_FUNCTION_TRACER=y
+CONFIG_GENEVE=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_IMA=y
+CONFIG_IMA_READ_POLICY=y
+CONFIG_IMA_WRITE_POLICY=y
+CONFIG_IP_NF_FILTER=y
+CONFIG_IP_NF_RAW=y
+CONFIG_IP_NF_TARGET_SYNPROXY=y
+CONFIG_IPV6=y
 CONFIG_IPV6_FOU=m
 CONFIG_IPV6_FOU_TUNNEL=m
-CONFIG_MPLS=y
-CONFIG_NET_MPLS_GSO=m
-CONFIG_MPLS_ROUTING=m
-CONFIG_MPLS_IPTUNNEL=m
+CONFIG_IPV6_GRE=y
+CONFIG_IPV6_SEG6_BPF=y
 CONFIG_IPV6_SIT=m
-CONFIG_BPF_JIT=y
-CONFIG_BPF_LSM=y
-CONFIG_SECURITY=y
-CONFIG_RC_CORE=y
+CONFIG_IPV6_TUNNEL=y
 CONFIG_LIRC=y
-CONFIG_BPF_LIRC_MODE2=y
-CONFIG_IMA=y
-CONFIG_SECURITYFS=y
-CONFIG_IMA_WRITE_POLICY=y
-CONFIG_IMA_READ_POLICY=y
-CONFIG_BLK_DEV_LOOP=y
-CONFIG_FUNCTION_TRACER=y
-CONFIG_DYNAMIC_FTRACE=y
+CONFIG_LWTUNNEL=y
+CONFIG_MPLS=y
+CONFIG_MPLS_IPTUNNEL=m
+CONFIG_MPLS_ROUTING=m
+CONFIG_MPTCP=y
+CONFIG_NET_CLS_ACT=y
+CONFIG_NET_CLS_BPF=y
+CONFIG_NET_CLS_FLOWER=m
+CONFIG_NET_FOU=m
+CONFIG_NET_FOU_IP_TUNNELS=y
+CONFIG_NET_IPGRE=y
+CONFIG_NET_IPGRE_DEMUX=y
+CONFIG_NET_IPIP=y
+CONFIG_NET_MPLS_GSO=m
+CONFIG_NET_SCH_INGRESS=y
+CONFIG_NET_SCHED=y
+CONFIG_NETDEVSIM=m
 CONFIG_NETFILTER=y
+CONFIG_NETFILTER_SYNPROXY=y
+CONFIG_NETFILTER_XT_MATCH_STATE=y
+CONFIG_NETFILTER_XT_TARGET_CT=y
+CONFIG_NF_CONNTRACK=y
 CONFIG_NF_DEFRAG_IPV4=y
 CONFIG_NF_DEFRAG_IPV6=y
-CONFIG_NF_CONNTRACK=y
+CONFIG_RC_CORE=y
+CONFIG_SECURITY=y
+CONFIG_SECURITYFS=y
+CONFIG_TEST_BPF=m
 CONFIG_USERFAULTFD=y
-CONFIG_FPROBE=y
-CONFIG_IKCONFIG=y
-CONFIG_IKCONFIG_PROC=y
-CONFIG_MPTCP=y
-CONFIG_NETFILTER_SYNPROXY=y
-CONFIG_NETFILTER_XT_TARGET_CT=y
-CONFIG_NETFILTER_XT_MATCH_STATE=y
-CONFIG_IP_NF_FILTER=y
-CONFIG_IP_NF_TARGET_SYNPROXY=y
-CONFIG_IP_NF_RAW=y
+CONFIG_VXLAN=y
+CONFIG_XDP_SOCKETS=y
Signature · 1 line
2.30.2
#2 lore raw

PATCH v3 2/3 selftests/bpf: Copy over libbpf configs

In reply to Daniel Müller

This change integrates libbpf maintained configurations and black/white lists [0] into the repository, co-located with the BPF selftests themselves. We minimize the kernel configurations to keep future updates as small as possible [1]. Furthermore, we make both kernel configurations build on top of the existing configuration tools/testing/selftests/bpf/config (to be concatenated before build). Lastly, we replaced the terms blacklist & whitelist with denylist and allowlist, respectively.

[0] https://github.com/libbpf/libbpf/tree/20f03302350a4143825cedcbd210c4d7112c1898/travis-ci/vmtest/configs
[1] https://lore.kernel.org/bpf/[email protected]/T/#m30a53648352ed494e556ac003042a9ad0a8f98c6

---
 tools/testing/selftests/bpf/DENYLIST       |   6 +
 tools/testing/selftests/bpf/DENYLIST.s390x |  67 ++++++
 tools/testing/selftests/bpf/config.s390x   | 147 ++++++++++++
 tools/testing/selftests/bpf/config.x86_64  | 251 +++++++++++++++++++++
 4 files changed, 471 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/DENYLIST
 create mode 100644 tools/testing/selftests/bpf/DENYLIST.s390x
 create mode 100644 tools/testing/selftests/bpf/config.s390x
 create mode 100644 tools/testing/selftests/bpf/config.x86_64
tools/testing/selftests/bpf/DENYLIST new file +6 -0
new file mode 100644
index 0000000..939de574
@@ -0,0 +1,6 @@
+# TEMPORARY
+get_stack_raw_tp    # spams with kernel warnings until next bpf -> bpf-next merge
+stacktrace_build_id_nmi
+stacktrace_build_id
+task_fd_query_rawtp
+varlen
tools/testing/selftests/bpf/DENYLIST.s390x new file +67 -0

67 lines not on this page · raw message

tools/testing/selftests/bpf/config.s390x new file +147 -0

147 lines not on this page · raw message

tools/testing/selftests/bpf/config.x86_64 new file +251 -0

251 lines not on this page · raw message

Signature · 1 line
2.30.2
#3 lore raw

PATCH v3 3/3 selftests/bpf: Adjust vmtest.sh to use local kernel configuration

In reply to Daniel Müller

So far the vmtest.sh script, which can be used as a convenient way to run bpf selftests, has obtained the kernel config safe to use for testing from the libbpf/libbpf GitHub repository [0].
Given that we now have included this configuration into this very repository, we can just consume it from here as well, eliminating the necessity of remote accesses.
With this change we adjust the logic in the script to use the configuration from below tools/testing/selftests/bpf/configs/ instead of pulling it over the network.

[0]: https://github.com/libbpf/libbpf

---
 tools/testing/selftests/bpf/vmtest.sh | 51 +++++++++++++++++----------
 1 file changed, 33 insertions(+), 18 deletions(-)
tools/testing/selftests/bpf/vmtest.sh +33 -18
index e0bb04a..b86ae4 100755
@@ -30,8 +30,7 @@ DEFAULT_COMMAND="./test_progs"
 MOUNT_DIR="mnt"
 ROOTFS_IMAGE="root.img"
 OUTPUT_DIR="$HOME/.bpf_selftests"
-KCONFIG_URL="https://raw.githubusercontent.com/libbpf/libbpf/master/travis-ci/vmtest/configs/config-latest.${ARCH}"
-KCONFIG_API_URL="https://api.github.com/repos/libbpf/libbpf/contents/travis-ci/vmtest/configs/config-latest.${ARCH}"
+KCONFIG_REL_PATHS=("tools/testing/selftests/bpf/config" "tools/testing/selftests/bpf/config.${ARCH}")
 INDEX_URL="https://raw.githubusercontent.com/libbpf/ci/master/INDEX"
 NUM_COMPILE_JOBS="$(nproc)"
 LOG_FILE_BASE="$(date +"bpf_selftests.%Y-%m-%d_%H-%M-%S")"
@@ -269,26 +268,42 @@ is_rel_path()
 	[[ ${path:0:1} != "/" ]]
 }
 
+do_update_kconfig()
+{
+	local kernel_checkout="$1"
+	local kconfig_file="$2"
+
+	rm -f "$kconfig_file" 2> /dev/null
+
+	for config in "${KCONFIG_REL_PATHS[@]}"; do
+		local kconfig_src="${kernel_checkout}/${config}"
+		cat "$kconfig_src" >> "$kconfig_file"
+	done
+}
+
 update_kconfig()
 {
-	local kconfig_file="$1"
-	local update_command="curl -sLf ${KCONFIG_URL} -o ${kconfig_file}"
-	# Github does not return the "last-modified" header when retrieving the
-	# raw contents of the file. Use the API call to get the last-modified
-	# time of the kernel config and only update the config if it has been
-	# updated after the previously cached config was created. This avoids
-	# unnecessarily compiling the kernel and selftests.
+	local kernel_checkout="$1"
+	local kconfig_file="$2"
+
 	if [[ -f "${kconfig_file}" ]]; then
-		local last_modified_date="$(curl -sL -D - "${KCONFIG_API_URL}" -o /dev/null | \
-			grep "last-modified" | awk -F ': ' '{print $2}')"
-		local remote_modified_timestamp="$(date -d "${last_modified_date}" +"%s")"
-		local local_creation_timestamp="$(stat -c %Y "${kconfig_file}")"
+		local local_modified="$(stat -c %Y "${kconfig_file}")"
 
-		if [[ "${remote_modified_timestamp}" -gt "${local_creation_timestamp}" ]]; then
-			${update_command}
-		fi
+		for config in "${KCONFIG_REL_PATHS[@]}"; do
+			local kconfig_src="${kernel_checkout}/${config}"
+			local src_modified="$(stat -c %Y "${kconfig_src}")"
+			# Only update the config if it has been updated after the
+			# previously cached config was created. This avoids
+			# unnecessarily compiling the kernel and selftests.
+			if [[ "${src_modified}" -gt "${local_modified}" ]]; then
+				do_update_kconfig "$kernel_checkout" "$kconfig_file"
+				# Once we have found one outdated configuration
+				# there is no need to check other ones.
+				break
+			fi
+		done
 	else
-		${update_command}
+		do_update_kconfig "$kernel_checkout" "$kconfig_file"
 	fi
 }
 
@@ -372,7 +387,7 @@ main()
 
 	mkdir -p "${OUTPUT_DIR}"
 	mkdir -p "${mount_dir}"
-	update_kconfig "${kconfig_file}"
+	update_kconfig "${kernel_checkout}" "${kconfig_file}"
 
 	recompile_kernel "${kernel_checkout}" "${make_command}"
 
Signature · 1 line
2.30.2
Acked-by on the v2 series
#4 lore raw
In reply to Daniel Müller
Quoted 56 lines · Daniel Müller

BPF selftests mandate certain kernel configuration options to be present in order to pass. Currently the "reference" config files containing these options are hosted in a separate repository [0]. From there they are picked up by the BPF continuous integration system as well as the in-tree vmtest.sh helper script, which allows for running tests in a VM-based setup locally.

But it gets worse, as "BPF CI" is really two CI systems: one for libbpf (mentioned above) and one for the bpf-next kernel repository (or more precisely:
family of repositories, as bpf-rc is using the system). As such, we have an additional -- and slightly divergent -- copy of these configurations.

This patch set proposes the merging of said configurations into this repository. Doing so provides several benefits:
1) the vmtest.sh script is now self-contained, no longer requiring to pull configurations over the network
2) we can have a single copy of these configurations, eliminating the maintenance burden of keeping two versions in-sync
3) the kernel tree is the place where most development happens, so it is the most natural to adjust configurations as changes are proposed there, as opposed to out-of-tree, where they would always remain an afterthought

The patch set is structured in such a way that we first integrate the external configuration [0] and then adjust the vmtest.sh script to pick up the local configuration instead of reaching out to GitHub.

[0] https://github.com/libbpf/libbpf/tree/20f03302350a4143825cedcbd210c4d7112c1898/travis-ci/vmtest/configs

---
Changelog:
v2 -> v3:
- removed seven more options from s390x configuration that overlapped with pre-existing config
v1 -> v2:
- minimized imported kernel configs and made them build on top of existing tools/testing/selftests/bpf/config
- moved them directly into tools/testing/selftests/bpf/
- sorted and cleaned up tools/testing/selftests/bpf/config
- removed "selftests/bpf: Integrate vmtest configs" from patch set
- removed 4.9 & 5.5 configs

Daniel Müller (3):
selftests/bpf: Sort configuration
selftests/bpf: Copy over libbpf configs
selftests/bpf: Adjust vmtest.sh to use local kernel configuration

tools/testing/selftests/bpf/DENYLIST       |   6 +
tools/testing/selftests/bpf/DENYLIST.s390x |  67 ++++++
tools/testing/selftests/bpf/config         | 101 ++++-----
tools/testing/selftests/bpf/config.s390x   | 147 ++++++++++++
tools/testing/selftests/bpf/config.x86_64  | 251 +++++++++++++++++++++
tools/testing/selftests/bpf/vmtest.sh      |  51 +++--
6 files changed, 554 insertions(+), 69 deletions(-)
Quoted 8 lines

create mode 100644 tools/testing/selftests/bpf/DENYLIST create mode 100644 tools/testing/selftests/bpf/DENYLIST.s390x create mode 100644 tools/testing/selftests/bpf/config.s390x create mode 100644 tools/testing/selftests/bpf/config.x86_64

Signature · 1 line
2.30.2
#5 lore raw

Hey Daniel,

While each patch in series shows v3, the cover latter subject still shows v2, although description shows v2 -> v3 changes. This also explains why CI showed v2 version in PR. Something to check in your setup.

Regards,
Mykola

Quoted 68 lines · Mykola Lysenko
Quoted 57 lines · Daniel Müller

BPF selftests mandate certain kernel configuration options to be present in order to pass. Currently the "reference" config files containing these options are hosted in a separate repository [0]. From there they are picked up by the BPF continuous integration system as well as the in-tree vmtest.sh helper script, which allows for running tests in a VM-based setup locally.

But it gets worse, as "BPF CI" is really two CI systems: one for libbpf (mentioned above) and one for the bpf-next kernel repository (or more precisely:
family of repositories, as bpf-rc is using the system). As such, we have an additional -- and slightly divergent -- copy of these configurations.

This patch set proposes the merging of said configurations into this repository. Doing so provides several benefits:
1) the vmtest.sh script is now self-contained, no longer requiring to pull configurations over the network
2) we can have a single copy of these configurations, eliminating the maintenance burden of keeping two versions in-sync
3) the kernel tree is the place where most development happens, so it is the most natural to adjust configurations as changes are proposed there, as opposed to out-of-tree, where they would always remain an afterthought

The patch set is structured in such a way that we first integrate the external configuration [0] and then adjust the vmtest.sh script to pick up the local configuration instead of reaching out to GitHub.

[0] https://github.com/libbpf/libbpf/tree/20f03302350a4143825cedcbd210c4d7112c1898/travis-ci/vmtest/configs

---
Changelog:
v2 -> v3:
- removed seven more options from s390x configuration that overlapped with pre-existing config
v1 -> v2:
- minimized imported kernel configs and made them build on top of existing tools/testing/selftests/bpf/config
- moved them directly into tools/testing/selftests/bpf/
- sorted and cleaned up tools/testing/selftests/bpf/config
- removed "selftests/bpf: Integrate vmtest configs" from patch set
- removed 4.9 & 5.5 configs

Daniel Müller (3):
selftests/bpf: Sort configuration
selftests/bpf: Copy over libbpf configs
selftests/bpf: Adjust vmtest.sh to use local kernel configuration

tools/testing/selftests/bpf/DENYLIST       |   6 +
tools/testing/selftests/bpf/DENYLIST.s390x |  67 ++++++
tools/testing/selftests/bpf/config         | 101 ++++-----
tools/testing/selftests/bpf/config.s390x   | 147 ++++++++++++
tools/testing/selftests/bpf/config.x86_64  | 251 +++++++++++++++++++++
tools/testing/selftests/bpf/vmtest.sh      |  51 +++--
6 files changed, 554 insertions(+), 69 deletions(-)
Quoted 8 lines

create mode 100644 tools/testing/selftests/bpf/DENYLIST create mode 100644 tools/testing/selftests/bpf/DENYLIST.s390x create mode 100644 tools/testing/selftests/bpf/config.s390x create mode 100644 tools/testing/selftests/bpf/config.x86_64

Signature · 1 line
2.30.2
#6 lore raw
Quoted 3 lines · Mykola Lysenko

Hey Daniel,

While each patch in series shows v3, the cover latter subject still shows v2, although description shows v2 -> v3 changes. This also explains why CI showed v2 version in PR. Something to check in your setup.

Thanks for pointing that out. Let me resend a clean v4 with correct cover letter (to prevent any confusion from having two v3 floating around).

Sorry about that.

[..]

Thanks,
Daniel