Skip to contentSkip to conversation

Introduce --btf_features=+extra_features syntax

PATCH In review Arnaldo Carvalho de Melo 4 participants 11 messages Apr 19, 2024 – Apr 29, 2024 BPF

lore
Share & tools
lore
#0 lore raw

cover Introduce --btf_features=+extra_features syntax

Hi,

Please take a look if you agree this is a more compact, less
confusing way of asking for the set of standard BTF features + some extra features such as 'reproducible_build'.

We have this in perf, for things like:

⬢[acme@toolbox pahole]$ perf report -h -F

Usage: perf report [<options>]

-F, --fields <key[,keys...]>
output field(s): overhead period sample overhead overhead_sys overhead_us overhead_guest_sys overhead_guest_us overhead_children sample period weight1 weight2 weight3 ins_lat retire_lat p_stage_cyc pid comm dso symbol parent cpu socket srcline srcfile local_weight weight transaction trace symbol_size dso_size cgroup cgroup_id ipc_null time code_page_size local_ins_lat ins_lat local_p_stage_cyc p_stage_cyc addr local_retire_lat retire_lat simd type typeoff symoff dso_from dso_to symbol_from symbol_to mispredict abort in_tx cycles srcline_from srcline_to ipc_lbr addr_from addr_to symbol_daddr dso_daddr locked tlb mem snoop dcacheline symbol_iaddr phys_daddr data_page_size blocked

⬢[acme@toolbox pahole]$

From the 'perf report' man page for '-F':

If the keys starts with a prefix '+', then it will append the specified field(s) to the default field order. For example: perf report -F +period,sample.

- Arnaldo

Arnaldo Carvalho de Melo (2):
pahole: Factor out routine to process "--btf_features=all"
pahole: Allow asking for extra features using the '+' prefix in
--btf_features

 man-pages/pahole.1          |  6 ++++++
 pahole.c                    | 23 ++++++++++++++++-------
 tests/reproducible_build.sh |  2 +-
 3 files changed, 23 insertions(+), 8 deletions(-)
Signature · 1 line
2.44.0
#1 lore raw

PATCH 1/2 pahole: Factor out routine to process "--btf_features=all"

From: Arnaldo Carvalho de Melo <[email protected]>

As we'll use it to process "--btf_features=+reproducible_build" meaning "all + reproducible_build".

Cc Alan Maguire <[email protected]>
Cc Andrii Nakryiko <[email protected]>
Cc Daniel Xu <[email protected]>
Cc Eduard Zingerman <[email protected]>
Cc Jiri Olsa <[email protected]>
---
 pahole.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)
pahole.c +10 -7
index 38cc6362015fd95b..af94d2a45ee96cbe 100644
@@ -1340,6 +1340,14 @@ static void show_supported_btf_features(FILE *output)
 	fprintf(output, "\n");
 }
 
+static void btf_features__enable_for_all(void)
+{
+	for (int i = 0; i < ARRAY_SIZE(btf_features); i++) {
+		if (btf_features[i].enable_for_all)
+			enable_btf_feature(&btf_features[i]);
+	}
+}
+
 /* Translate --btf_features=feature1[,feature2] into conf_load values.
  * Explicitly ignores unrecognized features to allow future specification
  * of new opt-in features.
@@ -1352,12 +1360,7 @@ static void parse_btf_features(const char *features, bool strict)
 	init_btf_features();
 
 	if (strcmp(features, "all") == 0) {
-		int i;
-
-		for (i = 0; i < ARRAY_SIZE(btf_features); i++) {
-			if (btf_features[i].enable_for_all)
-				enable_btf_feature(&btf_features[i]);
-		}
+		btf_features__enable_for_all();
 		return;
 	}
 
@@ -1371,7 +1374,7 @@ static void parse_btf_features(const char *features, bool strict)
 			 * allowed.
 			 */
 			if (strcmp(feature_name, "all") == 0) {
-				parse_btf_features(feature_name, strict);
+				btf_features__enable_for_all();
 			} else if (strict) {
 				fprintf(stderr, "Feature '%s' in '%s' is not supported.  Supported BTF features are:\n",
 					feature_name, features);
Signature · 1 line
2.44.0
#2 lore raw

PATCH 2/2 pahole: Allow asking for extra features using the '+' prefix in --btf_features

In reply to Arnaldo Carvalho de Melo

From: Arnaldo Carvalho de Melo <[email protected]>

Instead of the somewhat confusing:

--btf_features=all,reproducible_build

That means "'all' the standard BTF features plus the 'reproducible_build' extra BTF feature", use + directly, making it more compact:

--btf_features=+reproducible_build

In the future we may want the '-' counterpart as a way to _remove_ some of the standard set of BTF features.

Cc Alan Maguire <[email protected]>
Cc Andrii Nakryiko <[email protected]>
Cc Daniel Xu <[email protected]>
Cc Eduard Zingerman <[email protected]>
Cc Jiri Olsa <[email protected]>
---
 man-pages/pahole.1          | 6 ++++++
 pahole.c                    | 6 ++++++
 tests/reproducible_build.sh | 2 +-
 3 files changed, 13 insertions(+), 1 deletion(-)
man-pages/pahole.1 +6 -0
index 64de3438b5f9a77a..2f4f42f8323efd6e 100644
@@ -320,6 +320,12 @@ Supported non-standard features (not enabled for 'all')
 
 So for example, specifying \-\-btf_encode=var,enum64 will result in a BTF encoding that (as well as encoding basic BTF information) will contain variables and enum64 values.
 
+.fi
+
+If one wants to add an extra feature to the set of standard ones, the '+' prefix can be used, i.e.:
+\-\-btf_features=+reproducible_build will add all standard features plus the 'reproducible_build' extra
+feature.
+
 .TP
 .B \-\-btf_features_strict
 Identical to \-\-btf_features above, but pahole will exit if it encounters an unrecognized feature.
pahole.c +6 -0
index af94d2a45ee96cbe..42c5b03ee1d1a8f8 100644
@@ -1364,6 +1364,12 @@ static void parse_btf_features(const char *features, bool strict)
 		return;
 	}
 
+	// Adding extra features to the set of standard features.
+	if (strstarts(features, "+")) {
+		btf_features__enable_for_all();
+		++features;
+	}
+
 	strncpy(f, features, BTF_MAX_FEATURE_STR)[BTF_MAX_FEATURE_STR] = '\0';
 	s = f;
 	while ((feature_name = strtok_r(s, ",", &saveptr)) != NULL) {
tests/reproducible_build.sh +1 -1
index e2f836081b125119..1222cb42c6639235 100755
@@ -29,7 +29,7 @@ nr_proc=$(getconf _NPROCESSORS_ONLN)
 
 for threads in $(seq $nr_proc) ; do
 	test -n "$VERBOSE" && echo $threads threads encoding
-	pahole -j$threads --btf_features=all,reproducible_build --btf_encode_detached=$outdir/vmlinux.btf.parallel.reproducible $vmlinux &
+	pahole -j$threads --btf_features=+reproducible_build --btf_encode_detached=$outdir/vmlinux.btf.parallel.reproducible $vmlinux &
 	pahole=$!
 	# HACK: Wait a bit for pahole to start its threads
 	sleep 0.3s
Signature · 1 line
2.44.0
#8 lore raw
In reply to Arnaldo Carvalho de Melo

Hi Arnaldo,

Quoted 27 lines · Arnaldo Carvalho de Melo

Hi,

Please take a look if you agree this is a more compact, less
confusing way of asking for the set of standard BTF features + some extra features such as 'reproducible_build'.

We have this in perf, for things like:

⬢[acme@toolbox pahole]$ perf report -h -F

Usage: perf report [<options>]

-F, --fields <key[,keys...]>
output field(s): overhead period sample overhead overhead_sys overhead_us overhead_guest_sys overhead_guest_us overhead_children sample period weight1 weight2 weight3 ins_lat retire_lat p_stage_cyc pid comm dso symbol parent cpu socket srcline srcfile local_weight weight transaction trace symbol_size dso_size cgroup cgroup_id ipc_null time code_page_size local_ins_lat ins_lat local_p_stage_cyc p_stage_cyc addr local_retire_lat retire_lat simd type typeoff symoff dso_from dso_to symbol_from symbol_to mispredict abort in_tx cycles srcline_from srcline_to ipc_lbr addr_from addr_to symbol_daddr dso_daddr locked tlb mem snoop dcacheline symbol_iaddr phys_daddr data_page_size blocked

Quoted 6 lines

⬢[acme@toolbox pahole]$

From the 'perf report' man page for '-F':

If the keys starts with a prefix '+', then it will append the specified field(s) to the default field order. For example: perf report -F +period,sample.

I think for perf it makes sense to have compact representation b/c folks might be doing a lot of ad-hoc work. But encoding BTF seems more like a write-once, read mostly. So having + notation doesn't feel like it'd help that much.

As someone who's not seen that style of syntax before, it's not immediately obvious what it does. But seeing all, I have a pretty good idea.

[..]

Thanks,
Daniel

#9 lore raw
Quoted 37 lines · Daniel Xu

Hi Arnaldo,

Quoted 27 lines · Arnaldo Carvalho de Melo

Hi,

Please take a look if you agree this is a more compact, less
confusing way of asking for the set of standard BTF features + some extra features such as 'reproducible_build'.

We have this in perf, for things like:

⬢[acme@toolbox pahole]$ perf report -h -F

Usage: perf report [<options>]

-F, --fields <key[,keys...]>
output field(s): overhead period sample overhead overhead_sys overhead_us overhead_guest_sys overhead_guest_us overhead_children sample period weight1 weight2 weight3 ins_lat retire_lat p_stage_cyc pid comm dso symbol parent cpu socket srcline srcfile local_weight weight transaction trace symbol_size dso_size cgroup cgroup_id ipc_null time code_page_size local_ins_lat ins_lat local_p_stage_cyc p_stage_cyc addr local_retire_lat retire_lat simd type typeoff symoff dso_from dso_to symbol_from symbol_to mispredict abort in_tx cycles srcline_from srcline_to ipc_lbr addr_from addr_to symbol_daddr dso_daddr locked tlb mem snoop dcacheline symbol_iaddr phys_daddr data_page_size blocked

Quoted 6 lines

⬢[acme@toolbox pahole]$

From the 'perf report' man page for '-F':

If the keys starts with a prefix '+', then it will append the specified field(s) to the default field order. For example: perf report -F +period,sample.

Quoted 9 lines

I think for perf it makes sense to have compact representation b/c folks might be doing a lot of ad-hoc work. But encoding BTF seems more like a write-once, read mostly. So having + notation doesn't feel like it'd help that much.

As someone who's not seen that style of syntax before, it's not immediately obvious what it does. But seeing all, I have a pretty good idea.

One thing we should probably bear in mind here is that for kernel builds we will always explicitly call out the set of features we want rather than use "all". So the "all" support is really more of a shortcut for developers who run pahole standalone for testing BTF encoding. It is still confusing though.

The +/- approach seems fine to me especially if there are precedents in other tools; maybe we should also switch name to "default" instead of "all" at the same time tho? The notion of default values internal to pahole (when BTF features aren't explicitly set) isn't exposed to the user, so I _think_ we can get away with using that term. We could probably do a bit of internal renaming - set_btf_features_default() -> set_btf_features_minimal() - to call these the minimal BTF features or something similar..

Quoted 4 lines

[..]

Thanks,
Daniel

#10 lore raw
Quoted 29 lines · Alan Maguire
Quoted 28 lines · Daniel Xu
Quoted 27 lines · Arnaldo Carvalho de Melo

Please take a look if you agree this is a more compact, less
confusing way of asking for the set of standard BTF features + some extra features such as 'reproducible_build'.

We have this in perf, for things like:

⬢[acme@toolbox pahole]$ perf report -h -F

Usage: perf report [<options>]

-F, --fields <key[,keys...]>
output field(s): overhead period sample overhead overhead_sys overhead_us overhead_guest_sys overhead_guest_us overhead_children sample period weight1 weight2 weight3 ins_lat retire_lat p_stage_cyc pid comm dso symbol parent cpu socket srcline srcfile local_weight weight transaction trace symbol_size dso_size cgroup cgroup_id ipc_null time code_page_size local_ins_lat ins_lat local_p_stage_cyc p_stage_cyc addr local_retire_lat retire_lat simd type typeoff symoff dso_from dso_to symbol_from symbol_to mispredict abort in_tx cycles srcline_from srcline_to ipc_lbr addr_from addr_to symbol_daddr dso_daddr locked tlb mem snoop dcacheline symbol_iaddr phys_daddr data_page_size blocked

⬢[acme@toolbox pahole]$

Quoted 8 lines
Quoted 8 lines
Quoted 4 lines

From the 'perf report' man page for '-F':

If the keys starts with a prefix '+', then it will append the specified field(s) to the default field order. For example: perf report -F +period,sample.

I think for perf it makes sense to have compact representation b/c folks might be doing a lot of ad-hoc work. But encoding BTF seems more like a write-once, read mostly. So having + notation doesn't feel like

In the case where documentation style is prefered, i.e. a write once read mostly, as you said, then use the most descriptive form.

Quoted 11 lines
Quoted 5 lines

it'd help that much.

As someone who's not seen that style of syntax before, it's not immediately obvious what it does. But seeing all, I have a pretty good idea.

One thing we should probably bear in mind here is that for kernel builds we will always explicitly call out the set of features we want rather than use "all". So the "all" support is really more of a shortcut for developers who run pahole standalone for testing BTF encoding. It is still confusing though.

Agreed, multiple people agreed 'all' is confusing as not _all_ BTF features are selected by it.

Quoted 1 line

The +/- approach seems fine to me especially if there are precedents in

Yeah, so we'll have a very compact way of adding (and removing, if we feel the need, by prefixing a undesired feature that is present in the 'default' set with -) features, in addition to a more detailed way, i.e. these will be equivalent:

--btf_features=default,reproducible_build

and:

--btf_features=+reproducible_build

Quoted 2 lines

other tools; maybe we should also switch name to "default" instead of "all" at the same time tho? The notion of default values internal to

I'm ok with this, so please send a patch renaming 'all' to 'default', on top of what is now in the 'next' branch.

- Arnaldo

Quoted 5 lines

pahole (when BTF features aren't explicitly set) isn't exposed to the user, so I _think_ we can get away with using that term. We could probably do a bit of internal renaming - set_btf_features_default() -> set_btf_features_minimal() - to call these the minimal BTF features or something similar..

#3 lore raw

Re: [PATCH 2/2] pahole: Allow asking for extra features using the '+' prefix in --btf_features

In reply to Arnaldo Carvalho de Melo
Quoted 12 lines · Arnaldo Carvalho de Melo

From: Arnaldo Carvalho de Melo <[email protected]>

Instead of the somewhat confusing:

--btf_features=all,reproducible_build

That means "'all' the standard BTF features plus the 'reproducible_build' extra BTF feature", use + directly, making it more compact:

--btf_features=+reproducible_build

for older paholes that don't yet know about + syntax, but support --btf_features, will this effectively disable all features or how will it work?

I'm thinking from the perspective of using +reproducible_build unconditionally in kernel's build scripts, will it regress something for current pahole versions?

Quoted 15 lines

In the future we may want the '-' counterpart as a way to _remove_ some of the standard set of BTF features.

Cc Alan Maguire <[email protected]>
Cc Andrii Nakryiko <[email protected]>
Cc Daniel Xu <[email protected]>
Cc Eduard Zingerman <[email protected]>
Cc Jiri Olsa <[email protected]>
---
 man-pages/pahole.1          | 6 ++++++
 pahole.c                    | 6 ++++++
 tests/reproducible_build.sh | 2 +-
 3 files changed, 13 insertions(+), 1 deletion(-)

[...]

#4 lore raw

Re: [PATCH 2/2] pahole: Allow asking for extra features using the '+' prefix in --btf_features

Quoted 15 lines · Andrii Nakryiko
Quoted 12 lines · Arnaldo Carvalho de Melo

From: Arnaldo Carvalho de Melo <[email protected]>

Instead of the somewhat confusing:

--btf_features=all,reproducible_build

That means "'all' the standard BTF features plus the 'reproducible_build' extra BTF feature", use + directly, making it more compact:

--btf_features=+reproducible_build

Quoted 7 lines

for older paholes that don't yet know about + syntax, but support --btf_features, will this effectively disable all features or how will it work?

I'm thinking from the perspective of using +reproducible_build unconditionally in kernel's build scripts, will it regress something for current pahole versions?

Well, I think it will end up being discarded just like "all" or "default", no? I.e. those were keywords not grokked by older pahole versions, so ignored as we're not using --btf_features_strict, right?

Alan?

But then we're not yet using --btf_features in scripts/Makefile.btf, right?

But as Daniel pointed out and Alan (I think) agreed, for things like scripts we probably end up using the most verbose thing as:

--btf_features=default,reproducible_build

to mean a set (the default set of BTF options) + an optional/extra feature (reproducibe_build), that for people not used to the + syntax may be more descriptive (I really think that both are confusing for beginners knowing nothing about BTF and its evolution, etc).

Alan, also we released 1.26 with "all" meaning what we now call "default", so we need to keep both meaning the same thing, right?

- Arnaldo

Quoted 17 lines
Quoted 15 lines

In the future we may want the '-' counterpart as a way to _remove_ some of the standard set of BTF features.

Cc Alan Maguire <[email protected]>
Cc Andrii Nakryiko <[email protected]>
Cc Daniel Xu <[email protected]>
Cc Eduard Zingerman <[email protected]>
Cc Jiri Olsa <[email protected]>
---
 man-pages/pahole.1          | 6 ++++++
 pahole.c                    | 6 ++++++
 tests/reproducible_build.sh | 2 +-
 3 files changed, 13 insertions(+), 1 deletion(-)

[...]

#5 lore raw

Re: [PATCH 2/2] pahole: Allow asking for extra features using the '+' prefix in --btf_features

Quoted 24 lines · Arnaldo Carvalho de Melo
Quoted 15 lines · Andrii Nakryiko
Quoted 12 lines · Arnaldo Carvalho de Melo

From: Arnaldo Carvalho de Melo <[email protected]>

Instead of the somewhat confusing:

--btf_features=all,reproducible_build

That means "'all' the standard BTF features plus the 'reproducible_build' extra BTF feature", use + directly, making it more compact:

--btf_features=+reproducible_build

Quoted 7 lines

for older paholes that don't yet know about + syntax, but support --btf_features, will this effectively disable all features or how will it work?

I'm thinking from the perspective of using +reproducible_build unconditionally in kernel's build scripts, will it regress something for current pahole versions?

Quoted 6 lines

Well, I think it will end up being discarded just like "all" or "default", no? I.e. those were keywords not grokked by older pahole versions, so ignored as we're not using --btf_features_strict, right?

Alan?

Yep, it would just be ignored, so wouldn't have the desired behaviour of enabling defaults + reproducible build option.

Quoted 8 lines

But then we're not yet using --btf_features in scripts/Makefile.btf, right?

But as Daniel pointed out and Alan (I think) agreed, for things like scripts we probably end up using the most verbose thing as:

--btf_features=default,reproducible_build

Quoted 8 lines

to mean a set (the default set of BTF options) + an optional/extra feature (reproducibe_build), that for people not used to the + syntax may be more descriptive (I really think that both are confusing for beginners knowing nothing about BTF and its evolution, etc).

Alan, also we released 1.26 with "all" meaning what we now call "default", so we need to keep both meaning the same thing, right?

I might be missing something here, but I think we should always call out explicitly the set of features we want in the kernel Makefile.btf (something like [1]). The reason for this is that the concept of what is "default" may evolve over time; for example it's going to include Daniel's kfunc definitions for soon. That's a good thing, but it could conceivably cause problems down the line. Consider a newer pahole - with a newer set of defaults - running on an older kernel. In that case, we could end up encoding BTF features we don't want. By contrast, if we always call out the full set of BTF features we want via --btf_features=feature1,feature2 etc we'll always get the expected set. Plus for folks consulting the code, it's much clearer which BTF features are in use when they look at the Makefiles for a particular kernel. So my sense of the value of "default" is as a shortcut for testing the latest and greatest set of BTF feature encoding, but not for use in the kernel tree Makefiles. Thanks!

Alan

[1]
https://lore.kernel.org/bpf/[email protected]/

Quoted 19 lines

- Arnaldo

Quoted 17 lines
Quoted 15 lines

In the future we may want the '-' counterpart as a way to _remove_ some of the standard set of BTF features.

Cc Alan Maguire <[email protected]>
Cc Andrii Nakryiko <[email protected]>
Cc Daniel Xu <[email protected]>
Cc Eduard Zingerman <[email protected]>
Cc Jiri Olsa <[email protected]>
---
 man-pages/pahole.1          | 6 ++++++
 pahole.c                    | 6 ++++++
 tests/reproducible_build.sh | 2 +-
 3 files changed, 13 insertions(+), 1 deletion(-)

[...]

#6 lore raw

Re: [PATCH 2/2] pahole: Allow asking for extra features using the '+' prefix in --btf_features

Quoted 53 lines · Alan Maguire
Quoted 24 lines · Arnaldo Carvalho de Melo
Quoted 15 lines · Andrii Nakryiko
Quoted 12 lines · Arnaldo Carvalho de Melo

From: Arnaldo Carvalho de Melo <[email protected]>

Instead of the somewhat confusing:

--btf_features=all,reproducible_build

That means "'all' the standard BTF features plus the 'reproducible_build' extra BTF feature", use + directly, making it more compact:

--btf_features=+reproducible_build

Quoted 7 lines

for older paholes that don't yet know about + syntax, but support --btf_features, will this effectively disable all features or how will it work?

I'm thinking from the perspective of using +reproducible_build unconditionally in kernel's build scripts, will it regress something for current pahole versions?

Quoted 6 lines

Well, I think it will end up being discarded just like "all" or "default", no? I.e. those were keywords not grokked by older pahole versions, so ignored as we're not using --btf_features_strict, right?

Alan?

Yep, it would just be ignored, so wouldn't have the desired behaviour of enabling defaults + reproducible build option.

Quoted 8 lines

But then we're not yet using --btf_features in scripts/Makefile.btf, right?

But as Daniel pointed out and Alan (I think) agreed, for things like scripts we probably end up using the most verbose thing as:

--btf_features=default,reproducible_build

Quoted 8 lines

to mean a set (the default set of BTF options) + an optional/extra feature (reproducibe_build), that for people not used to the + syntax may be more descriptive (I really think that both are confusing for beginners knowing nothing about BTF and its evolution, etc).

Alan, also we released 1.26 with "all" meaning what we now call "default", so we need to keep both meaning the same thing, right?

Quoted 15 lines

I might be missing something here, but I think we should always call out explicitly the set of features we want in the kernel Makefile.btf (something like [1]). The reason for this is that the concept of what is "default" may evolve over time; for example it's going to include Daniel's kfunc definitions for soon. That's a good thing, but it could conceivably cause problems down the line. Consider a newer pahole - with a newer set of defaults - running on an older kernel. In that case, we could end up encoding BTF features we don't want. By contrast, if we always call out the full set of BTF features we want via --btf_features=feature1,feature2 etc we'll always get the expected set. Plus for folks consulting the code, it's much clearer which BTF features are in use when they look at the Makefiles for a particular kernel. So my sense of the value of "default" is as a shortcut for testing the latest and greatest set of BTF feature encoding, but not for use in the kernel tree Makefiles. Thanks!

Yep, I agree, the whole point was to not regress older kernel builds with newer pahole, so we need to explicitly list used features.

Quoted 25 lines

Alan

[1]
https://lore.kernel.org/bpf/[email protected]/

Quoted 19 lines

- Arnaldo

Quoted 17 lines
Quoted 15 lines

In the future we may want the '-' counterpart as a way to _remove_ some of the standard set of BTF features.

Cc Alan Maguire <[email protected]>
Cc Andrii Nakryiko <[email protected]>
Cc Daniel Xu <[email protected]>
Cc Eduard Zingerman <[email protected]>
Cc Jiri Olsa <[email protected]>
---
 man-pages/pahole.1          | 6 ++++++
 pahole.c                    | 6 ++++++
 tests/reproducible_build.sh | 2 +-
 3 files changed, 13 insertions(+), 1 deletion(-)

[...]

#7 lore raw

Re: [PATCH 2/2] pahole: Allow asking for extra features using the '+' prefix in --btf_features

In reply to Alan Maguire
Quoted 33 lines · Alan Maguire
Quoted 10 lines · Arnaldo Carvalho de Melo
Quoted 8 lines · Andrii Nakryiko

On Fri, Apr 19, 2024 at 1:58 PM Arnaldo Carvalho de Melo <[email protected]> wrote:
for older paholes that don't yet know about + syntax, but support --btf_features, will this effectively disable all features or how will it work?

I'm thinking from the perspective of using +reproducible_build unconditionally in kernel's build scripts, will it regress something for current pahole versions?

Quoted 5 lines

Well, I think it will end up being discarded just like "all" or "default", no? I.e. those were keywords not grokked by older pahole versions, so ignored as we're not using --btf_features_strict, right?

Alan?

Yep, it would just be ignored, so wouldn't have the desired behaviour of enabling defaults + reproducible build option.

Quoted 12 lines

But then we're not yet using --btf_features in scripts/Makefile.btf, right?

But as Daniel pointed out and Alan (I think) agreed, for things like scripts we probably end up using the most verbose thing as:

--btf_features=default,reproducible_build

to mean a set (the default set of BTF options) + an optional/extra feature (reproducibe_build), that for people not used to the + syntax may be more descriptive (I really think that both are confusing for beginners knowing nothing about BTF and its evolution, etc).

Quoted 4 lines
Quoted 2 lines

Alan, also we released 1.26 with "all" meaning what we now call "default", so we need to keep both meaning the same thing, right?

I might be missing something here, but I think we should always call out

No you're not, I was just talking about what Daniel pointed out, that when using a script using 'default,extra_feature' is more descriptive than '+extra_feature', but you're right, for the kernel we go on adding the features we need and older pahole versions will just ignore those.

The explanation you gave on this message is interesting to clarify when 'default' should be used and perhaps also use the kernel build process needs/use of a list of features, etc. It would be good to have it in the man page, can you provide a formal patch doing that?

Thanks,

- Arnaldo

Quoted 39 lines

explicitly the set of features we want in the kernel Makefile.btf (something like [1]). The reason for this is that the concept of what is "default" may evolve over time; for example it's going to include Daniel's kfunc definitions for soon. That's a good thing, but it could conceivably cause problems down the line. Consider a newer pahole - with a newer set of defaults - running on an older kernel. In that case, we could end up encoding BTF features we don't want. By contrast, if we always call out the full set of BTF features we want via --btf_features=feature1,feature2 etc we'll always get the expected set. Plus for folks consulting the code, it's much clearer which BTF features are in use when they look at the Makefiles for a particular kernel. So my sense of the value of "default" is as a shortcut for testing the latest and greatest set of BTF feature encoding, but not for use in the kernel tree Makefiles. Thanks!

Alan

[1]
https://lore.kernel.org/bpf/[email protected]/

Quoted 19 lines

- Arnaldo

Quoted 17 lines
Quoted 15 lines

In the future we may want the '-' counterpart as a way to _remove_ some of the standard set of BTF features.

Cc Alan Maguire <[email protected]>
Cc Andrii Nakryiko <[email protected]>
Cc Daniel Xu <[email protected]>
Cc Eduard Zingerman <[email protected]>
Cc Jiri Olsa <[email protected]>
---
 man-pages/pahole.1          | 6 ++++++
 pahole.c                    | 6 ++++++
 tests/reproducible_build.sh | 2 +-
 3 files changed, 13 insertions(+), 1 deletion(-)

[...]