tools/bootconfig: Preserve tracing settings during export

Hi Masami,

Here are four more fixes for ftrace2bconf.sh, based on bootconfig/fixes. They address settings that change or fail to restore after export:

- Event filters containing punctuation are quoted as one bootconfig value.
- Generic trigger counts are written in the syntax accepted by tracefs.
- Histogram variable tracking starts fresh for each tracing instance.
- Paused histograms keep their paused state when restored.

The base includes the trigger-filter and underscore fixes you picked up. The new sed expression also uses POSIX syntax, following your feedback.

I tested each fix and the full series in temporary tracefs instances on my Ubuntu machine, running kernel 7.0.0-38-generic, with recording disabled. The patch emails include the individual reproduction details. The fixture checks also cover each step of the series.

Feedback is welcome.

Thanks,
Eduardo

Eduardo Mucelli Rezende Oliveira (4):
tools/bootconfig: Quote event filters in ftrace2bconf.sh tools/bootconfig: Export replayable event trigger counts tools/bootconfig: Reset histogram variable tracking for each instance tools/bootconfig: Preserve paused histograms in ftrace2bconf.sh

 tools/bootconfig/scripts/ftrace2bconf.sh | 24 +++++++++++++++++++++---
 1 file changed, 21 insertions(+), 3 deletions(-)

base-commit: 1dc6d173c43e10a1d0ee36b43d0d2f371f4d262a

2.53.0

ftrace2bconf.sh emits event filters without enclosing bootconfig quotes. A filter such as prev_comm == "a,b" is therefore parsed as an array, while a # truncates the value and a semicolon or closing brace can make the exported configuration invalid.

Quote filters containing bootconfig delimiters so that the complete expression remains one value. Choose the enclosing quote in the same way as the bootconfig utility. Leave filters without delimiters as they are, including expressions that contain both kinds of quote.

---
I tested event filters containing a comma, semicolon, # and closing brace in a temporary tracefs instance on my Ubuntu machine, running kernel 7.0.0-38-generic, with recording disabled.

The original export split or truncated these filters, or produced a config that could not be parsed. With the fix, each expression remained one value and was restored unchanged. A filter containing an ordinary space worked in both versions.

 tools/bootconfig/scripts/ftrace2bconf.sh | 8 ++++++++
 1 file changed, 8 insertions(+)

index b01297217cf3..8c5889d0420c 100755
@@ -126,6 +126,14 @@ per_event_options() { # event-dir
 	fi
 	val=`cat $evdir/filter`
 	if [ "$val" != "none" ]; then
+		case "$val" in
+		*[',;#}']*)
+			case "$val" in
+			*\"*) val="'$val'";;
+			*) val="\"$val\"";;
+			esac
+			;;
+		esac
 		emit_kv $PREFIX.event.$group.$event.filter = "$val"
 	fi
 }

2.53.0

Hi Eduardo,

Masami Hiramatsu (Google) <[email protected]>

Tracefs displays generic event trigger counts as :unlimited or :count=N. ftrace2bconf.sh copies these annotations into the exported action, but the trigger input syntax requires an omitted count or :N. Replaying an exported traceoff or enable_event action therefore fails.

Remove :unlimited and convert :count=N to :N in the command token before emitting the action. Keep the filter text unchanged and leave histogram commands alone, where count can be a variable name.

---
I tested traceoff with an unlimited count, traceoff:7 with a filter, and enable_event with a finite count and filter in a temporary tracefs instance on my Ubuntu machine, running kernel 7.0.0-38-generic, with recording disabled.

The original exported count annotations were rejected during replay. With the fix, the triggers and their filters were restored correctly. A histogram defining a variable named count also worked unchanged.

 tools/bootconfig/scripts/ftrace2bconf.sh | 9 +++++++++
 1 file changed, 9 insertions(+)

index 8c5889d0420c..82c4fe021fb7 100755
@@ -118,6 +118,15 @@ per_event_options() { # event-dir
 	grep -v "^#" $evdir/trigger |
 		sed -e 's/ \[active\]$//' -e 's/ \[paused\]$//' |
 		while IFS= read -r action; do
+		action_cmd=${action%% *}
+		action_rest=${action#"$action_cmd"}
+		case "$action_cmd" in
+		hist:*) ;;
+		*:unlimited)
+			action="${action_cmd%:unlimited}$action_rest";;
+		*:count=*)
+			action="${action_cmd%:count=*}:${action_cmd##*:count=}$action_rest";;
+		esac
 		emit_kv $PREFIX.event.$group.$event.actions += "'$action'"
 	done
 

2.53.0

Histogram variables are scoped to a trace instance, but ftrace2bconf.sh keeps DEFINED_VARS across calls to event_options(). If an earlier instance has defined ts0, a later instance can export a consumer of its own ts0 before exporting that instance's provider. Replaying the configuration then fails because the local variable has not been defined yet.

Initialize the variable resolver state when starting each instance so that its consumers are deferred until their local providers are exported.

---
I reproduced this with two temporary tracefs instances on my Ubuntu machine, running kernel 7.0.0-38-generic, with recording disabled. Both instances defined a histogram variable with the same name. In the second instance, a consumer sorted before its local provider.

The original exporter put that consumer first and replay failed with an undefined variable. With the fix, the local provider was exported first and all three histograms were restored correctly.

 tools/bootconfig/scripts/ftrace2bconf.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

index 82c4fe021fb7..6d36219eb997 100755
@@ -84,8 +84,6 @@ synth_event_options() {
 }
 
 # Variables resolver
-DEFINED_VARS=
-UNRESOLVED_EVENTS=
 
 defined_vars() { # event-dir
 	grep "^hist" $1/trigger | grep -o ':[a-zA-Z0-9_]*='

@@ -159,6 +157,8 @@ retry_unresolved() {
 
 event_options() {
 	# PREFIX and INSTANCE must be set
+	DEFINED_VARS=
+	UNRESOLVED_EVENTS=
 	if [ $PREFIX = "ftrace" ]; then
 		# define the dynamic events
 		kprobe_event_options

2.53.0

The trigger file reports a paused histogram with a trailing [paused] annotation. Removing that annotation without adding a pause parameter exports the histogram as an active trigger. Replaying the exported configuration therefore enables aggregation that was previously paused.

Add the pause parameter after the hist prefix before removing the display annotation. This keeps the parameter before any filter expression and preserves both the paused state and the filter.

---
I tested paused histograms with and without a filter, plus an active histogram, in a temporary tracefs instance on my Ubuntu machine, running kernel 7.0.0-38-generic, with recording disabled.

The original exporter restored the paused histograms as active. With the fix, both paused histograms kept their state and filters, and the active histogram still restored as active. One filter contained the literal string [paused], which was preserved as well.

 tools/bootconfig/scripts/ftrace2bconf.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

index 6d36219eb997..93879181dab9 100755
@@ -114,7 +114,8 @@ per_event_options() { # event-dir
 		DEFINED_VARS="$DEFINED_VARS "`defined_vars $evdir`
 	fi
 	grep -v "^#" $evdir/trigger |
-		sed -e 's/ \[active\]$//' -e 's/ \[paused\]$//' |
+		sed -e '/^hist:.* \[paused\]$/s/^hist:/hist:pause:/' \
+			-e 's/ \[active\]$//' -e 's/ \[paused\]$//' |
 		while IFS= read -r action; do
 		action_cmd=${action%% *}
 		action_rest=${action#"$action_cmd"}

2.53.0

Hi Masami,

Thanks for the review. I've updated the event-filter patch to use the explicit delimiter pattern you suggested. It now warns and skips filters that contain delimiters and both quote types, since bootconfig cannot represent them. The other three patches are unchanged.

Changes in v2:
- Patch 1: use explicit escapes in *[,\;\#\}]* for the delimiter check. The semicolon also needs escaping for the shell to parse the pattern.
- Patch 1: detect both quote types, warn on stderr and skip the filter.
- Patches 2-4: no changes.

The series keeps event filters, generic trigger counts, histogram variable ordering and paused histogram state during export. It is still based on bootconfig/fixes, including the two fixes you picked up earlier.

I retested the filter change in a temporary tracefs instance on my Ubuntu machine, running kernel 7.0.0-38-generic, with recording disabled. I also checked the mixed-quote cases against real tracefs filters and added regression checks for both quote orders and all four delimiters. The fixture checks cover each fix and the full series.

Feedback is welcome.

Thanks,
Eduardo

Eduardo Mucelli Rezende Oliveira (4):
tools/bootconfig: Quote event filters in ftrace2bconf.sh tools/bootconfig: Export replayable event trigger counts tools/bootconfig: Reset histogram variable tracking for each instance tools/bootconfig: Preserve paused histograms in ftrace2bconf.sh

 tools/bootconfig/scripts/ftrace2bconf.sh | 29 +++++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)

base-commit: 1dc6d173c43e10a1d0ee36b43d0d2f371f4d262a

2.53.0

ftrace2bconf.sh emits event filters without enclosing bootconfig quotes. A filter such as prev_comm == "a,b" is therefore parsed as an array, while a # truncates the value and a semicolon or closing brace can make the exported configuration invalid.

Quote filters containing bootconfig delimiters so that the complete expression remains one value. Choose an enclosing quote that does not occur in the expression. When both quote types occur alongside a delimiter, bootconfig cannot represent the filter because it does not support escape sequences. Warn and skip that filter instead of emitting invalid bootconfig. Leave filters without delimiters unchanged.

---
I tested event filters containing a comma, semicolon, # and closing brace in a temporary tracefs instance on my Ubuntu machine, running kernel 7.0.0-38-generic, with recording disabled.

The original export split or truncated these filters, or produced a config that could not be parsed. With the fix, each expression remained one value and was restored unchanged. A filter containing an ordinary space worked in both versions.

I also checked the mixed-quote cases from your review. These now produce a warning and are skipped, while filters without delimiters still export.

 tools/bootconfig/scripts/ftrace2bconf.sh | 13 +++++++++++++
 1 file changed, 13 insertions(+)

index b01297217cf3..8f3edea79ca0 100755
@@ -126,6 +126,19 @@ per_event_options() { # event-dir
 	fi
 	val=`cat $evdir/filter`
 	if [ "$val" != "none" ]; then
+		case "$val" in
+		*[,\;\#\}]*)
+			case "$val" in
+			*\"*\'*|*\'*\"*)
+				cat 1>&2 << EOF
+# WARN: $PREFIX.event.$group.$event.filter cannot be quoted for bootconfig. Skip it.
+EOF
+				return;;
+			*\"*) val="'$val'";;
+			*) val="\"$val\"";;
+			esac
+			;;
+		esac
 		emit_kv $PREFIX.event.$group.$event.filter = "$val"
 	fi
 }

2.53.0

Tracefs displays generic event trigger counts as :unlimited or :count=N. ftrace2bconf.sh copies these annotations into the exported action, but the trigger input syntax requires an omitted count or :N. Replaying an exported traceoff or enable_event action therefore fails.

Remove :unlimited and convert :count=N to :N in the command token before emitting the action. Keep the filter text unchanged and leave histogram commands alone, where count can be a variable name.

---
I tested traceoff with an unlimited count, traceoff:7 with a filter, and enable_event with a finite count and filter in a temporary tracefs instance on my Ubuntu machine, running kernel 7.0.0-38-generic, with recording disabled.

The original exported count annotations were rejected during replay. With the fix, the triggers and their filters were restored correctly. A histogram defining a variable named count also worked unchanged.

 tools/bootconfig/scripts/ftrace2bconf.sh | 9 +++++++++
 1 file changed, 9 insertions(+)

index 8f3edea79ca0..9100ebcd5af6 100755
@@ -118,6 +118,15 @@ per_event_options() { # event-dir
 	grep -v "^#" $evdir/trigger |
 		sed -e 's/ \[active\]$//' -e 's/ \[paused\]$//' |
 		while IFS= read -r action; do
+		action_cmd=${action%% *}
+		action_rest=${action#"$action_cmd"}
+		case "$action_cmd" in
+		hist:*) ;;
+		*:unlimited)
+			action="${action_cmd%:unlimited}$action_rest";;
+		*:count=*)
+			action="${action_cmd%:count=*}:${action_cmd##*:count=}$action_rest";;
+		esac
 		emit_kv $PREFIX.event.$group.$event.actions += "'$action'"
 	done
 

2.53.0

Histogram variables are scoped to a trace instance, but ftrace2bconf.sh keeps DEFINED_VARS across calls to event_options(). If an earlier instance has defined ts0, a later instance can export a consumer of its own ts0 before exporting that instance's provider. Replaying the configuration then fails because the local variable has not been defined yet.

Initialize the variable resolver state when starting each instance so that its consumers are deferred until their local providers are exported.

---
I reproduced this with two temporary tracefs instances on my Ubuntu machine, running kernel 7.0.0-38-generic, with recording disabled. Both instances defined a histogram variable with the same name. In the second instance, a consumer sorted before its local provider.

The original exporter put that consumer first and replay failed with an undefined variable. With the fix, the local provider was exported first and all three histograms were restored correctly.

 tools/bootconfig/scripts/ftrace2bconf.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

index 9100ebcd5af6..b63e04bf9400 100755
@@ -84,8 +84,6 @@ synth_event_options() {
 }
 
 # Variables resolver
-DEFINED_VARS=
-UNRESOLVED_EVENTS=
 
 defined_vars() { # event-dir
 	grep "^hist" $1/trigger | grep -o ':[a-zA-Z0-9_]*='

@@ -164,6 +162,8 @@ retry_unresolved() {
 
 event_options() {
 	# PREFIX and INSTANCE must be set
+	DEFINED_VARS=
+	UNRESOLVED_EVENTS=
 	if [ $PREFIX = "ftrace" ]; then
 		# define the dynamic events
 		kprobe_event_options

2.53.0

The trigger file reports a paused histogram with a trailing [paused] annotation. Removing that annotation without adding a pause parameter exports the histogram as an active trigger. Replaying the exported configuration therefore enables aggregation that was previously paused.

Add the pause parameter after the hist prefix before removing the display annotation. This keeps the parameter before any filter expression and preserves both the paused state and the filter.

---
I tested paused histograms with and without a filter, plus an active histogram, in a temporary tracefs instance on my Ubuntu machine, running kernel 7.0.0-38-generic, with recording disabled.

The original exporter restored the paused histograms as active. With the fix, both paused histograms kept their state and filters, and the active histogram still restored as active. One filter contained the literal string [paused], which was preserved as well.

 tools/bootconfig/scripts/ftrace2bconf.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

index b63e04bf9400..31702be0c68d 100755
@@ -114,7 +114,8 @@ per_event_options() { # event-dir
 		DEFINED_VARS="$DEFINED_VARS "`defined_vars $evdir`
 	fi
 	grep -v "^#" $evdir/trigger |
-		sed -e 's/ \[active\]$//' -e 's/ \[paused\]$//' |
+		sed -e '/^hist:.* \[paused\]$/s/^hist:/hist:pause:/' \
+			-e 's/ \[active\]$//' -e 's/ \[paused\]$//' |
 		while IFS= read -r action; do
 		action_cmd=${action%% *}
 		action_rest=${action#"$action_cmd"}

2.53.0

Thanks, this series looks good to me. Let me pick those.

Masami Hiramatsu (Google) <[email protected]>

BTW, I will move these fixes (including your previous series of patches) to bootcofig/core, instead of bootconfig/fixes, because it fixes a script which is not related to kernel build/boot process but just helping specific users - who can easily get the latest kernel.

Thanks,

Masami Hiramatsu (Google) <[email protected]>

Thanks, Masami!

Eduardo Mucelli R. Oliveira -- http://eduar.do