bpftool: introduce btf c dump sorting
Mykyta Yatsenko [email protected]
PATCH v3 1/1 bpftool: introduce btf c dump sorting
From: Mykyta Yatsenko <[email protected]>
Sort bpftool c dump output; aiming to simplify vmlinux.h diffing and forcing more natural type definitions ordering.
Definitions are sorted first by their BTF kind ranks, then by their base type name and by their own name.
Type ranks
Assign ranks to btf kinds (defined in function btf_type_rank) to set next order:
1. Anonymous enums/enums64
2. Named enums/enums64
3. Trivial types typedefs (ints, then floats)
4. Structs/Unions
5. Function prototypes
6. Forward declarations
Type rank is set to maximum for unnamed reference types, structs and unions to avoid emitting those types early. They will be emitted as part of the type chain starting with named type.
Lexicographical ordering
Each type is assigned a sort_name and own_name.sort_name is the resolved name of the final base type for reference types (typedef, pointer, array etc). Sorting by sort_name allows to group typedefs of the same base type. sort_name for non-reference type is the same as own_name. own_name is a direct name of particular type, is used as final sorting step.
--- .../bpf/bpftool/Documentation/bpftool-btf.rst | 5 +- tools/bpf/bpftool/bash-completion/bpftool | 3 + tools/bpf/bpftool/btf.c | 138 +++++++++++++++++- 3 files changed, 139 insertions(+), 7 deletions(-)
tools/bpf/bpftool/Documentation/bpftool-btf.rst +3 -2
@@ -28,7 +28,7 @@ BTF COMMANDS | **bpftool** **btf help** | | *BTF_SRC* := { **id** *BTF_ID* | **prog** *PROG* | **map** *MAP* [{**key** | **value** | **kv** | **all**}] | **file** *FILE* } -| *FORMAT* := { **raw** | **c** } +| *FORMAT* := { **raw** | **c** [**unsorted**]} | *MAP* := { **id** *MAP_ID* | **pinned** *FILE* } | *PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* | **name** *PROG_NAME* }
@@ -63,7 +63,8 @@ bpftool btf dump *BTF_SRC* pahole. **format** option can be used to override default (raw) output format. Raw - (**raw**) or C-syntax (**c**) output formats are supported. + (**raw**) or C-syntax (**c**) output formats are supported. (**unsorted**) + option can be used with (**c**) to avoid sorting the output. bpftool btf help Print short help message.
tools/bpf/bpftool/bash-completion/bpftool +3 -0
@@ -930,6 +930,9 @@ _bpftool() format) COMPREPLY=( $( compgen -W "c raw" -- "$cur" ) ) ;; + c) + COMPREPLY=( $( compgen -W "unsorted" -- "$cur" ) ) + ;; *) # emit extra options case ${words[3]} in
tools/bpf/bpftool/btf.c +133 -5
192 lines not on this page · raw message
Signature · 1 line
2.44.0
Quoted 33 lines · Mykyta Yatsenko
From: Mykyta Yatsenko <[email protected]>
Sort bpftool c dump output; aiming to simplify
vmlinux.hdiffing and forcing more natural type definitions ordering.Definitions are sorted first by their BTF kind ranks, then by their base type name and by their own name.
Type ranks
Assign ranks to btf kinds (defined in function
btf_type_rank) to set next order:
1. Anonymous enums/enums64
2. Named enums/enums64
3. Trivial types typedefs (ints, then floats)
4. Structs/Unions
5. Function prototypes
6. Forward declarationsType rank is set to maximum for unnamed reference types, structs and unions to avoid emitting those types early. They will be emitted as part of the type chain starting with named type.
Lexicographical ordering
Each type is assigned a
sort_nameandown_name.sort_nameis the resolved name of the final base type for reference types (typedef, pointer, array etc). Sorting bysort_nameallows to group typedefs of the same base type.sort_namefor non-reference type is the same asown_name.own_nameis a direct name of particular type, is used as final sorting step.
Quoted 7 lines
Signed-off-by Mykyta Yatsenko <[email protected]>--- .../bpf/bpftool/Documentation/bpftool-btf.rst | 5 +- tools/bpf/bpftool/bash-completion/bpftool | 3 + tools/bpf/bpftool/btf.c | 138 +++++++++++++++++- 3 files changed, 139 insertions(+), 7 deletions(-)
LGTM, tried it locally and it works well. In fact, see 6.8 kernel vs latest bpf-next/master (with basically the same config) comparison. It's quite minimal and easy to use to see what changes about some of the BPF internal types.
[0] https://gist.github.com/anakryiko/8fd8ebf2aba73961ebd3cf6587de6822
[...]
Quoted 49 lines · Andrii Nakryiko
Quoted 33 lines · Mykyta Yatsenko
From: Mykyta Yatsenko <[email protected]>
Sort bpftool c dump output; aiming to simplify
vmlinux.hdiffing and forcing more natural type definitions ordering.Definitions are sorted first by their BTF kind ranks, then by their base type name and by their own name.
Type ranks
Assign ranks to btf kinds (defined in function
btf_type_rank) to set next order:
1. Anonymous enums/enums64
2. Named enums/enums64
3. Trivial types typedefs (ints, then floats)
4. Structs/Unions
5. Function prototypes
6. Forward declarationsType rank is set to maximum for unnamed reference types, structs and unions to avoid emitting those types early. They will be emitted as part of the type chain starting with named type.
Lexicographical ordering
Each type is assigned a
sort_nameandown_name.sort_nameis the resolved name of the final base type for reference types (typedef, pointer, array etc). Sorting bysort_nameallows to group typedefs of the same base type.sort_namefor non-reference type is the same asown_name.own_nameis a direct name of particular type, is used as final sorting step.Quoted 7 lines
Signed-off-by Mykyta Yatsenko <[email protected]>--- .../bpf/bpftool/Documentation/bpftool-btf.rst | 5 +- tools/bpf/bpftool/bash-completion/bpftool | 3 + tools/bpf/bpftool/btf.c | 138 +++++++++++++++++- 3 files changed, 139 insertions(+), 7 deletions(-)LGTM, tried it locally and it works well. In fact, see 6.8 kernel vs latest bpf-next/master (with basically the same config) comparison. It's quite minimal and easy to use to see what changes about some of the BPF internal types.
Quoted 4 lines
Acked-by Andrii Nakryiko <[email protected]>Tested-by Andrii Nakryiko <[email protected]>[0] https://gist.github.com/anakryiko/8fd8ebf2aba73961ebd3cf6587de6822
Just noticed:
+ ETHTOOL_A_TS_STAT_UNSPEC = 0,
+ ETHTOOL_A_TS_STAT_TX_PKTS = 1,
+ ETHTOOL_A_TS_STAT_TX_LOST = 2,
+ ETHTOOL_A_TS_STAT_TX_ERR = 3,
+ __ETHTOOL_A_TS_STAT_CNT = 4,
+ ETHTOOL_A_TS_STAT_MAX = 3,
};
I'm a bit surprised that enum values are not sorted. I'm guessing the enum names come in dwarf order and copied the same way in BTF ?
I guess it's not an issue.
Quoted 67 lines · Alexei Starovoitov
Quoted 49 lines · Andrii Nakryiko
Quoted 33 lines · Mykyta Yatsenko
From: Mykyta Yatsenko <[email protected]>
Sort bpftool c dump output; aiming to simplify
vmlinux.hdiffing and forcing more natural type definitions ordering.Definitions are sorted first by their BTF kind ranks, then by their base type name and by their own name.
Type ranks
Assign ranks to btf kinds (defined in function
btf_type_rank) to set next order:
1. Anonymous enums/enums64
2. Named enums/enums64
3. Trivial types typedefs (ints, then floats)
4. Structs/Unions
5. Function prototypes
6. Forward declarationsType rank is set to maximum for unnamed reference types, structs and unions to avoid emitting those types early. They will be emitted as part of the type chain starting with named type.
Lexicographical ordering
Each type is assigned a
sort_nameandown_name.sort_nameis the resolved name of the final base type for reference types (typedef, pointer, array etc). Sorting bysort_nameallows to group typedefs of the same base type.sort_namefor non-reference type is the same asown_name.own_nameis a direct name of particular type, is used as final sorting step.Quoted 7 lines
Signed-off-by Mykyta Yatsenko <[email protected]>--- .../bpf/bpftool/Documentation/bpftool-btf.rst | 5 +- tools/bpf/bpftool/bash-completion/bpftool | 3 + tools/bpf/bpftool/btf.c | 138 +++++++++++++++++- 3 files changed, 139 insertions(+), 7 deletions(-)LGTM, tried it locally and it works well. In fact, see 6.8 kernel vs latest bpf-next/master (with basically the same config) comparison. It's quite minimal and easy to use to see what changes about some of the BPF internal types.
Quoted 4 lines
Acked-by Andrii Nakryiko <[email protected]>Tested-by Andrii Nakryiko <[email protected]>[0] https://gist.github.com/anakryiko/8fd8ebf2aba73961ebd3cf6587de6822
Just noticed:
+
ETHTOOL_A_TS_STAT_UNSPEC= 0,
+ETHTOOL_A_TS_STAT_TX_PKTS= 1,
+ETHTOOL_A_TS_STAT_TX_LOST= 2,
+ETHTOOL_A_TS_STAT_TX_ERR= 3,
+__ETHTOOL_A_TS_STAT_CNT= 4,
+ETHTOOL_A_TS_STAT_MAX= 3,
};
Quoted 4 lines
I'm a bit surprised that enum values are not sorted. I'm guessing the enum names come in dwarf order and copied the same way in BTF ?
I guess it's not an issue.
Yes, it's the order in which enumerators are defined (which DWARF and thus BTF preserves). I think it makes sense to keep the original order for those.
2024-05-13 20:29 UTC+0100 ~ Mykyta Yatsenko [email protected]
Quoted 44 lines
From: Mykyta Yatsenko <[email protected]>
Sort bpftool c dump output; aiming to simplify
vmlinux.hdiffing and forcing more natural type definitions ordering.Definitions are sorted first by their BTF kind ranks, then by their base type name and by their own name.
Type ranks
Assign ranks to btf kinds (defined in function
btf_type_rank) to set next order:
1. Anonymous enums/enums64
2. Named enums/enums64
3. Trivial types typedefs (ints, then floats)
4. Structs/Unions
5. Function prototypes
6. Forward declarationsType rank is set to maximum for unnamed reference types, structs and unions to avoid emitting those types early. They will be emitted as part of the type chain starting with named type.
Lexicographical ordering
Each type is assigned a
sort_nameandown_name.sort_nameis the resolved name of the final base type for reference types (typedef, pointer, array etc). Sorting bysort_nameallows to group typedefs of the same base type.sort_namefor non-reference type is the same asown_name.own_nameis a direct name of particular type, is used as final sorting step.Signed-off-by Mykyta Yatsenko <[email protected]>--- .../bpf/bpftool/Documentation/bpftool-btf.rst | 5 +- tools/bpf/bpftool/bash-completion/bpftool | 3 + tools/bpf/bpftool/btf.c | 138 +++++++++++++++++- 3 files changed, 139 insertions(+), 7 deletions(-)tools/bpf/bpftool/Documentation/bpftool-btf.rst +0
-0
tools/bpf/bpftool/Documentation/bpftool-btf.rst BTF COMMANDS · line 31
Quoted 5 lines
tools/bpf/bpftool/Documentation/bpftool-btf.rst +1
-1@@ -28,7 +28,7 @@ BTF COMMANDS | **bpftool** **btf help** | | *BTF_SRC* := { **id** *BTF_ID* | **prog** *PROG* | **map** *MAP* [{**key** | **value** | **kv** | **all**}] | **file** *FILE* } -| *FORMAT* := { **raw** | **c** } +| *FORMAT* := { **raw** | **c** [**unsorted**]}
Missing space at the end, "[**unsorted**] }"
tools/bpf/bpftool/Documentation/bpftool-btf.rst bpftool btf dump *BTF_SRC* · line 67
Quoted 10 lines
tools/bpf/bpftool/Documentation/bpftool-btf.rst +2
-1| *MAP* := { **id** *MAP_ID* | **pinned** *FILE* } | *PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* | **name** *PROG_NAME* }@@ -63,7 +63,8 @@ bpftool btf dump *BTF_SRC* pahole. **format** option can be used to override default (raw) output format. Raw - (**raw**) or C-syntax (**c**) output formats are supported. + (**raw**) or C-syntax (**c**) output formats are supported. (**unsorted**) + option can be used with (**c**) to avoid sorting the output.
The parenthesis are not part of the formatting here, they are used in the previous sentence to mention the keyword related to the options mentioned in the sentence. So here we could have:
"With C-style formatting, the output is sorted by default. Use the **unsorted** option to avoid sorting the output."
Quoted 21 lines
tools/bpf/bpftool/Documentation/bpftool-btf.rst +0
-0bpftool btf help Print short help message.tools/bpf/bpftool/bash-completion/bpftool +3
-0@@ -930,6 +930,9 @@ _bpftool() format) COMPREPLY=( $( compgen -W "c raw" -- "$cur" ) ) ;; + c) + COMPREPLY=( $( compgen -W "unsorted" -- "$cur" ) ) + ;; *) # emit extra options case ${words[3]} intools/bpf/bpftool/btf.c +0
-0
[...]
tools/bpf/bpftool/btf.c static int do_help(int argc, char **argv) · line 1194
Quoted 6 lines
tools/bpf/bpftool/btf.c +1
-1@@ -1063,7 +1191,7 @@ static int do_help(int argc, char **argv)" %1$s %2$s help\n" "\n" " BTF_SRC := { id BTF_ID | prog PROG | map MAP [{key | value | kv | all}] | file FILE }\n" -" FORMAT := { raw | c }\n" +" FORMAT := { raw | c [unsorted]}\n"
Missing space as well: "[unsorted] }\n"
tools/bpf/bpftool/btf.c static int do_help(int argc, char **argv) · line 1197
Quoted 3 lines
tools/bpf/bpftool/btf.c +0
-0" " HELP_SPEC_MAP "\n" " " HELP_SPEC_PROGRAM "\n" " " HELP_SPEC_OPTIONS " |\n"
With the above addressed: