kbuild: allow symbol whitelisting with TRIM_UNUSED_KSYMS
PATCH v2 1/1 kbuild: allow symbol whitelisting with TRIM_UNUSED_KSYMS
CONFIG_TRIM_UNUSED_KSYMS currently removes all unused exported symbols from ksymtab. This works really well when using in-tree drivers, but cannot be used in its current form if some of them are out-of-tree.
Indeed, even if the list of symbols required by out-of-tree drivers is known at compile time, the only solution today to guarantee these don't get trimmed is to set CONFIG_TRIM_UNUSED_KSYMS=n. This not only wastes space, but also makes it difficult to control the ABI usable by vendor modules in distribution kernels such as Android. Being able to control the kernel ABI surface is particularly useful to ship a unique Generic Kernel Image (GKI) for all vendors.
As such, attempt to improve the situation by enabling users to specify a symbol 'whitelist' at compile time. Any symbol specified in this whitelist will be kept exported when CONFIG_TRIM_UNUSED_KSYMS is set, even if it has no in-tree user. The whitelist is defined as a simple text file, listing symbols, one per line.
---
v2: make sure to quote the whitelist path properly (Nicolas)
--- init/Kconfig | 12 ++++++++++++ scripts/adjust_autoksyms.sh | 1 + 2 files changed, 13 insertions(+)
init/Kconfig +12 -0
@@ -2180,6 +2180,18 @@ config TRIM_UNUSED_KSYMS If unsure, or if you need to build out-of-tree modules, say N. +config UNUSED_KSYMS_WHITELIST + string "Whitelist of symbols to keep in ksymtab" + depends on TRIM_UNUSED_KSYMS + help + By default, all unused exported symbols will be trimmed from the + build when TRIM_UNUSED_KSYMS is selected. + + UNUSED_KSYMS_WHITELIST allows to whitelist symbols that must be kept + exported at all times, even in absence of in-tree users. The value to + set here is the path to a text file containing the list of symbols, + one per line. + endif # MODULES config MODULES_TREE_LOOKUP
scripts/adjust_autoksyms.sh +1 -0
@@ -48,6 +48,7 @@ cat > "$new_ksyms_file" << EOT EOT sed 's/ko$/mod/' modules.order | xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- | +cat - "${CONFIG_UNUSED_KSYMS_WHITELIST:-/dev/null}" | sort -u | sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$new_ksyms_file"
Signature · 1 line
2.25.0.341.g760bfbb309-goog
Quoted 12 lines · Quentin Perret
CONFIG_TRIM_UNUSED_KSYMScurrently removes all unused exported symbols from ksymtab. This works really well when using in-tree drivers, but cannot be used in its current form if some of them are out-of-tree.Indeed, even if the list of symbols required by out-of-tree drivers is known at compile time, the only solution today to guarantee these don't get trimmed is to set
CONFIG_TRIM_UNUSED_KSYMS=n. This not only wastes space, but also makes it difficult to control the ABI usable by vendor modules in distribution kernels such as Android. Being able to control the kernel ABI surface is particularly useful to ship a unique Generic Kernel Image (GKI) for all vendors.
Quoted 7 lines
As such, attempt to improve the situation by enabling users to specify a symbol 'whitelist' at compile time. Any symbol specified in this whitelist will be kept exported when
CONFIG_TRIM_UNUSED_KSYMSis set, even if it has no in-tree user. The whitelist is defined as a simple text file, listing symbols, one per line.Signed-off-by Quentin Perret <[email protected]>
Quoted 46 lines
---
v2: make sure to quote the whitelist path properly (Nicolas)--- init/Kconfig | 12 ++++++++++++ scripts/adjust_autoksyms.sh | 1 + 2 files changed, 13 insertions(+)init/Kconfig +12
-0@@ -2180,6 +2180,18 @@ config TRIM_UNUSED_KSYMS If unsure, or if you need to build out-of-tree modules, say N. +config UNUSED_KSYMS_WHITELIST + string "Whitelist of symbols to keep in ksymtab" + depends on TRIM_UNUSED_KSYMS + help + By default, all unused exported symbols will be trimmed from the + build when TRIM_UNUSED_KSYMS is selected. + + UNUSED_KSYMS_WHITELIST allows to whitelist symbols that must be kept + exported at all times, even in absence of in-tree users. The value to + set here is the path to a text file containing the list of symbols, + one per line. + endif # MODULES config MODULES_TREE_LOOKUPscripts/adjust_autoksyms.sh +1
-0@@ -48,6 +48,7 @@ cat > "$new_ksyms_file" << EOT EOT sed 's/ko$/mod/' modules.order | xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- | +cat - "${CONFIG_UNUSED_KSYMS_WHITELIST:-/dev/null}" | sort -u | sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$new_ksyms_file"Signature · 1 line
2.25.0.341.g760bfbb309-goog
Quoted 12 lines · Quentin Perret
CONFIG_TRIM_UNUSED_KSYMScurrently removes all unused exported symbols from ksymtab. This works really well when using in-tree drivers, but cannot be used in its current form if some of them are out-of-tree.Indeed, even if the list of symbols required by out-of-tree drivers is known at compile time, the only solution today to guarantee these don't get trimmed is to set
CONFIG_TRIM_UNUSED_KSYMS=n. This not only wastes space, but also makes it difficult to control the ABI usable by vendor modules in distribution kernels such as Android. Being able to control the kernel ABI surface is particularly useful to ship a unique Generic Kernel Image (GKI) for all vendors.
Quoted 5 lines
As such, attempt to improve the situation by enabling users to specify a symbol 'whitelist' at compile time. Any symbol specified in this whitelist will be kept exported when
CONFIG_TRIM_UNUSED_KSYMSis set, even if it has no in-tree user. The whitelist is defined as a simple text file, listing symbols, one per line.
Thank you for working on this! I like the idea!
Quoted 38 lines
Signed-off-by Quentin Perret <[email protected]>---
v2: make sure to quote the whitelist path properly (Nicolas)
---
init/Kconfig | 12 ++++++++++++scripts/adjust_autoksyms.sh| 1 +2 files changed, 13 insertions(+)init/Kconfig +12
-0@@ -2180,6 +2180,18 @@ config TRIM_UNUSED_KSYMS If unsure, or if you need to build out-of-tree modules, say N. +config UNUSED_KSYMS_WHITELIST + string "Whitelist of symbols to keep in ksymtab" + depends on TRIM_UNUSED_KSYMS + help + By default, all unused exported symbols will be trimmed from the + build when TRIM_UNUSED_KSYMS is selected. + + UNUSED_KSYMS_WHITELIST allows to whitelist symbols that must be kept + exported at all times, even in absence of in-tree users. The value to + set here is the path to a text file containing the list of symbols, + one per line. +endif # MODULES
config
MODULES_TREE_LOOKUPscripts/adjust_autoksyms.sh +0
-0@@ -48,6 +48,7 @@ cat > "$new_ksyms_file" << EOT
Quoted 4 lines
EOT
sed 's/ko$/mod/' modules.order |
xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- | +cat - "${CONFIG_UNUSED_KSYMS_WHITELIST:-/dev/null}" |
This handles absolute paths very well. I wonder whether we can make this more useful for folks that want to maintain such a whitelist in their copy of the tree. Lets say, I have in my sources arch/x86/configs/x86_64_symbol_whitelist and in my config I have CONFIG_UNUSED_KSYMS_WHITELIST="arch/x86/configs/x86_64_symbol_whitelist".
If I see it correctly, UNUSED_KSYMS_WHITELIST is currently either an absolute path or a relative path to the current build directory. I would prefer if relative paths would be relative to the source directory to support the above use case. (Note, that scenario above works if I build directly in the sources, but fails if I build O=/somewhere/else.)
Cheers,
Matthias
Quoted 6 lines
sort -u |
sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$new_ksyms_file"Signature · 1 line
2.25.0.341.g760bfbb309-goog
Quoted 17 lines · Matthias Maennich' via kernel-team
Quoted 9 lines · Quentin Perret
scripts/adjust_autoksyms.sh +0
-0@@ -48,6 +48,7 @@ cat > "$new_ksyms_file" << EOTEOT
sed 's/ko$/mod/' modules.order |
xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- | +cat - "${CONFIG_UNUSED_KSYMS_WHITELIST:-/dev/null}" |This handles absolute paths very well. I wonder whether we can make this more useful for folks that want to maintain such a whitelist in their copy of the tree. Lets say, I have in my sources
arch/x86/configs/x86_64_symbol_whitelistand in my config I haveCONFIG_UNUSED_KSYMS_WHITELIST="arch/x86/configs/x86_64_symbol_whitelist".
Quoted 5 lines
If I see it correctly,
UNUSED_KSYMS_WHITELISTis currently either an absolute path or a relative path to the current build directory. I would prefer if relative paths would be relative to the source directory to support the above use case. (Note, that scenario above works if I build directly in the sources, but fails if I build O=/somewhere/else.)
Right, that is an interesting use case. I suppose something like the below should work (with appropriate documentation of the config option).
---8<---
scripts/adjust_autoksyms.sh +7 -2
@@ -38,6 +38,12 @@ esac # We need access to CONFIG_ symbols . include/config/auto.conf +ksym_wl="${CONFIG_UNUSED_KSYMS_WHITELIST:-/dev/null}" +# If the path is relative, it must be relative to the source tree +if [ "$ksym_wl" == "${ksym_wl#/}" ]; then + ksym_wl="$abs_srctree/$ksym_wl" +fi + # Generate a new ksym list file with symbols needed by the current # set of modules. cat > "$new_ksyms_file" << EOT
@@ -48,7 +54,7 @@ cat > "$new_ksyms_file" << EOT EOT sed 's/ko$/mod/' modules.order | xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- | -cat - "${CONFIG_UNUSED_KSYMS_WHITELIST:-/dev/null}" | +cat - "$ksym_wl" | sort -u | sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$new_ksyms_file" --->8---
Thoughts ?
Thanks,
Quentin
Quoted 55 lines · Quentin Perret
Quoted 17 lines · Matthias Maennich' via kernel-team
Quoted 9 lines · Quentin Perret
scripts/adjust_autoksyms.sh +0
-0@@ -48,6 +48,7 @@ cat > "$new_ksyms_file" << EOTEOT
sed 's/ko$/mod/' modules.order |
xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- | +cat - "${CONFIG_UNUSED_KSYMS_WHITELIST:-/dev/null}" |This handles absolute paths very well. I wonder whether we can make this more useful for folks that want to maintain such a whitelist in their copy of the tree. Lets say, I have in my sources
arch/x86/configs/x86_64_symbol_whitelistand in my config I haveCONFIG_UNUSED_KSYMS_WHITELIST="arch/x86/configs/x86_64_symbol_whitelist".Quoted 5 lines
If I see it correctly,
UNUSED_KSYMS_WHITELISTis currently either an absolute path or a relative path to the current build directory. I would prefer if relative paths would be relative to the source directory to support the above use case. (Note, that scenario above works if I build directly in the sources, but fails if I build O=/somewhere/else.)Right, that is an interesting use case. I suppose something like the below should work (with appropriate documentation of the config option).
---8<---
scripts/adjust_autoksyms.sh +0
-0@@ -38,6 +38,12 @@ esac # We need access to CONFIG_ symbols.
include/config/auto.confscripts/adjust_autoksyms.sh +6
-0+ksym_wl="${CONFIG_UNUSED_KSYMS_WHITELIST:-/dev/null}" +# If the path is relative, it must be relative to the source tree +if [ "$ksym_wl" == "${ksym_wl#/}" ]; then + ksym_wl="$abs_srctree/$ksym_wl" +fi +# Generate a new ksym list file with symbols needed by the current
# set of modules. cat > "$new_ksyms_file" << EOTscripts/adjust_autoksyms.sh +0
-0@@ -48,7 +54,7 @@ cat > "$new_ksyms_file" << EOTEOT
sed 's/ko$/mod/' modules.order |
xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- |scripts/adjust_autoksyms.sh +1
-1-cat - "${CONFIG_UNUSED_KSYMS_WHITELIST:-/dev/null}" | +cat - "$ksym_wl" |sort -u |
sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$new_ksyms_file" --->8---
Quoted 1 line
Thoughts ?
That definitely looks like I would expect that config option to work. Thanks for looking into that!
Cheers,
Matthias
Quoted 3 lines
Thanks,
Quentin
Quoted 1 line · Matthias Maennich' via kernel-team
That definitely looks like I would expect that config option to work.
Good, thank you, I'll send an update.
And while at it, I realized we could actually pre-populate autoksyms.h very early on with the content of the whitelist instead of having it empty. That way, there's a whole lot of things that will be compiled 'correctly' from the start (that is, adjust_autoksyms.sh won't need to re-compile them later on). And in fact, if all symbols used by in-tree modules happen to already be on the whitelist, you'll build a trimmed kernel in a single pass, which should improve build time a bit I hope.
I'll add something to v3 in addition to your suggestion.
Thanks,
Quentin
+++ Quentin Perret [29/01/20 18:15 +0000]:
Quoted 32 lines
CONFIG_TRIM_UNUSED_KSYMScurrently removes all unused exported symbols from ksymtab. This works really well when using in-tree drivers, but cannot be used in its current form if some of them are out-of-tree.Indeed, even if the list of symbols required by out-of-tree drivers is known at compile time, the only solution today to guarantee these don't get trimmed is to set
CONFIG_TRIM_UNUSED_KSYMS=n. This not only wastes space, but also makes it difficult to control the ABI usable by vendor modules in distribution kernels such as Android. Being able to control the kernel ABI surface is particularly useful to ship a unique Generic Kernel Image (GKI) for all vendors.As such, attempt to improve the situation by enabling users to specify a symbol 'whitelist' at compile time. Any symbol specified in this whitelist will be kept exported when
CONFIG_TRIM_UNUSED_KSYMSis set, even if it has no in-tree user. The whitelist is defined as a simple text file, listing symbols, one per line.Signed-off-by Quentin Perret <[email protected]>---
v2: make sure to quote the whitelist path properly (Nicolas)
---
init/Kconfig | 12 ++++++++++++scripts/adjust_autoksyms.sh| 1 +2 files changed, 13 insertions(+)init/Kconfig +0
-0
init/Kconfig config TRIM_UNUSED_KSYMS · line 2188
Quoted 9 lines
init/Kconfig +6
-0@@ -2180,6 +2180,18 @@ config TRIM_UNUSED_KSYMS If unsure, or if you need to build out-of-tree modules, say N. +config UNUSED_KSYMS_WHITELIST + string "Whitelist of symbols to keep in ksymtab" + depends on TRIM_UNUSED_KSYMS + help + By default, all unused exported symbols will be trimmed from the + build when TRIM_UNUSED_KSYMS is selected.
Hm, I thought TRIM_UNUSED_KSYMS just *unexports* unused symbols, no? "Trimmed from the build" sounds like the symbols are not compiled in or dropped completely. Please correct me if I misunderstood.
Quoted 13 lines
init/Kconfig +5
-0+ UNUSED_KSYMS_WHITELIST allows to whitelist symbols that must be kept + exported at all times, even in absence of in-tree users. The value to + set here is the path to a text file containing the list of symbols, + one per line. +endif # MODULES
config
MODULES_TREE_LOOKUPscripts/adjust_autoksyms.sh +0
-0@@ -48,6 +48,7 @@ cat > "$new_ksyms_file" << EOT
Quoted 6 lines
EOT
sed 's/ko$/mod/' modules.order |
xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- | +cat - "${CONFIG_UNUSED_KSYMS_WHITELIST:-/dev/null}" | sort -u |
sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$new_ksyms_file"
In general, I agree with the motivation behind this patch, even though we try not to provide too much support for out-of-tree modules. However, in this particular case, I think it's fair to provide some mechanism to keep some exported symbols around that we know will have users, despite having no in-tree users for a particular configuration/build. For example, livepatch exports symbols that have no in-tree users (except for the sample livepatch module, but you'd have to enable SAMPLES), and all livepatch users will always be out of tree.
I also agree with Matthias' feedback, so assuming that gets incorporated into v3:
Thanks!
Quoted 3 lines · Jessica Yu
Hm, I thought
TRIM_UNUSED_KSYMSjust *unexports* unused symbols, no? "Trimmed from the build" sounds like the symbols are not compiled in or dropped completely. Please correct me if I misunderstood.
If they are unexposed, then it doesn't make much sense to keep them around wasting space. So yes, the compiler is free to optimize away the unused code at that point.
Please see the first part of the following article where effective kernel size reduction is is achieved with this feature:
https://lwn.net/Articles/746780/
Nicolas
Hi Jessica,
Quoted 9 lines · Jessica Yu
+++ Quentin Perret [29/01/20 18:15 +0000]:
init/Kconfig config TRIM_UNUSED_KSYMS · line 2188
Quoted 6 lines
init/Kconfig +6
-0+config UNUSED_KSYMS_WHITELIST + string "Whitelist of symbols to keep in ksymtab" + depends on TRIM_UNUSED_KSYMS + help + By default, all unused exported symbols will be trimmed from the + build when TRIM_UNUSED_KSYMS is selected.Hm, I thought
TRIM_UNUSED_KSYMSjust *unexports* unused symbols, no?
Right.
Quoted 2 lines
"Trimmed from the build" sounds like the symbols are not compiled in or dropped completely. Please correct me if I misunderstood.
Good point. I tried to re-use the wording from the help text of TRIM_UNUSED_KSYMS ("This option allows for unused exported symbols to be dropped from the build"), but I agree this is a bit confusing.
I'll re-write the help text in v3.
Quoted 31 lines
Quoted 14 lines
init/Kconfig +5
-0+ UNUSED_KSYMS_WHITELIST allows to whitelist symbols that must be kept + exported at all times, even in absence of in-tree users. The value to + set here is the path to a text file containing the list of symbols, + one per line. +endif # MODULES
config
MODULES_TREE_LOOKUPscripts/adjust_autoksyms.sh +0
-0@@ -48,6 +48,7 @@ cat > "$new_ksyms_file" << EOTQuoted 6 lines
EOT
sed 's/ko$/mod/' modules.order |
xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- | +cat - "${CONFIG_UNUSED_KSYMS_WHITELIST:-/dev/null}" | sort -u |
sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$new_ksyms_file"In general, I agree with the motivation behind this patch, even though we try not to provide too much support for out-of-tree modules. However, in this particular case, I think it's fair to provide some mechanism to keep some exported symbols around that we know will have users, despite having no in-tree users for a particular configuration/build. For example, livepatch exports symbols that have no in-tree users (except for the sample livepatch module, but you'd have to enable SAMPLES), and all livepatch users will always be out of tree.
Quoted 4 lines
I also agree with Matthias' feedback, so assuming that gets incorporated into v3:
Acked-by Jessica Yu <[email protected]>
Thanks for the review!
Quentin