Skip to contentSkip to conversation

Verify hardened builds

RFC PATCH Superseded Stefan Sørensen 3 participants 10 messages May 3, 2018 – Feb 6, 2019

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Verify hardened builds

This patch series introduces a new package post install check that verifies that the correct build hardening flags has been applied.

Most of the work here is done by the annobin GCC plugin that annotates all objects files, libraries and executables with the flags used in the build.

The checking functionality is heavily based on the check-bin-arch functionality with only minor adjustments, and with the validation itself performed by the hardened.sh script from the annobin package.

At the end of the package install step, it will output any failed
checks:
hardened.sh: output/target/usr/bin/foo: FAIL: compiled with -fstack-protector-off hardened.sh: output/target/usr/bin/foo: FAIL: optimization level of -O0 used hardened.sh: output/target/usr/bin/foo: FAIL: insufficient value for -D_FORTIFY_SOURCE=0 hardened.sh: output/target/usr/bin/foo: FAIL: -Wl,-z,now not used

Stefan S?rensen (2):
annobin: New package
core: Verify that hardening flags are used

 Config.in                                     | 15 ++++
 ...1-Only-issue-warning-for-PIC-PIE-mix.patch | 47 ++++++++++++
 package/annobin/Config.in                     | 12 +++
 package/annobin/annobin.hash                  |  2 +
 package/annobin/annobin.mk                    | 44 +++++++++++
 package/gcc/gcc-final/gcc-final.mk            |  3 +
 package/pkg-generic.mk                        | 36 +++++++++
 support/scripts/check-hardened                | 74 +++++++++++++++++++
 toolchain/Config.in                           |  2 +
 .../pkg-toolchain-external.mk                 |  3 +
 toolchain/toolchain-wrapper.c                 |  3 +
 toolchain/toolchain/toolchain.mk              |  4 +
 12 files changed, 245 insertions(+)
 create mode 100644 package/annobin/0001-Only-issue-warning-for-PIC-PIE-mix.patch
 create mode 100644 package/annobin/Config.in
 create mode 100644 package/annobin/annobin.hash
 create mode 100644 package/annobin/annobin.mk
 create mode 100755 support/scripts/check-hardened
Signature · 1 line
2.17.0
#1 lore raw

RFC PATCH 1/2 annobin: New package

---
 ...1-Only-issue-warning-for-PIC-PIE-mix.patch | 47 +++++++++++++++++++
 package/annobin/Config.in                     | 12 +++++
 package/annobin/annobin.hash                  |  2 +
 package/annobin/annobin.mk                    | 44 +++++++++++++++++
 package/gcc/gcc-final/gcc-final.mk            |  3 ++
 toolchain/Config.in                           |  2 +
 .../pkg-toolchain-external.mk                 |  3 ++
 toolchain/toolchain-wrapper.c                 |  3 ++
 toolchain/toolchain/toolchain.mk              |  4 ++
 9 files changed, 120 insertions(+)
 create mode 100644 package/annobin/0001-Only-issue-warning-for-PIC-PIE-mix.patch
 create mode 100644 package/annobin/Config.in
 create mode 100644 package/annobin/annobin.hash
 create mode 100644 package/annobin/annobin.mk
package/annobin/0001-Only-issue-warning-for-PIC-PIE-mix.patch new file +47 -0
new file mode 100644
index 0000000000..21d5d8f01f
@@ -0,0 +1,47 @@
+From dcd48f47e73e7d03e42d4de8449edc0b31afb812 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Stefan=20S=C3=B8rensen?= <[email protected]>
+Date: Thu, 3 May 2018 12:21:25 +0200
+Subject: [PATCH] Only issue warning for PIC/PIE mix
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+A lot of packages build with a mix of -fPIC and -fPIE, so bump this
+down from a failure to just issuing a warning.
+
+Signed-off-by: Stefan S?rensen <[email protected]>
+---
+ scripts/hardened.sh | 10 +++++++++-
+ 1 file changed, 9 insertions(+), 1 deletion(-)
+
+diff --git a/scripts/hardened.sh b/scripts/hardened.sh
+index b12574e..cc7cc8f 100755
+--- a/scripts/hardened.sh
++++ b/scripts/hardened.sh
+@@ -173,6 +173,14 @@ fail ()
+     vulnerable=1
+ }
+ 
++warn ()
++{
++    if [ $report -gt 1 ]
++    then
++        report "$file: WARN:" ${1+"$@"}
++    fi
++}
++
+ pass ()
+ {
+     if [ $report -gt 2 ]
+@@ -708,7 +716,7 @@ check_for_pie_or_pic ()
+     else
+ 	if [ ${#hard[*]} -gt 1 ];
+ 	then
+-	    fail "multiple, different, settings of -fpic/-fpie used"
++	    warn "multiple, different, settings of -fpic/-fpie used"
+ 	else
+ 	    if [[ $filetype = lib || ( $filetype = auto && $file == *.so ) ]] ;
+ 	    then
+-- 
+2.17.0
+
package/annobin/Config.in new file +12 -0
new file mode 100644
index 0000000000..64f1ff6963
@@ -0,0 +1,12 @@
+config BR2_TOOLCHAIN_ANNOBIN_GCC_PLUGIN
+	bool "annobin"
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_6
+	help
+	  A plugin for GCC that records extra information in the files
+	  that it compiles, and a set of scripts that analyze the
+	  recorded information.  These scripts can determine things
+	  ABI clashes in compiled binaries, or the absence of required
+	  hardening options
+
+	  Enabling this will slightly (1-2%) increase the size of
+	  built binaries.
package/annobin/annobin.hash new file +2 -0
new file mode 100644
index 0000000000..0340e55291
@@ -0,0 +1,2 @@
+# Locally calculated
+sha256	176a8588088af40361f14415c2a1dfaae0723c46cd6df7765605090d6c4cea49  annobin-5.6.tar.xz
package/annobin/annobin.mk new file +44 -0

44 lines not on this page · raw message

package/gcc/gcc-final/gcc-final.mk +3 -0
index 9897d18682..9e739bccf6 100644
@@ -116,6 +116,9 @@ endef
 HOST_GCC_FINAL_POST_INSTALL_HOOKS += HOST_GCC_FINAL_CREATE_CC_SYMLINKS
 
 HOST_GCC_FINAL_TOOLCHAIN_WRAPPER_ARGS += $(HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS)
+ifeq ($(BR2_TOOLCHAIN_ANNOBIN_GCC_PLUGIN),y)
+HOST_GCC_FINAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_ANNOBIN_GCC_PLUGIN='"$(ANNOBIN_GCC_PLUGIN)"'
+endif
 HOST_GCC_FINAL_POST_BUILD_HOOKS += TOOLCHAIN_WRAPPER_BUILD
 HOST_GCC_FINAL_POST_INSTALL_HOOKS += TOOLCHAIN_WRAPPER_INSTALL
 # Note: this must be done after CREATE_CC_SYMLINKS, otherwise the
toolchain/Config.in +2 -0
index 121ddb4fa4..dc3f1d8cc6 100644
@@ -533,4 +533,6 @@ config BR2_TOOLCHAIN_HAS_LIBQUADMATH
 	bool
 	default y if BR2_i386 || BR2_x86_64
 
+source "package/annobin/Config.in"
+
 endmenu
toolchain/toolchain-external/pkg-toolchain-external.mk +3 -0
index 8b2c283654..457c23ddf6 100644
@@ -241,6 +241,9 @@ TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += \
 	-DBR_CROSS_PATH_REL='"$(TOOLCHAIN_EXTERNAL_BIN:$(HOST_DIR)/%=%)"'
 endif
 
+ifeq ($(BR2_TOOLCHAIN_ANNOBIN_GCC_PLUGIN),y)
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_ANNOBIN_GCC_PLUGIN='"$(ANNOBIN_GCC_PLUGIN)"'
+endif
 
 #
 # The following functions creates the symbolic links needed to get the
toolchain/toolchain-wrapper.c +3 -0
index c5eb813dd0..d45c9d4f59 100644
@@ -94,6 +94,9 @@ static char *predef_args[] = {
 #if defined(BR_MIPS_TARGET_BIG_ENDIAN) || defined(BR_ARC_TARGET_BIG_ENDIAN)
 	"-EB",
 #endif
+#ifdef BR_ANNOBIN_GCC_PLUGIN
+        "-fplugin=" BR_ANNOBIN_GCC_PLUGIN,
+#endif
 #ifdef BR_ADDITIONAL_CFLAGS
 	BR_ADDITIONAL_CFLAGS
 #endif
toolchain/toolchain/toolchain.mk +4 -0
index 91c9ca2eff..2b7ef05703 100644
@@ -10,6 +10,10 @@ else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
 TOOLCHAIN_DEPENDENCIES += toolchain-external
 endif
 
+ifeq ($(BR2_TOOLCHAIN_ANNOBIN_GCC_PLUGIN),y)
+TOOLCHAIN_DEPENDENCIES += host-annobin
+endif
+
 TOOLCHAIN_ADD_TOOLCHAIN_DEPENDENCY = NO
 
 # Apply a hack that Rick Felker suggested[1] to avoid conflicts between libc
Signature · 1 line
2.17.0
#2 lore raw

RFC PATCH 1/2 annobin: New package

Hi Stefan,

Quoted 7 lines · Stefan S?rensen
---
 ...1-Only-issue-warning-for-PIC-PIE-mix.patch | 47 +++++++++++++++++++
 package/annobin/Config.in                     | 12 +++++
 package/annobin/annobin.hash                  |  2 +
 package/annobin/annobin.mk                    | 44 +++++++++++++++++
 package/gcc/gcc-final/gcc-final.mk            |  3 ++

It might be useful to split off the integration into the toolchain into a
separate patch. annobin is by itself already usable by explicitly specifying -fplugin= (e.g. in the build of a custom package), right?

It's not strictly necessary to do that split though. Just that the integration
with the toolchain may be a little more controversial than the package itself.

Quoted 16 lines
 toolchain/Config.in                           |  2 +
 .../pkg-toolchain-external.mk                 |  3 ++
 toolchain/toolchain-wrapper.c                 |  3 ++
 toolchain/toolchain/toolchain.mk              |  4 ++
 9 files changed, 120 insertions(+)
 create mode 100644 package/annobin/0001-Only-issue-warning-for-PIC-PIE-mix.patch
 create mode 100644 package/annobin/Config.in
 create mode 100644 package/annobin/annobin.hash
 create mode 100644 package/annobin/annobin.mk
package/annobin/0001-Only-issue-warning-for-PIC-PIE-mix.patch new file +0 -0
new file mode 100644
index 0000000000..21d5d8f01f

package/annobin/0001-Only-issue-warning-for-PIC-PIE-mix.patch line 10

Quoted 10 lines
package/annobin/0001-Only-issue-warning-for-PIC-PIE-mix.patch new file +10 -0
@@ -0,0 +1,47 @@
+From dcd48f47e73e7d03e42d4de8449edc0b31afb812 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Stefan=20S=C3=B8rensen?= <[email protected]>
+Date: Thu, 3 May 2018 12:21:25 +0200
+Subject: [PATCH] Only issue warning for PIC/PIE mix
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+A lot of packages build with a mix of -fPIC and -fPIE, so bump this
+down from a failure to just issuing a warning.

Is that really the case? I mean, if an executable contains code (directly, not
in a shared library) that has not been compiled with -fPIE/-fpie, then the executable is not (or may not be) completely position-independent, right?

TBH, I don't really understand how this position independent executable is used
in the end. Does the kernel's ELF loader perform ASLR while loading it?

[snip]

package/annobin/Config.in

Quoted 5 lines
package/annobin/Config.in new file +0 -0
new file mode 100644
index 0000000000..64f1ff6963

Should be Config.in.host.

package/annobin/Config.in line 1

Quoted 2 lines
package/annobin/Config.in new file +1 -0
@@ -0,0 +1,12 @@
+config BR2_TOOLCHAIN_ANNOBIN_GCC_PLUGIN

Please call it BR2_PACKAGE_HOST_ANNOBIN so the package infra is used in full.

package/annobin/Config.in line 3

Quoted 2 lines
package/annobin/Config.in new file +2 -0
+	bool "annobin"
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_6

Can you add a comment explaining why 6 is the minimum, and not 4.5 (first GCC
supporting plugins)?

package/annobin/Config.in line 7

Quoted 4 lines
package/annobin/Config.in new file +4 -0
+	help
+	  A plugin for GCC that records extra information in the files
+	  that it compiles, and a set of scripts that analyze the
+	  recorded information.  These scripts can determine things
                                                                   ^^like

package/annobin/Config.in line 12

Quoted 5 lines
package/annobin/Config.in new file +5 -0
+	  ABI clashes in compiled binaries, or the absence of required
+	  hardening options
+
+	  Enabling this will slightly (1-2%) increase the size of
+	  built binaries.

Really? Isn't this info stripped off in the strip step?

Please add an upstream URL. E.g.
https://developers.redhat.com/blog/2018/02/20/annobin-storing-information-binaries/
or https://fedoraproject.org/wiki/Toolchain/Watermark

[snip]

package/annobin/Config.in

Quoted 11 lines
package/annobin/Config.in new file +11 -0
+ANNOBIN_VERSION = 5.6
+ANNOBIN_SOURCE = annobin-$(ANNOBIN_VERSION).tar.xz
+ANNOBIN_SITE = https://nickc.fedorapeople.org
+
+# toolchain depends on host-annobin, so shortcircuit the reverse
+# dependency to avoid a circular dependency
+ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
+HOST_ANNOBIN_DEPENDENCIES += toolchain-buildroot
+else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
+HOST_ANNOBIN_DEPENDENCIES += toolchain-external
+endif

So in a first patch I'd just make it depend on toolchain, and then when
integrating it with the toolchain change it into this dependency.

package/annobin/Config.in

Quoted 10 lines
package/annobin/Config.in new file +10 -0
+# The plugin has to be configured with the same arcane configure
+# scripts used by gcc, this prevents regeneration of the scripts.
+define ANNOBIN_PRE_CONFIGURE_FIXUP
+	(cd $(@D); touch aclocal.m4 plugin/config.h.in configure */configure \
+		Makefile.in */Makefile.in)
+endef
+
+HOST_ANNOBIN_PRE_CONFIGURE_HOOKS += ANNOBIN_PRE_CONFIGURE_FIXUP
+
+# If using an external toolchain, we cannot install the plugin in the standard

I guess you mean a pre-installed external toolchain.

package/annobin/Config.in

Quoted 3 lines
package/annobin/Config.in new file +3 -0
+# location, so provide our own and put the includes from the standard location in
+# CXX_FLAGS.
+ANNOBIN_PLUGIN_DIR = $(HOST_DIR)/libexec/annobin

Variable should be named HOST_ANNOBIN_...

We can choose any location we like, right? Then I'd use
$(HOST_DIR)/lib/gcc/plugin/annobin.

package/annobin/Config.in

Quoted 10 lines
package/annobin/Config.in new file +10 -0
+ANNOBIN_CXXFLAGS = $(HOST_CXXFLAGS) -I$(shell $(TARGET_CC) --print-file-name=plugin)/include
+
+# The host and target options are mixed up, so override the defaults
+HOST_ANNOBIN_CONF_OPTS = \
+	--build=$(GNU_HOST_NAME) \
+	--host=$(GNU_TARGET_NAME) \
+	--with-gcc-plugin-dir=$(ANNOBIN_PLUGIN_DIR) \
+	CXXFLAGS="$(ANNOBIN_CXXFLAGS)"
+
+ANNOBIN_GCC_PLUGIN=$(ANNOBIN_PLUGIN_DIR)/annobin.so

Also here HOST_ANNOBIN_...

Quoted 1 line

+HARDENED_SH=$(HOST_DIR)/bin/hardened.sh

And here HOST_ANNOBIN_HARDENED_SH. Doesn't check-package warn about this?

Though personally I don't think something like this needs to be put in a
variable - other people have a different opinion however.

package/gcc/gcc-final/gcc-final.mk endef · line 120

Quoted 12 lines
package/annobin/Config.in new file +2 -0
+
+$(eval $(host-autotools-package))
package/gcc/gcc-final/gcc-final.mk +2 -0
index 9897d18682..9e739bccf6 100644
@@ -116,6 +116,9 @@ endef
 HOST_GCC_FINAL_POST_INSTALL_HOOKS += HOST_GCC_FINAL_CREATE_CC_SYMLINKS
 
 HOST_GCC_FINAL_TOOLCHAIN_WRAPPER_ARGS += $(HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS)
+ifeq ($(BR2_TOOLCHAIN_ANNOBIN_GCC_PLUGIN),y)
+HOST_GCC_FINAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_ANNOBIN_GCC_PLUGIN='"$(ANNOBIN_GCC_PLUGIN)"'

Is there a reason to repeat this in gcc-final.mk and pkg-toolchain-external.mk,
rather than specifying it once in toolchain/toolchain-wrapper.mk (like most of the wrapper options)?

toolchain/Config.in config BR2_TOOLCHAIN_HAS_LIBQUADMATH · line 536

Quoted 13 lines
package/gcc/gcc-final/gcc-final.mk +1 -0
+endif
 HOST_GCC_FINAL_POST_BUILD_HOOKS += TOOLCHAIN_WRAPPER_BUILD
 HOST_GCC_FINAL_POST_INSTALL_HOOKS += TOOLCHAIN_WRAPPER_INSTALL
 # Note: this must be done after CREATE_CC_SYMLINKS, otherwise the
toolchain/Config.in +1 -0
index 121ddb4fa4..dc3f1d8cc6 100644
@@ -533,4 +533,6 @@ config BR2_TOOLCHAIN_HAS_LIBQUADMATH
 	bool
 	default y if BR2_i386 || BR2_x86_64
 
+source "package/annobin/Config.in"

I would make it a full-fledged host package, not a toolchain option. So include
it from package/Config.in.host.

Regards,
Arnout

Quoted 46 lines
toolchain/Config.in +1 -0
+
 endmenu
toolchain/toolchain-external/pkg-toolchain-external.mk +3 -0
index 8b2c283654..457c23ddf6 100644
@@ -241,6 +241,9 @@ TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += \
 	-DBR_CROSS_PATH_REL='"$(TOOLCHAIN_EXTERNAL_BIN:$(HOST_DIR)/%=%)"'
 endif
 
+ifeq ($(BR2_TOOLCHAIN_ANNOBIN_GCC_PLUGIN),y)
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_ANNOBIN_GCC_PLUGIN='"$(ANNOBIN_GCC_PLUGIN)"'
+endif
 
 #
 # The following functions creates the symbolic links needed to get the
toolchain/toolchain-wrapper.c +3 -0
index c5eb813dd0..d45c9d4f59 100644
@@ -94,6 +94,9 @@ static char *predef_args[] = {
 #if defined(BR_MIPS_TARGET_BIG_ENDIAN) || defined(BR_ARC_TARGET_BIG_ENDIAN)
 	"-EB",
 #endif
+#ifdef BR_ANNOBIN_GCC_PLUGIN
+        "-fplugin=" BR_ANNOBIN_GCC_PLUGIN,
+#endif
 #ifdef BR_ADDITIONAL_CFLAGS
 	BR_ADDITIONAL_CFLAGS
 #endif
toolchain/toolchain/toolchain.mk +4 -0
index 91c9ca2eff..2b7ef05703 100644
@@ -10,6 +10,10 @@ else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
 TOOLCHAIN_DEPENDENCIES += toolchain-external
 endif
 
+ifeq ($(BR2_TOOLCHAIN_ANNOBIN_GCC_PLUGIN),y)
+TOOLCHAIN_DEPENDENCIES += host-annobin
+endif
+
 TOOLCHAIN_ADD_TOOLCHAIN_DEPENDENCY = NO
 
 # Apply a hack that Rick Felker suggested[1] to avoid conflicts between libc
Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
#3 lore raw

RFC PATCH 1/2 annobin: New package

Quoted 7 lines · Arnout Vandecappelle
Quoted 2 lines

+A lot of packages build with a mix of -fPIC and -fPIE, so bump this down from a failure to just issuing a warning.

Is that really the case? I mean, if an executable contains code (directly, not in a shared library) that has not been compiled with -fPIE/-fpie, then the executable is not (or may not be) completely position-independent, right?

You are right, it does not fail on the mixed static+pic/pie case - it turns out that the original script does not fail on the pure static case. I will add a fix for this.

Quoted 3 lines

TBH, I don't really understand how this position independent executable is used in the end. Does the kernel's ELF loader perform ASLR while loading it?

I believe so - I think that the executable it handled more or less like a shared library.

Quoted 5 lines

package/annobin/Config.in line 3

Quoted 2 lines
package/annobin/Config.in new file +2 -0
+	bool "annobin"
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_6

Can you add a comment explaining why 6 is the minimum, and not 4.5
(first GCC supporting plugins)?

To be honest, I don't recall why this was added. I will test it and drop it if not needed.

Quoted 4 lines

package/annobin/Config.in line 12

Quoted 2 lines
package/annobin/Config.in new file +2 -0
+	  Enabling this will slightly (1-2%) increase the size of
+	  built binaries.

Really? Isn't this info stripped off in the strip step?

No, this is not touched by strip. It is intended that you should be able to verify the final binaries - though this is probably more relevant on ordianry Linux distributions.

Quoted 9 lines

HOST_GCC_FINAL_TOOLCHAIN_WRAPPER_ARGS +=

Review

Quoted 4 lines

$(HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS)

(quoted diff) +2 -1
+ifeq ($(BR2_TOOLCHAIN_ANNOBIN_GCC_PLUGIN),y)
+HOST_GCC_FINAL_TOOLCHAIN_WRAPPER_ARGS +=
-DBR_ANNOBIN_GCC_PLUGIN='"$(ANNOBIN_GCC_PLUGIN)"'

Is there a reason to repeat this in gcc-final.mk and pkg-toolchain-
external.mk, rather than specifying it once in toolchain/toolchain- wrapper.mk (like most of the wrapper options)?

Adding it in toolchain/toolchain-wrapper.mk will also cause it to be added in the wrapper of gcc-initial, requiring that the annobin plugin be built much earlier.

Stefan

#4 lore raw

RFC PATCH 1/2 annobin: New package

Quoted 5 lines · Stefan
Quoted 3 lines · Arnout Vandecappelle
Quoted 2 lines

+A lot of packages build with a mix of -fPIC and -fPIE, so bump this down from a failure to just issuing a warning.

Quoted 8 lines
Quoted 4 lines

Is that really the case? I mean, if an executable contains code (directly, not in a shared library) that has not been compiled with -fPIE/-fpie, then the executable is not (or may not be) completely position-independent, right?

You are right, it does not fail on the mixed static+pic/pie case - it turns out that the original script does not fail on the pure static case. I will add a fix for this.

So can you explain in the commit log of the patch why exactly it is needed
then? And maybe whatever you do would be upstreamable in the end?

Quoted 7 lines
Quoted 3 lines

TBH, I don't really understand how this position independent executable is used in the end. Does the kernel's ELF loader perform ASLR while loading it?

I believe so - I think that the executable it handled more or less like a shared library.

Quoted 7 lines
Quoted 5 lines

package/annobin/Config.in line 3

Quoted 2 lines
package/annobin/Config.in new file +2 -0
+	bool "annobin"
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_6

Can you add a comment explaining why 6 is the minimum, and not 4.5
(first GCC supporting plugins)?

To be honest, I don't recall why this was added.

That's exactly why there should be a comment :-)

Quoted 11 lines

I will test it and
drop it if not needed.

Quoted 4 lines

package/annobin/Config.in line 12

Quoted 2 lines
package/annobin/Config.in new file +2 -0
+	  Enabling this will slightly (1-2%) increase the size of
+	  built binaries.

Really? Isn't this info stripped off in the strip step?

No, this is not touched by strip. It is intended that you should be able to verify the final binaries - though this is probably more relevant on ordianry Linux distributions.

Really really? I haven't tried it, but the documentation of annobin says that
it will store the information in a NOTES section, and we strip with --remove-section=.note so it should be gone.

Oh, hang on, it's stores in a .note.something section, and our strip command is
missing a * at the end. Which is probably a bug, I think we really do want to strip off the notes in the target... But that's a pretty dramatic change.

Quoted 10 lines
Quoted 9 lines

HOST_GCC_FINAL_TOOLCHAIN_WRAPPER_ARGS +=

Review

Quoted 4 lines

$(HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS)

(quoted diff) +2 -1
+ifeq ($(BR2_TOOLCHAIN_ANNOBIN_GCC_PLUGIN),y)
+HOST_GCC_FINAL_TOOLCHAIN_WRAPPER_ARGS +=
-DBR_ANNOBIN_GCC_PLUGIN='"$(ANNOBIN_GCC_PLUGIN)"'

Is there a reason to repeat this in gcc-final.mk and pkg-toolchain-
external.mk, rather than specifying it once in toolchain/toolchain- wrapper.mk (like most of the wrapper options)?

Quoted 3 lines

Adding it in toolchain/toolchain-wrapper.mk will also cause it to be added in the wrapper of gcc-initial, requiring that the annobin plugin be built much earlier.

Ah indeed, I forgot that we now build the wrapper for gcc-initial as well.

Regards,
Arnout

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
#5 lore raw

RFC PATCH 1/2 annobin: New package

In reply to Stefan Sørensen

Hello Stefan,

Quoted 1 line · On Thu, 3 May 2018 16:31:46 +0200 Stefan S?rensen…

In the mean time, the package checksec was added, which is able to do the same sort of checks on binaries to verify if they have been built with specific security hardening options:

config BR2_PACKAGE_HOST_CHECKSEC
bool "host checksec"
help
This tool provides a shell script to check the properties of executables
(PIE,RELRO,Stack Canaries,Fortify Source). It also has a kernel test mode that can run on target for testing of PaX, ASLR, heap and config hardening.

NOTE: when using this tool as a host tool, the tool can offline check a target folder of elf files for hardening features enabled in those elf files. There are other features of this tool, like the kernel test feature that are not functional offline, but require the user to execute in a chroot or on target.

https://github.com/slimm609/checksec.sh.git

This one is already in Buildroot, and is a lot easier to integrate than a gcc plugin. So unless you see an issue with checksec that is solved by annobin, we'll probably stick to using checksec.

Thanks,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#6 lore raw

RFC PATCH 1/2 annobin: New package

Quoted 3 lines · Thomas Petazzoni

This one is already in Buildroot, and is a lot easier to integrate than a gcc plugin. So unless you see an issue with checksec that is solved by annobin, we'll probably stick to using checksec.

The issue with checksec is that is only checks if *some* of the code in the binary is compiled with the correct security options - it does not detect that the correct options has not been used for compiling all of the code.

Unfortunately I do not have any time right now to spend on this, but I hope that I will be able to update this to a more recent annobin relase soon.

Stefan

#7 lore raw

RFC PATCH 1/2 annobin: New package

Hello,

Quoted 10 lines · Stefan
Quoted 3 lines · Thomas Petazzoni

This one is already in Buildroot, and is a lot easier to integrate than a gcc plugin. So unless you see an issue with checksec that is solved by annobin, we'll probably stick to using checksec.

The issue with checksec is that is only checks if *some* of the code in the binary is compiled with the correct security options - it does not detect that the correct options has not been used for compiling all of the code.

Ah, OK.

Quoted 3 lines

Unfortunately I do not have any time right now to spend on this, but I hope that I will be able to update this to a more recent annobin relase soon.

Yes, please send a new series when you have some time then.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#8 lore raw

RFC PATCH 2/2 core: Verify that hardening flags are used

In reply to Stefan Sørensen

This patch add a new package post install check that verifies that configured hardening options are used.

Using the ELF notes added by the GCC annobin plugin, it verifies that the following build options are used:
* Stack protector
* RELRO
* FORTIFY_SOURCE
* Optimization
* Possition Independent Code/Executeable (-fPIC/-fPIE)

---
 Config.in                      | 15 +++++++
 package/pkg-generic.mk         | 36 +++++++++++++++++
 support/scripts/check-hardened | 74 ++++++++++++++++++++++++++++++++++
 3 files changed, 125 insertions(+)
 create mode 100755 support/scripts/check-hardened
Config.in +15 -0
index 6b5b2b043c..43fd15f3a2 100644
@@ -826,6 +826,21 @@ endchoice
 
 comment "Fortify Source needs a glibc toolchain and optimization"
 	depends on (!BR2_TOOLCHAIN_USES_GLIBC || BR2_OPTIMIZE_0)
+
+
+config BR2_CHECK_HARDENING
+       bool "Verify hardened build"
+       depends on BR2_TOOLCHAIN_ANNOBIN_GCC_PLUGIN
+       depends on !BR2_SSP_REGULAR
+       depends on !BR2_FORTIFY_SOURCE_1
+       help
+         This option enables a packet post install step that verifies
+         that the selected hardning options was actually used during
+         the build.
+
+comment "Verifying hardened build needs the annobin GCC plugin and it not compatible with the regular stack protector and the conservative buffer overflow protector"
+	 depends on !BR2_TOOLCHAIN_ANNOBIN_GCC_PLUGIN || BR2_SSP_REGULAR || BR2_FORTIFY_SOURCE_1
+
 endmenu
 
 source "toolchain/Config.in"
package/pkg-generic.mk +36 -0
index a303dc2e07..9567d260bd 100644
@@ -94,6 +94,42 @@ endef
 
 GLOBAL_INSTRUMENTATION_HOOKS += check_bin_arch
 
+ifeq ($(BR2_CHECK_HARDENING),y)
+# For now, no support for operator[] range check, control flow
+# enforcement, stack clash protection and control flow protection
+# hardening
+HARDENED_OPTS = -s operator -s cet -s clash -s cf
+
+ifneq ($(BR2_SSP_STRONG)$(BR2_SSP_ALL),y)
+HARDENED_OPTS += -s opt
+endif
+ifneq ($(BR2_OPTIMIZE_2)$(BR2_OPTIMIZE_3)$(BR2_OPTIMIZE_S),y)
+HARDENED_OPTS += -s opt
+endif
+ifneq ($(BR2_FORTIFY_SOURCE_2),y)
+HARDENED_OPTS += -s fort
+endif
+ifneq ($(BR2_RELRO_PARTIAL)$(BR2_RELRO_FULL),y)
+HARDENED_OPTS += -s relro
+endif
+ifneq ($(BR2_RELRO_FULL),y)
+HARDENED_OPTS += -s now -s pic
+endif
+
+define check_hardened
+	$(if $(filter end-install-target,$(1)-$(2)),\
+		support/scripts/check-hardened \
+			-p $(3) \
+			-l $(BUILD_DIR)/packages-file-list.txt \
+			$(foreach i,$($(PKG)_HARDENED_EXCLUDE),-i "$(i)") \
+			$(HARDENED_OPTS) \
+			-r $(TARGET_READELF) \
+			-h $(HARDENED_SH))
+endef
+
+GLOBAL_INSTRUMENTATION_HOOKS += check_hardened
+endif
+
 # This hook checks that host packages that need libraries that we build
 # have a proper DT_RPATH or DT_RUNPATH tag
 define check_host_rpath
support/scripts/check-hardened new file +74 -0

74 lines not on this page · raw message

Signature · 1 line
2.17.0
#9 lore raw

RFC PATCH 2/2 core: Verify that hardening flags are used

Quoted 24 lines · Stefan S?rensen

This patch add a new package post install check that verifies that configured hardening options are used.

Using the ELF notes added by the GCC annobin plugin, it verifies that the following build options are used:
* Stack protector
* RELRO
* FORTIFY_SOURCE
* Optimization
* Possition Independent Code/Executeable (-fPIC/-fPIE)

---
 Config.in                      | 15 +++++++
 package/pkg-generic.mk         | 36 +++++++++++++++++
 support/scripts/check-hardened | 74 ++++++++++++++++++++++++++++++++++
 3 files changed, 125 insertions(+)
 create mode 100755 support/scripts/check-hardened
Config.in +0 -0
index 6b5b2b043c..43fd15f3a2 100644

Config.in endchoice · line 833

Quoted 8 lines
Config.in +5 -0
@@ -826,6 +826,21 @@ endchoice
 
 comment "Fortify Source needs a glibc toolchain and optimization"
 	depends on (!BR2_TOOLCHAIN_USES_GLIBC || BR2_OPTIMIZE_0)
+
+
+config BR2_CHECK_HARDENING
+       bool "Verify hardened build"
+       depends on BR2_TOOLCHAIN_ANNOBIN_GCC_PLUGIN

select instead of depends (and then of course propagate the dependency from
host-annobin).

Config.in endchoice · line 835

Quoted 2 lines
Config.in +2 -0
+       depends on !BR2_SSP_REGULAR
+       depends on !BR2_FORTIFY_SOURCE_1

Well, it still works for the other options if SSP_REGULAR or FORTIFY_1 is
chosen, right? I don't think we need to specify these dependencies. At worst, nothing is checked at all, but that's fine as well I'd say.

Config.in endchoice · line 837

Quoted 2 lines
Config.in +2 -0
+       help
+         This option enables a packet post install step that verifies

package

Config.in endchoice · line 838

Quoted 1 line
Config.in +1 -0
+         that the selected hardning options was actually used during

hardening were

Config.in endchoice · line 842

Quoted 4 lines
Config.in +4 -0
+         the build.
+
+comment "Verifying hardened build needs the annobin GCC plugin and it not compatible with the regular stack protector and the conservative buffer overflow protector"
+	 depends on !BR2_TOOLCHAIN_ANNOBIN_GCC_PLUGIN || BR2_SSP_REGULAR || BR2_FORTIFY_SOURCE_1

Only a comment on the gcc version is needed.

Quoted 9 lines
Config.in +1 -0
+
 endmenu
 
 source "toolchain/Config.in"
package/pkg-generic.mk +0 -0
index a303dc2e07..9567d260bd 100644

package/pkg-generic.mk endef · line 100

Quoted 7 lines
package/pkg-generic.mk +4 -0
@@ -94,6 +94,42 @@ endef
 
 GLOBAL_INSTRUMENTATION_HOOKS += check_bin_arch
 
+ifeq ($(BR2_CHECK_HARDENING),y)
+# For now, no support for operator[] range check, control flow
+# enforcement, stack clash protection and control flow protection
+# hardening

Why not? [Because we don't build with those options, but mention that in the
comment.]

package/pkg-generic.mk endef · line 103

Quoted 3 lines
package/pkg-generic.mk +3 -0
+HARDENED_OPTS = -s operator -s cet -s clash -s cf
+
+ifneq ($(BR2_SSP_STRONG)$(BR2_SSP_ALL),y)

Nit: I think we'd prefer positive options here, so
ifeq ($(BR2_SSP_NONE)$(BR2_SSP_REGULAR),y)

That's more consistent with the fact that we're skipping.

package/pkg-generic.mk endef · line 104

Quoted 1 line
package/pkg-generic.mk +1 -0
+HARDENED_OPTS += -s opt

stack, I guess?

Quoted 10 lines
package/pkg-generic.mk +10 -0
+endif
+ifneq ($(BR2_OPTIMIZE_2)$(BR2_OPTIMIZE_3)$(BR2_OPTIMIZE_S),y)
+HARDENED_OPTS += -s opt
+endif
+ifneq ($(BR2_FORTIFY_SOURCE_2),y)
+HARDENED_OPTS += -s fort
+endif
+ifneq ($(BR2_RELRO_PARTIAL)$(BR2_RELRO_FULL),y)
+HARDENED_OPTS += -s relro
+endif

package/pkg-generic.mk endef · line 124

Quoted 10 lines
package/pkg-generic.mk +10 -0
+ifneq ($(BR2_RELRO_FULL),y)
+HARDENED_OPTS += -s now -s pic
+endif
+
+define check_hardened
+	$(if $(filter end-install-target,$(1)-$(2)),\
+		support/scripts/check-hardened \
+			-p $(3) \
+			-l $(BUILD_DIR)/packages-file-list.txt \
+			$(foreach i,$($(PKG)_HARDENED_EXCLUDE),-i "$(i)") \

So, we have a space-separated list, which is then converted into individual -i
options, which the script then collects in an array, which it finally iterates over in a for loop.

Wouldn't it be simpler to just pass -i '$($(PKG)_HARDENED_EXCLUDE)', and in the
script:

for ignore in $ignores; do

package/pkg-generic.mk endef · line 127

Quoted 3 lines
package/pkg-generic.mk +3 -0
+			$(HARDENED_OPTS) \
+			-r $(TARGET_READELF) \
+			-h $(HARDENED_SH))

Since HOST_DIR is already exported, the script could also just hardcode
$(HOST_DIR)/bin/hardened.sh.

Quoted 14 lines
package/pkg-generic.mk +5 -0
+endef
+
+GLOBAL_INSTRUMENTATION_HOOKS += check_hardened
+endif
+
 # This hook checks that host packages that need libraries that we build
 # have a proper DT_RPATH or DT_RUNPATH tag
 define check_host_rpath
support/scripts/check-hardened new file +0 -0
new file mode 100755
index 0000000000..8f4d6628cf

support/scripts/check-hardened line 3

Quoted 3 lines
support/scripts/check-hardened new file +3 -0
@@ -0,0 +1,74 @@
+#!/usr/bin/env bash
+
+# Heavily based on check-bin-arch

Hm, refactoring opportunity :-) But that can be done later.

Quoted 21 lines
support/scripts/check-hardened new file +21 -0
+
+# List of hardcoded paths that should be ignored, as they are
+# contain binaries for an architecture different from the
+# architecture of the target.
+declare -a IGNORES=(
+	# Skip firmware files, they could be ELF files for other
+	# architectures without hardening
+	"/lib/firmware"
+	"/usr/lib/firmware"
+
+	# Skip kernel modules
+	"/lib/modules"
+	"/usr/lib/modules"
+
+	# Skip files in /usr/share, several packages (qemu,
+	# pru-software-support) legitimately install ELF binaries that
+	# are not for the target architecture and are not hardened
+	"/usr/share"
+)
+
+declare -a skip

support/scripts/check-hardened line 34

Quoted 10 lines
support/scripts/check-hardened new file +10 -0
+
+while getopts p:l:h:r:i:s: OPT ; do
+	case "${OPT}" in
+	p) package="${OPTARG}";;
+	l) pkg_list="${OPTARG}";;
+	h) hardened="${OPTARG}";;
+	i)
+		# Ensure we do have single '/' as separators,
+		# and that we have a leading one.
+		pattern="$(sed -r -e 's:/+:/:g; s:^/*:/:;' <<<"${OPTARG}")"

This could move into the loop (needed in case you follow my suggestion of a
single -i option).

support/scripts/check-hardened line 38

Quoted 4 lines
support/scripts/check-hardened new file +4 -0
+		IGNORES+=("${pattern}")
+		;;
+	r) readelf="${OPTARG}";;
+	s) skip+=("--skip=${OPTARG}");;

The short form of --skip in the hardened.sh script is -k, so it would be
logical to use the same letter for this script.

support/scripts/check-hardened line 50

Quoted 12 lines
support/scripts/check-hardened new file +12 -0
+	:) error "option '%s' expects a mandatory argument\n" "${OPTARG}";;
+	\?) error "unknown option '%s'\n" "${OPTARG}";;
+	esac
+done
+
+if test -z "${package}" -o -z "${pkg_list}" -o -z "${hardened}" ; then
+	echo "Usage: $0 -p <pkg> -l <pkg-file-list> -h <hardened> -r <readelf> [-i PATH ...]"
+	exit 1
+fi
+
+if [ ! -e ${hardened} ]; then
+	exit 0

Do we want that? We should fail hard if it doesn't exist, no?

Regards,
Arnout

Quoted 25 lines
support/scripts/check-hardened new file +24 -0
+fi
+
+exitcode=0
+
+# Only split on new lines, for filenames-with-spaces
+IFS="
+"
+
+while read f; do
+	for ignore in "${IGNORES[@]}"; do
+		if [[ "${f}" =~ ^"${ignore}" ]]; then
+			continue 2
+		fi
+	done
+
+	# Only check regular files
+	if [[ ! -f "${TARGET_DIR}/${f}" ]]; then
+		continue
+	fi
+
+	${hardened} --readelf=${readelf} --ignore-unknown ${skip[*]} ${TARGET_DIR}${f} || exitcode=1
+done < <( sed -r -e "/^${package},\.(.+)$/!d; s//\1/;" ${pkg_list} )
+
+exit ${exitcode}
Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF