objtool: fix libelf detection in non-standard locations

PATCH In review Robin Jarry 4 participants 20 messages Feb 23, 2018 – Apr 5, 2018

Subsystems (3)

Documentation Kbuild objtool

lore

Share & tools

lore

Robin Jarry

author

2018-02-23 14:22 UTC #0 lore raw

cover objtool: fix libelf detection in non-standard locations

When libelf headers and libs are not in standard locations, the check fails and host flags are ignored.

I have encountered this issue when building linux into the buildroot framework. host-* versions of libs may be compiled and installed in a host staging dir removing the need to install them on the build system.

Comments are welcome.

Regards,

Robin Jarry (2):
kbuild: introduce HOST_EXTRALDFLAGS
objtool: use host flags to detect if libelf is missing

 Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Signature · 1 line

2.11.0

Robin Jarry

author

2018-02-23 14:22 UTC #1 lore raw

PATCH 1/2 kbuild: introduce HOST_EXTRALDFLAGS

Add a new user specifiable variable to provide specific linker options (-L, -rpath, ...) for host programs build.

Append these options to HOSTLDFLAGS.

Signed-off-by Robin Jarry <[email protected]>

Cc Josh Poimboeuf <[email protected]>

Cc Ingo Molnar <[email protected]>

---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

Makefile +1 -0

index d9cf3a40eda9..e9609319d2b8 100644
@@ -365,6 +365,7 @@ HOSTCFLAGS   := -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 \
 		-fomit-frame-pointer -std=gnu89 $(HOST_LFS_CFLAGS)
 HOSTCXXFLAGS := -O2 $(HOST_LFS_CFLAGS)
 HOSTLDFLAGS  := $(HOST_LFS_LDFLAGS)
+HOSTLDFLAGS  += $(HOST_EXTRALDFLAGS)
 HOST_LOADLIBES := $(HOST_LFS_LIBS)
 
 # Make variables (CC, etc...)

Signature · 1 line

2.11.0

Robin Jarry

author

2018-02-23 14:23 UTC #2 lore raw

PATCH 2/2 objtool: use host flags to detect if libelf is missing

In reply to Robin Jarry

When libelf headers and libs are not found and CONFIG_UNWINDER_ORC is selected, there is a make error introduced by
commit 3dd40cb320fe ("objtool: Upgrade libelf-devel warning to error for CONFIG_ORC_UNWINDER").

Host headers and libs may be in a non-standard location and the check is performed without host flags. Make sure to use host flags for the check (they will be used later when building objtool anyway).

Fixes 3b27a0c85d70 ("objtool: Detect and warn if libelf is missing and don't break the build")

Signed-off-by Robin Jarry <[email protected]>

Cc Josh Poimboeuf <[email protected]>

Cc Ingo Molnar <[email protected]>

---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Makefile +1 -1

index e9609319d2b8..4e8c667b9f3f 100644
@@ -955,7 +955,7 @@ export mod_sign_cmd
 
 ifdef CONFIG_STACK_VALIDATION
   has_libelf := $(call try-run,\
-		echo "int main() {}" | $(HOSTCC) -xc -o /dev/null -lelf -,1,0)
+		echo "int main() {}" | $(HOSTCC) $(host_clfags) $(HOSTLDFLAGS) -xc -o /dev/null -lelf -,1,0)
   ifeq ($(has_libelf),1)
     objtool_target := tools/objtool FORCE
   else

Signature · 1 line

2.11.0

Josh Poimboeuf

2018-02-23 15:37 UTC #3 lore raw

Re: [PATCH 2/2] objtool: use host flags to detect if libelf is missing

Quoted 33 lines · Robin Jarry

When libelf headers and libs are not found and CONFIG_UNWINDER_ORC is selected, there is a make error introduced by
commit 3dd40cb320fe ("objtool: Upgrade libelf-devel warning to error for CONFIG_ORC_UNWINDER").
Host headers and libs may be in a non-standard location and the check is performed without host flags. Make sure to use host flags for the check (they will be used later when building objtool anyway).
Fixes 3b27a0c85d70 ("objtool: Detect and warn if libelf is missing and don't break the build")
Signed-off-by Robin Jarry <[email protected]>
Cc Josh Poimboeuf <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Makefile +1 -1
index e9609319d2b8..4e8c667b9f3f 100644
@@ -955,7 +955,7 @@ export mod_sign_cmd
 
 ifdef CONFIG_STACK_VALIDATION
   has_libelf := $(call try-run,\
-		echo "int main() {}" | $(HOSTCC) -xc -o /dev/null -lelf -,1,0)
+		echo "int main() {}" | $(HOSTCC) $(host_clfags) $(HOSTLDFLAGS) -xc -o /dev/null -lelf -,1,0)
   ifeq ($(has_libelf),1)
     objtool_target := tools/objtool FORCE
   else
Signature · 1 line
2.11.0

Hi Robin,

Did you test this? I see some issues:

a) I don't see where the host_cflags variable comes from (and btw, it's misspelled...)

b) I don't see where objtool uses HOSTLDFLAGS.

Signature · 1 line

Josh

Robin Jarry

author

2018-02-23 16:23 UTC #4 lore raw

Re: [PATCH 2/2] objtool: use host flags to detect if libelf is missing

2018-02-23, Josh Poimboeuf:

Quoted 6 lines

Hi Robin,
Did you test this? I see some issues:
a) I don't see where the host_cflags variable comes from (and btw, it's misspelled...)

Hi Josh,

Shame on me... Last minute fix. I had tested this first by forcing HOSTCC="gcc <CFLAGS> <LDFLAGS>" but I found it rather dirty. As it turns out, my patch is not better.

Quoted 1 line

b) I don't see where objtool uses HOSTLDFLAGS.

Indeed, I also missed this (see above).

It looks like objtool does not use HOSTCFLAGS nor HOSTLDFLAGS (defined in top Makefile). Do you see a reason why? Maybe it would be a good idea to share these flags (adding some specific ones for objtool, if needed).

What do you think?

Signature · 1 line

Robin

Josh Poimboeuf

2018-02-23 16:34 UTC #5 lore raw

Re: [PATCH 2/2] objtool: use host flags to detect if libelf is missing

Quoted 16 lines · Robin Jarry

2018-02-23, Josh Poimboeuf:
Quoted 6 lines
Hi Robin,
Did you test this? I see some issues:
a) I don't see where the host_cflags variable comes from (and btw, it's misspelled...)
Hi Josh,
Shame on me... Last minute fix. I had tested this first by forcing HOSTCC="gcc <CFLAGS> <LDFLAGS>" but I found it rather dirty. As it turns out, my patch is not better.
Quoted 1 line
b) I don't see where objtool uses HOSTLDFLAGS.

Quoted 7 lines

Indeed, I also missed this (see above).
It looks like objtool does not use HOSTCFLAGS nor HOSTLDFLAGS (defined in top Makefile). Do you see a reason why? Maybe it would be a good idea to share these flags (adding some specific ones for objtool, if needed).
What do you think?

Yeah, I think it would be a good idea for objtool to use HOSTCC, HOSTCFLAGS, and HOSTLDFLAGS.

Signature · 1 line

Josh

Robin Jarry

author

2018-02-26 18:42 UTC #6 lore raw

v2 cover kbuild: fix host progs build with libs in non standard locations

In reply to Robin Jarry

This patchset allows to build host programs that depend on external libs installed in non standard locations (i.e. not in /usr/include, /usr/lib, etc.). For now, the only way is to force HOSTCC to include both the path to the host compiler and the build flags.

I'm not really satisfied with the new HOST_{C,LD}FLAGS variables. They are too similar to HOST{C,LD}FLAGS and I find them confusing. However, HOST_EXTRA*FLAGS are already reserved for local use in makefiles (see Documentation/kbuild/makefiles.txt). And I didn't want to have even longer USER_HOST_*FLAGS. If someone has a better proposition, I'll happily make a v3.

Comments are welcome.

Changes since v1:
- Fix typos.
- Do not use HOST_EXTRA*FLAGS. Add new user specifiable variables.
- Pass HOSTLDFLAGS to build single .c programs.
- Build objtool with host flags.

Robin Jarry (3):
kbuild: introduce HOST_{C,LD}FLAGS
kbuild: use HOSTLDFLAGS for single .c executables
objtool: use global host flags for compilation

 Documentation/admin-guide/README.rst | 16 ++++++++++++++++
 Makefile                             |  9 +++++----
 scripts/Makefile.host                |  2 +-
 tools/objtool/Makefile               |  4 ++--
 4 files changed, 24 insertions(+), 7 deletions(-)

Signature · 1 line

2.11.0

Robin Jarry

author

2018-02-26 18:42 UTC #7 lore raw

PATCH v2 1/3 kbuild: introduce HOST_{C,LD}FLAGS

Add user-specifiable compiler/linker flags for host programs. Append these to already existing HOST{C,CXX,LD}FLAGS.

Update documentation accordingly.

Signed-off-by Robin Jarry <[email protected]>

Cc Josh Poimboeuf <[email protected]>

Cc Ingo Molnar <[email protected]>

---
 Documentation/admin-guide/README.rst | 16 ++++++++++++++++
 Makefile                             |  7 ++++---
 2 files changed, 20 insertions(+), 3 deletions(-)

Documentation/admin-guide/README.rst +16 -0

index af5a437198d0..22ba3140e85f 100644
@@ -271,6 +271,22 @@ Compiling the kernel
    To have the build system also tell the reason for the rebuild of each
    target, use ``V=2``.  The default is ``V=0``.
 
+ - Host libs in non standard locations:
+
+   Some host programs are linked with external libraries, such as openssl
+   or libelf.  If these libs are installed in non standard locations on
+   your build system, you may specify it via the ``HOST_CFLAGS`` and
+   ``HOST_LDFLAGS`` parameters on the ``make`` command line.
+
+   For example, if you have installed host libs in ``/foo/staging``, you
+   may use the following command::
+
+     make HOST_CFLAGS="-I/foo/staging/include" \
+          HOST_LDFLAGS="-L/foo/staging/lib -Wl,-rpath=/foo/staging/lib"
+
+   Please note: If the ``HOST_CFLAGS`` or ``HOST_LDFLAGS`` options are
+   used, then they must be used for all invocations of make.
+
  - Keep a backup kernel handy in case something goes wrong.  This is
    especially true for the development releases, since each new release
    contains new code which has not been debugged.  Make sure you keep a

Makefile +4 -3

index 659a7780aeb3..f7c543cebfc6 100644
@@ -362,9 +362,10 @@ HOST_LFS_LIBS := $(shell getconf LFS_LIBS)
 HOSTCC       = gcc
 HOSTCXX      = g++
 HOSTCFLAGS   := -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 \
-		-fomit-frame-pointer -std=gnu89 $(HOST_LFS_CFLAGS)
-HOSTCXXFLAGS := -O2 $(HOST_LFS_CFLAGS)
-HOSTLDFLAGS  := $(HOST_LFS_LDFLAGS)
+		-fomit-frame-pointer -std=gnu89 $(HOST_LFS_CFLAGS) \
+		$(HOST_CFLAGS)
+HOSTCXXFLAGS := -O2 $(HOST_LFS_CFLAGS) $(HOST_CFLAGS)
+HOSTLDFLAGS  := $(HOST_LFS_LDFLAGS) $(HOST_LDFLAGS)
 HOST_LOADLIBES := $(HOST_LFS_LIBS)
 
 # Make variables (CC, etc...)

Signature · 1 line

2.11.0

Robin Jarry

author

2018-02-26 18:42 UTC #8 lore raw

PATCH v2 2/3 kbuild: use HOSTLDFLAGS for single .c executables

In reply to Robin Jarry

When compiling executables from a single .c file, the linker is also invoked. Pass the HOSTLDFLAGS like for other linker commands.

Signed-off-by Robin Jarry <[email protected]>

Cc Josh Poimboeuf <[email protected]>

Cc Ingo Molnar <[email protected]>

---
 scripts/Makefile.host | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

scripts/Makefile.host +1 -1

index e6dc6ae2d7c4..aa971cc3f339 100644
@@ -84,7 +84,7 @@ hostcxx_flags  = -Wp,-MD,$(depfile) $(__hostcxx_flags)
 # Create executable from a single .c file
 # host-csingle -> Executable
 quiet_cmd_host-csingle 	= HOSTCC  $@
-      cmd_host-csingle	= $(HOSTCC) $(hostc_flags) -o $@ $< \
+      cmd_host-csingle	= $(HOSTCC) $(hostc_flags) $(HOSTLDFLAGS) -o $@ $< \
 	  	$(HOST_LOADLIBES) $(HOSTLOADLIBES_$(@F))
 $(host-csingle): $(obj)/%: $(src)/%.c FORCE
 	$(call if_changed_dep,host-csingle)

Signature · 1 line

2.11.0

Masahiro Yamada

2018-04-05 14:24 UTC #9 lore raw

Re: [PATCH v2 2/3] kbuild: use HOSTLDFLAGS for single .c executables

2018-02-27 3:41 GMT+09:00 Robin Jarry <[email protected]>:

Quoted 15 lines

When compiling executables from a single .c file, the linker is also invoked. Pass the HOSTLDFLAGS like for other linker commands.
Signed-off-by Robin Jarry <[email protected]>
Cc Josh Poimboeuf <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 scripts/Makefile.host | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/Makefile.host +0 -0
index e6dc6ae2d7c4..aa971cc3f339 100644

scripts/Makefile.host hostcxx_flags = -Wp,-MD,$(depfile) $(__hostcxx_flags) · line 90

Quoted 8 lines

scripts/Makefile.host +1 -1

@@ -84,7 +84,7 @@ hostcxx_flags  = -Wp,-MD,$(depfile) $(__hostcxx_flags)
 # Create executable from a single .c file
 # host-csingle -> Executable
 quiet_cmd_host-csingle         = HOSTCC  $@
-      cmd_host-csingle = $(HOSTCC) $(hostc_flags) -o $@ $< \
+      cmd_host-csingle = $(HOSTCC) $(hostc_flags) $(HOSTLDFLAGS) -o $@ $< \
                $(HOST_LOADLIBES) $(HOSTLOADLIBES_$(@F))
 $(host-csingle): $(obj)/%: $(src)/%.c FORCE
        $(call if_changed_dep,host-csingle)

This one is correct anyway.

I applied to linux-kbuild.

Signature · 2 lines

Best Regards
Masahiro Yamada

Robin Jarry

author

2018-02-26 18:42 UTC #10 lore raw

PATCH v2 3/3 objtool: use global host flags for compilation

In reply to Robin Jarry

Host headers and libs may be in a non-standard location and the check is performed without host flags. Make sure to use host flags for the check and for building objtool. Remove duplicate flags.

Fixes 3b27a0c85d70 ("objtool: Detect and warn if libelf is missing and don't break the build")

Signed-off-by Robin Jarry <[email protected]>

Cc Josh Poimboeuf <[email protected]>

Cc Ingo Molnar <[email protected]>

---
 Makefile               | 2 +-
 tools/objtool/Makefile | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

Makefile +1 -1

index f7c543cebfc6..c9dc35fe0e48 100644
@@ -955,7 +955,7 @@ export mod_sign_cmd
 
 ifdef CONFIG_STACK_VALIDATION
   has_libelf := $(call try-run,\
-		echo "int main() {}" | $(HOSTCC) -xc -o /dev/null -lelf -,1,0)
+	echo "int main() {}" | $(HOSTCC) $(HOSTLDFLAGS) -xc -o /dev/null -lelf -,1,0)
   ifeq ($(has_libelf),1)
     objtool_target := tools/objtool FORCE
   else

tools/objtool/Makefile +2 -2

index e6acc281dd37..4d4b810313e2 100644
@@ -31,8 +31,8 @@ INCLUDES := -I$(srctree)/tools/include \
 	    -I$(srctree)/tools/arch/$(HOSTARCH)/include/uapi \
 	    -I$(srctree)/tools/objtool/arch/$(ARCH)/include
 WARNINGS := $(EXTRA_WARNINGS) -Wno-switch-default -Wno-switch-enum -Wno-packed
-CFLAGS   += -Wall -Werror $(WARNINGS) -fomit-frame-pointer -O2 -g $(INCLUDES)
-LDFLAGS  += -lelf $(LIBSUBCMD)
+CFLAGS   += $(HOSTCFLAGS) -Werror $(WARNINGS) -g $(INCLUDES)
+LDFLAGS  += $(HOSTLDFLAGS) -lelf $(LIBSUBCMD)
 
 # Allow old libelf to be used:
 elfshdr := $(shell echo '\#include <libelf.h>' | $(CC) $(CFLAGS) -x c -E - | grep elf_getshdr)

Signature · 1 line

2.11.0

Josh Poimboeuf

2018-02-27 17:49 UTC #11 lore raw

Re: [PATCH v2 3/3] objtool: use global host flags for compilation

Quoted 23 lines · Robin Jarry

When libelf headers and libs are not found and CONFIG_UNWINDER_ORC is selected, there is a make error introduced by
commit 3dd40cb320fe ("objtool: Upgrade libelf-devel warning to error for CONFIG_ORC_UNWINDER").
Host headers and libs may be in a non-standard location and the check is performed without host flags. Make sure to use host flags for the check and for building objtool. Remove duplicate flags.
Fixes 3b27a0c85d70 ("objtool: Detect and warn if libelf is missing and don't break the build")
Signed-off-by Robin Jarry <[email protected]>
Cc Josh Poimboeuf <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 Makefile               | 2 +-
 tools/objtool/Makefile | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)
Makefile +0 -0
index f7c543cebfc6..c9dc35fe0e48 100644

Makefile export mod_sign_cmd · line 958

Quoted 5 lines

Makefile +1 -1

@@ -955,7 +955,7 @@ export mod_sign_cmd
 
 ifdef CONFIG_STACK_VALIDATION
   has_libelf := $(call try-run,\
-		echo "int main() {}" | $(HOSTCC) -xc -o /dev/null -lelf -,1,0)
+	echo "int main() {}" | $(HOSTCC) $(HOSTLDFLAGS) -xc -o /dev/null -lelf -,1,0)

This should probably also have HOSTCFLAGS, since the objtool makefile uses it as well.

Signature · 1 line

Josh

Robin Jarry

author

2018-02-27 20:55 UTC #12 lore raw

Re: [PATCH v2 3/3] objtool: use global host flags for compilation

2018-02-27, Josh Poimboeuf:

Quoted 1 line · Robin Jarry

[snip]

Quoted 7 lines

Makefile export mod_sign_cmd · line 958
Quoted 4 lines
ifdef CONFIG_STACK_VALIDATION
has_libelf := $(call try-run,\
Makefile +1 -1
-		echo "int main() {}" | $(HOSTCC) -xc -o /dev/null -lelf -,1,0)
+	echo "int main() {}" | $(HOSTCC) $(HOSTLDFLAGS) -xc -o /dev/null -lelf -,1,0)
This should probably also have HOSTCFLAGS, since the objtool makefile uses it as well.

I had intentionally omitted HOSTCFLAGS since the compiled code does not contain any #include statements (only -lelf). But adding them cannot hurt.

Signature · 1 line

Robin

Josh Poimboeuf

2018-02-27 21:41 UTC #13 lore raw

Re: [PATCH v2 3/3] objtool: use global host flags for compilation

Quoted 11 lines · Robin Jarry

2018-02-27, Josh Poimboeuf:
Quoted 1 line · Robin Jarry
[snip]
Quoted 7 lines
Makefile export mod_sign_cmd · line 958
Quoted 4 lines
ifdef CONFIG_STACK_VALIDATION
has_libelf := $(call try-run,\
Makefile +1 -1
-		echo "int main() {}" | $(HOSTCC) -xc -o /dev/null -lelf -,1,0)
+	echo "int main() {}" | $(HOSTCC) $(HOSTLDFLAGS) -xc -o /dev/null -lelf -,1,0)
This should probably also have HOSTCFLAGS, since the objtool makefile uses it as well.

Quoted 3 lines

I had intentionally omitted HOSTCFLAGS since the compiled code does not contain any #include statements (only -lelf). But adding them cannot hurt.

Ah, right. The point of this check is to see if it can find libelf, so I guess using HOSTCFLAGS wouldn't have a point here. So it's fine the way it is.

Signature · 1 line

Josh

Josh Poimboeuf

2018-02-27 17:45 UTC #14 lore raw

Re: [PATCH v2 0/3] kbuild: fix host progs build with libs in non standard locations

In reply to Robin Jarry

Quoted 9 lines · Robin Jarry

This patchset allows to build host programs that depend on external libs installed in non standard locations (i.e. not in /usr/include, /usr/lib, etc.). For now, the only way is to force HOSTCC to include both the path to the host compiler and the build flags.
I have encountered this issue when building linux into the buildroot framework. host-* versions of libs may be compiled and installed in a host staging dir removing the need to install them on the build system.

Quoted 6 lines

I'm not really satisfied with the new HOST_{C,LD}FLAGS variables. They are too similar to HOST{C,LD}FLAGS and I find them confusing. However, HOST_EXTRA*FLAGS are already reserved for local use in makefiles (see Documentation/kbuild/makefiles.txt). And I didn't want to have even longer USER_HOST_*FLAGS. If someone has a better proposition, I'll happily make a v3.

In Documentation/kbuild/kbuild.txt, we have the following environment variables:

KCFLAGS
--------------------------------------------------
Additional options to the C compiler (for built-in and modules).

CFLAGS_KERNEL
--------------------------------------------------
Additional options for $(CC) when used to compile code that is compiled as built-in.

  CFLAGS_MODULE
  --------------------------------------------------
  Additional module specific options to use for $(CC).
  
  LDFLAGS_MODULE
  --------------------------------------------------
  Additional options used for $(LD) when linking modules.

LDFLAGS_vmlinux
--------------------------------------------------
Additional options passed to final link of vmlinux.

So instead of

HOST_CFLAGS
HOST_LDFLAGS

maybe it would be more consistent to call them

CFLAGS_HOST
LDFLAGS_HOST

Also, the new environment variables should be documented in the above file.

Signature · 1 line

Josh

Robin Jarry

author

2018-02-27 20:52 UTC #15 lore raw

Re: [PATCH v2 0/3] kbuild: fix host progs build with libs in non standard locations

2018-02-27, Josh Poimboeuf:

Quoted 29 lines

In Documentation/kbuild/kbuild.txt, we have the following environment variables:
KCFLAGS
--------------------------------------------------
Additional options to the C compiler (for built-in and modules).
CFLAGS_KERNEL
--------------------------------------------------
Additional options for $(CC) when used to compile code that is compiled as built-in.
  CFLAGS_MODULE
  --------------------------------------------------
  Additional module specific options to use for $(CC).
  
  LDFLAGS_MODULE
  --------------------------------------------------
  Additional options used for $(LD) when linking modules.
LDFLAGS_vmlinux
--------------------------------------------------
Additional options passed to final link of vmlinux.
So instead of
HOST_CFLAGS
HOST_LDFLAGS

Quoted 6 lines

maybe it would be more consistent to call them
CFLAGS_HOST
LDFLAGS_HOST
?

I had missed this file. Indeed {C,LD}FLAGS_HOST are much less confusing!

Quoted 2 lines

Also, the new environment variables should be documented in the above file.

Sure, I'll do that. However, I feel like I should also leave the paragraph in Documentation/admin-guide/README.txt.

What do you think?

Signature · 1 line

Robin

Josh Poimboeuf

2018-02-27 21:38 UTC #16 lore raw

Re: [PATCH v2 0/3] kbuild: fix host progs build with libs in non standard locations

Quoted 39 lines · Robin Jarry

2018-02-27, Josh Poimboeuf:
Quoted 29 lines
In Documentation/kbuild/kbuild.txt, we have the following environment variables:
KCFLAGS
--------------------------------------------------
Additional options to the C compiler (for built-in and modules).
CFLAGS_KERNEL
--------------------------------------------------
Additional options for $(CC) when used to compile code that is compiled as built-in.
  CFLAGS_MODULE
  --------------------------------------------------
  Additional module specific options to use for $(CC).
  
  LDFLAGS_MODULE
  --------------------------------------------------
  Additional options used for $(LD) when linking modules.
LDFLAGS_vmlinux
--------------------------------------------------
Additional options passed to final link of vmlinux.
So instead of
HOST_CFLAGS
HOST_LDFLAGS
Quoted 6 lines
maybe it would be more consistent to call them
CFLAGS_HOST
LDFLAGS_HOST
?
I had missed this file. Indeed {C,LD}FLAGS_HOST are much less confusing!

Quoted 7 lines

Quoted 2 lines
Also, the new environment variables should be documented in the above file.
Sure, I'll do that. However, I feel like I should also leave the paragraph in Documentation/admin-guide/README.txt.
What do you think?

[ Adding Jon Corbet to CC in case he has an opinion. ]

I think that paragraph in the admin guide seems a bit out of place, because that file only seems to cover the most common cases.

No other env variables are described in that file (and some of the previously existing env variables are much more likely to be used than these new ones anyway).

It might be a good idea to instead just add a reference to kbuild.txt, suggesting the user refer to that file if they want to learn more about what env variables can be used to customize the build.

Signature · 1 line

Josh

Randy Dunlap

reviewer

2018-02-28 00:23 UTC #17 lore raw

Re: [PATCH v2 0/3] kbuild: fix host progs build with libs in non standard locations

Quoted 53 lines · Josh Poimboeuf

Quoted 39 lines · Robin Jarry
2018-02-27, Josh Poimboeuf:
Quoted 29 lines
In Documentation/kbuild/kbuild.txt, we have the following environment variables:
KCFLAGS
--------------------------------------------------
Additional options to the C compiler (for built-in and modules).
CFLAGS_KERNEL
--------------------------------------------------
Additional options for $(CC) when used to compile code that is compiled as built-in.
  CFLAGS_MODULE
  --------------------------------------------------
  Additional module specific options to use for $(CC).
  
  LDFLAGS_MODULE
  --------------------------------------------------
  Additional options used for $(LD) when linking modules.
LDFLAGS_vmlinux
--------------------------------------------------
Additional options passed to final link of vmlinux.
So instead of
HOST_CFLAGS
HOST_LDFLAGS
Quoted 6 lines
maybe it would be more consistent to call them
CFLAGS_HOST
LDFLAGS_HOST
?
I had missed this file. Indeed {C,LD}FLAGS_HOST are much less confusing!
Quoted 7 lines
Quoted 2 lines
Also, the new environment variables should be documented in the above file.
Sure, I'll do that. However, I feel like I should also leave the paragraph in Documentation/admin-guide/README.txt.
What do you think?
[ Adding Jon Corbet to CC in case he has an opinion. ]
I think that paragraph in the admin guide seems a bit out of place, because that file only seems to cover the most common cases.

Quoted 7 lines

No other env variables are described in that file (and some of the previously existing env variables are much more likely to be used than these new ones anyway).
It might be a good idea to instead just add a reference to kbuild.txt, suggesting the user refer to that file if they want to learn more about what env variables can be used to customize the build.

(IMHO :) I agree, the addition to admin-guide/README.rst (not .txt) is in the wrong place, although it could contain a pointer to kbuild/ like Josh says.

And if you do move that text to kbuild/, please change "non standard" to "non-standard" in 2 places.

thanks,

Signature · 1 line

~Randy

Masahiro Yamada

2018-02-28 10:23 UTC #18 lore raw

Re: [PATCH v2 0/3] kbuild: fix host progs build with libs in non standard locations

In reply to Josh Poimboeuf

2018-02-28 2:45 GMT+09:00 Josh Poimboeuf <[email protected]>:

Quoted 51 lines · Robin Jarry

Quoted 9 lines
This patchset allows to build host programs that depend on external libs installed in non standard locations (i.e. not in /usr/include, /usr/lib, etc.). For now, the only way is to force HOSTCC to include both the path to the host compiler and the build flags.
I have encountered this issue when building linux into the buildroot framework. host-* versions of libs may be compiled and installed in a host staging dir removing the need to install them on the build system.
Quoted 6 lines
I'm not really satisfied with the new HOST_{C,LD}FLAGS variables. They are too similar to HOST{C,LD}FLAGS and I find them confusing. However, HOST_EXTRA*FLAGS are already reserved for local use in makefiles (see Documentation/kbuild/makefiles.txt). And I didn't want to have even longer USER_HOST_*FLAGS. If someone has a better proposition, I'll happily make a v3.
In Documentation/kbuild/kbuild.txt, we have the following environment variables:
KCFLAGS
--------------------------------------------------
Additional options to the C compiler (for built-in and modules).
CFLAGS_KERNEL
--------------------------------------------------
Additional options for $(CC) when used to compile code that is compiled as built-in.
  CFLAGS_MODULE
  --------------------------------------------------
  Additional module specific options to use for $(CC).

  LDFLAGS_MODULE
  --------------------------------------------------
  Additional options used for $(LD) when linking modules.
LDFLAGS_vmlinux
--------------------------------------------------
Additional options passed to final link of vmlinux.
So instead of
HOST_CFLAGS
HOST_LDFLAGS
maybe it would be more consistent to call them
CFLAGS_HOST
LDFLAGS_HOST

Quoted 7 lines

?
Also, the new environment variables should be documented in the above file.
--
Josh

A generic rule I see is almost like this:

[1] "KBUILD_" + (Executable Name) + "FLAGS" = (Internal-use Variable)

[2] (Executable Name) + "FLAGS" = (User Interface via Command Line)

They also derive

[3] (Internal-use Variable) =
"KBUILD_" + (User Interface via Command Line)

The following is the current situation:

[1] Flags for $(CC)

Internal use User interface via command line
---------------------------------------------------------------------------

(common)   KBUILD_CFLAGS                     KCFLAGS
(builtin)  KBUILD_CFLAGS_KERNEL              CFLAGS_KERNEL
(module)   KBUILD_CFLAGS_MODULE              CFLAGS_MODULE

[2] Flags for $(AS)

Internal use User interface via command line
--------------------------------------------------------------------------

(common)   KBUILD_AFLAGS                      KAFLAGS
(builtin)  KBUILD_AFLAGS_KERNEL               AFLAGS_KERNEL
(module)   KBUILD_AFLAGS_MODULE               AFLAGS_MODULE

[3] Flags for $(CPP)

Internal use User interface via command line
--------------------------------------------------------------------------
(common) KBUILD_CPPFLAGS KCPPFLAGS

[4] Flags for $(LD)

Internal use User interface via command line
--------------------------------------------------------------------------

(builtin) [None] #1 [None]
(module) KBUILD_LDFLAGS_MODULE LDFLAGS_MODULE

#1 (LDFLAGS_vmlinux is used for internal variable for builtin)

[5] Flags for HOSTCC, HOSTCXX

Internal use User interface via command line
-----------------------------------------------------------------------------

C             HOSTCFLAGS                          [None]
C++           HOSTCXXFLAGS                        [None]
Link          HOSTLDFLAGS                         [None]
Library       HOST_LOADLIBES                      [None]

KCFLAGS, KAFLAGS, KCPPFLAGS are exceptions.
[5] is also an exception.

A consistent way could be

[5] Flags for HOSTCC, HOSTCXX

Internal use User interface via command line
-----------------------------------------------------------------------------

C           KBUILD_HOSTCFLAGS                     HOSTCFLAGS
C++         KBUILD_HOSTCXXFLAGS                   HOSTCXXFLAGS
Link        KBUILD_HOSTLDFLAGS                    HOSTLDFLAGS
Lib         KBUILD_HOSTLDLIBS                     HOSTLDLIBS

Documentation/kbuild/kbuild.txt lists user interface parameters. It is difficult to change them. (probably, not allowed)

Documentation/kbuild/makefiles.txt lists Make variables used in kernel sources. So, it is easier to rename them.

Only the problem would be external modules that compile their own host-programs with own flags. It is rare. Anyway, external modules are often broken
when the kernel version is updated.

So, my idea is to rename existing

 HOSTCFLAGS     ->  KBUILD_HOSTCFLAGS
 HOSTCXXFLAGS   ->  KBUILD_HOSTCXXFLAGS
 HOSTLDFLAGS    ->  KBUILD_HOSTLDFLAGS
 HOST_LOADLIBES ->  KBUILD_HOSTLDLIBS

("LOADLIBES" is too long, so rename it to "LDLIBS")

Then, re-add no-prefix ones as user interface.

Signature · 2 lines

Best Regards
Masahiro Yamada

Robin Jarry

author

2018-02-28 17:24 UTC #19 lore raw

Re: [PATCH v2 0/3] kbuild: fix host progs build with libs in non standard locations

2018-02-28, Masahiro Yamada:
[snip]

Quoted 8 lines

So, my idea is to rename existing

 HOSTCFLAGS     ->  KBUILD_HOSTCFLAGS
 HOSTCXXFLAGS   ->  KBUILD_HOSTCXXFLAGS
 HOSTLDFLAGS    ->  KBUILD_HOSTLDFLAGS
 HOST_LOADLIBES ->  KBUILD_HOSTLDLIBS

Quoted 4 lines

("LOADLIBES" is too long, so rename it to "LDLIBS")
Then, re-add no-prefix ones as user interface.

Hi Masahiro,

Ok, I'll add the rename patch in a v3 of my series.

Signature · 1 line

Robin