linux-next: build failure after merge of the tip tree

PATCH In review Stephen Rothwell 6 participants 26 messages Mar 1, 2016 – Mar 3, 2016

Subsystems (5)

x86 Kbuild Linux-next tree perf events +1 more subsystems

lore

Share & tools

lore

Stephen Rothwell

author

2016-03-01 01:29 UTC #0 lore raw

Hi all,

After merging the tip tree, today's linux-next build (x86_64 allmodconfig) failed like this:

  DESCEND  objtool
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/builtin-check.o
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/special.o
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/elf.o
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/objtool.o
  MKDIR    /home/sfr/next/x86_64_allmodconfig/tools/objtool/arch/x86/insn/
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/libstring.o

elf.c:22:23: fatal error: sys/types.h: No such file or directory compilation terminated.
CC /home/sfr/next/x86_64_allmodconfig/tools/objtool/exec-cmd.o
CC /home/sfr/next/x86_64_allmodconfig/tools/objtool/help.o
builtin-check.c:28:20: fatal error: string.h: No such file or directory compilation terminated.
objtool.c:28:19: fatal error: stdio.h: No such file or directory compilation terminated.

and further errors ...

This build is done with a PowerPC hosted cross compiler with no glibc. I assume that some things here need to be built with HOSTCC?

Presumably caused by commit

9e54249679b4 ("Merge branch 'core/objtool'")

and the commit series that was merged.

I tried to revert that merge, but it had conflicts, so I just used the tip tree from next-20160229 for today.

Signature · 2 lines

Cheers,
Stephen Rothwell

Ingo Molnar

2016-03-01 07:07 UTC #1 lore raw

Quoted 21 lines · Stephen Rothwell

Hi all,
After merging the tip tree, today's linux-next build (x86_64 allmodconfig) failed like this:
  DESCEND  objtool
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/builtin-check.o
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/special.o
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/elf.o
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/objtool.o
  MKDIR    /home/sfr/next/x86_64_allmodconfig/tools/objtool/arch/x86/insn/
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/libstring.o
elf.c:22:23: fatal error: sys/types.h: No such file or directory compilation terminated.
CC /home/sfr/next/x86_64_allmodconfig/tools/objtool/exec-cmd.o
CC /home/sfr/next/x86_64_allmodconfig/tools/objtool/help.o
builtin-check.c:28:20: fatal error: string.h: No such file or directory compilation terminated.
objtool.c:28:19: fatal error: stdio.h: No such file or directory compilation terminated.

Quoted 3 lines

and further errors ...
This build is done with a PowerPC hosted cross compiler with no glibc.

Ugh, what a rare and weird way to build an x86 kernel, and you made linux-next dependent on it?

Quoted 1 line

I assume that some things here need to be built with HOSTCC?

I suspect that's the culprit. Do you now mandate people to have PowerPC systems as a requirement to merge to linux-next? How are people supposed to be able to test that rare type of build method which does not matter to 99.99% of our kernel testers and users?

Thanks,

Ingo

Sedat Dilek

2016-03-01 07:28 UTC #2 lore raw

Quoted 33 lines · Ingo Molnar

Quoted 21 lines · Stephen Rothwell
Hi all,
After merging the tip tree, today's linux-next build (x86_64 allmodconfig) failed like this:
  DESCEND  objtool
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/builtin-check.o
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/special.o
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/elf.o
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/objtool.o
  MKDIR    /home/sfr/next/x86_64_allmodconfig/tools/objtool/arch/x86/insn/
  CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/libstring.o
elf.c:22:23: fatal error: sys/types.h: No such file or directory compilation terminated.
CC /home/sfr/next/x86_64_allmodconfig/tools/objtool/exec-cmd.o
CC /home/sfr/next/x86_64_allmodconfig/tools/objtool/help.o
builtin-check.c:28:20: fatal error: string.h: No such file or directory compilation terminated.
objtool.c:28:19: fatal error: stdio.h: No such file or directory compilation terminated.
Quoted 3 lines
and further errors ...
This build is done with a PowerPC hosted cross compiler with no glibc.
Ugh, what a rare and weird way to build an x86 kernel, and you made linux-next dependent on it?
Quoted 1 line
I assume that some things here need to be built with HOSTCC?

Quoted 7 lines

I suspect that's the culprit. Do you now mandate people to have PowerPC systems as a requirement to merge to linux-next? How are people supposed to be able to test that rare type of build method which does not matter to 99.99% of our kernel testers and users?
From my experience in using different $COMPILER you should always pass

CC and HOSTCC in your make-line when building a Linux-kernel or playing with the Kconfig-system.

When building my llvm-toolchain with make/autoconf I had also to pass CC and CXX to my configure-line otherwise you got misconfiguration.

[ EXAMPLE: Linux Kconfig-system ]

$ MAKE="make V=1" ; COMPILER="mycompiler" ; MAKE_OPTS="CC=$COMPILER
HOSTCC=$COMPILER"

$ yes "" | $MAKE $MAKE_OPTS oldconfig && $MAKE $MAKE_OPTS
silentoldconfig < /dev/null

- Sedat -

H. Peter Anvin

reviewer

2016-03-01 07:40 UTC #3 lore raw

Quoted 62 lines · Sedat Dilek

Quoted 6 lines · Ingo Molnar
Quoted 3 lines · Stephen Rothwell
Hi all,
After merging the tip tree, today's linux-next build (x86_64
allmodconfig)
Quoted 4 lines
Quoted 4 lines
failed like this:
DESCEND objtool
CC
/home/sfr/next/x86_64_allmodconfig/tools/objtool/builtin-check.o
Quoted 1 line
Quoted 1 line
CC
/home/sfr/next/x86_64_allmodconfig/tools/objtool/special.o
Quoted 2 lines
Quoted 2 lines
CC /home/sfr/next/x86_64_allmodconfig/tools/objtool/elf.o
CC
/home/sfr/next/x86_64_allmodconfig/tools/objtool/objtool.o
Quoted 1 line
Quoted 1 line
MKDIR
/home/sfr/next/x86_64_allmodconfig/tools/objtool/arch/x86/insn/
Quoted 1 line
Quoted 1 line
CC
/home/sfr/next/x86_64_allmodconfig/tools/objtool/libstring.o
Quoted 3 lines
Quoted 3 lines
elf.c:22:23: fatal error: sys/types.h: No such file or directory compilation terminated.
CC
/home/sfr/next/x86_64_allmodconfig/tools/objtool/exec-cmd.o
Quoted 2 lines
Quoted 2 lines
CC /home/sfr/next/x86_64_allmodconfig/tools/objtool/help.o
builtin-check.c:28:20: fatal error: string.h: No such file or
directory
Quoted 7 lines
Quoted 7 lines
compilation terminated.
objtool.c:28:19: fatal error: stdio.h: No such file or directory compilation terminated.
and further errors ...
This build is done with a PowerPC hosted cross compiler with no
glibc.
Quoted 2 lines
Ugh, what a rare and weird way to build an x86 kernel, and you made
linux-next
Quoted 5 lines
dependent on it?
Quoted 1 line
I assume that some things here need to be built with HOSTCC?
I suspect that's the culprit. Do you now mandate people to have
PowerPC systems as
Quoted 1 line
a requirement to merge to linux-next? How are people supposed to be
able to test
Quoted 1 line
that rare type of build method which does not matter to 99.99% of our
kernel
Quoted 2 lines
testers and users?
From my experience in using different $COMPILER you should always pass CC and HOSTCC in your make-line when building a Linux-kernel or playing with the Kconfig-system.
When building my llvm-toolchain with make/autoconf I had also to pass CC and CXX to my configure-line otherwise you got misconfiguration.
[ EXAMPLE: Linux Kconfig-system ]

Quoted 7 lines

$ MAKE="make V=1" ; COMPILER="mycompiler" ; MAKE_OPTS="CC=$COMPILER
HOSTCC=$COMPILER"

$ yes "" | $MAKE $MAKE_OPTS oldconfig && $MAKE $MAKE_OPTS
silentoldconfig < /dev/null

- Sedat -

That is not the issue here. The problem is clearly that objtool is a host program and not compiled with host cc. So it is a good thing to test even though it is weird, because it affects real use cases.

As x86 is far and beyond the most widely used host architecture, cross-compiling x86 is more likely to involve compiler differences than actually using another host, or possibly compiling 32 bits on a system without the 32-bit libc installed, but it should still work.

Signature · 1 line

Sent from my Android device with K-9 Mail. Please excuse brevity and formatting.

Sedat Dilek

2016-03-01 08:41 UTC #4 lore raw

Quoted 66 lines · H. Peter Anvin

Quoted 65 lines · Sedat Dilek
Quoted 6 lines · Ingo Molnar
Quoted 3 lines · Stephen Rothwell
Hi all,
After merging the tip tree, today's linux-next build (x86_64
allmodconfig)
Quoted 4 lines
Quoted 4 lines
failed like this:
DESCEND objtool
CC
/home/sfr/next/x86_64_allmodconfig/tools/objtool/builtin-check.o
Quoted 1 line
Quoted 1 line
CC
/home/sfr/next/x86_64_allmodconfig/tools/objtool/special.o
Quoted 2 lines
Quoted 2 lines
CC /home/sfr/next/x86_64_allmodconfig/tools/objtool/elf.o
CC
/home/sfr/next/x86_64_allmodconfig/tools/objtool/objtool.o
Quoted 1 line
Quoted 1 line
MKDIR
/home/sfr/next/x86_64_allmodconfig/tools/objtool/arch/x86/insn/
Quoted 1 line
Quoted 1 line
CC
/home/sfr/next/x86_64_allmodconfig/tools/objtool/libstring.o
Quoted 3 lines
Quoted 3 lines
elf.c:22:23: fatal error: sys/types.h: No such file or directory compilation terminated.
CC
/home/sfr/next/x86_64_allmodconfig/tools/objtool/exec-cmd.o
Quoted 2 lines
Quoted 2 lines
CC /home/sfr/next/x86_64_allmodconfig/tools/objtool/help.o
builtin-check.c:28:20: fatal error: string.h: No such file or
directory
Quoted 7 lines
Quoted 7 lines
compilation terminated.
objtool.c:28:19: fatal error: stdio.h: No such file or directory compilation terminated.
and further errors ...
This build is done with a PowerPC hosted cross compiler with no
glibc.
Quoted 2 lines
Ugh, what a rare and weird way to build an x86 kernel, and you made
linux-next
Quoted 5 lines
dependent on it?
Quoted 1 line
I assume that some things here need to be built with HOSTCC?
I suspect that's the culprit. Do you now mandate people to have
PowerPC systems as
Quoted 1 line
a requirement to merge to linux-next? How are people supposed to be
able to test
Quoted 1 line
that rare type of build method which does not matter to 99.99% of our
kernel
Quoted 2 lines
testers and users?
From my experience in using different $COMPILER you should always pass CC and HOSTCC in your make-line when building a Linux-kernel or playing with the Kconfig-system.
When building my llvm-toolchain with make/autoconf I had also to pass CC and CXX to my configure-line otherwise you got misconfiguration.
[ EXAMPLE: Linux Kconfig-system ]
$ MAKE="make V=1" ; COMPILER="mycompiler" ; MAKE_OPTS="CC=$COMPILER
HOSTCC=$COMPILER"

Quoted 9 lines

Quoted 4 lines
$ yes "" | $MAKE $MAKE_OPTS oldconfig && $MAKE $MAKE_OPTS
silentoldconfig < /dev/null
- Sedat -
That is not the issue here. The problem is clearly that objtool is a host program and not compiled with host cc. So it is a good thing to test even though it is weird, because it affects real use cases.
As x86 is far and beyond the most widely used host architecture, cross-compiling x86 is more likely to involve compiler differences than actually using another host, or possibly compiling 32 bits on a system without the 32-bit libc installed, but it should still work.

Thanks for the clarification.

I talked about my experiences in software building in general.

When this is a "tools/objtool problem" then someone should address this to "tools/objtool maintainers/folks" or whoever else is responsible.

- Sedat -

Ingo Molnar

2016-03-01 09:45 UTC #5 lore raw

In reply to H. Peter Anvin

Quoted 3 lines · H. Peter Anvin

That is not the issue here. The problem is clearly that objtool is a host program and not compiled with host cc. So it is a good thing to test even though it is weird, because it affects real use cases.

Absolutely, it's a real bug that should be fixed. What is counterproductive is that this rare and hard to replicate build modus is now a must-have test for linux-next inclusion.

I already cross-build to 20+ weird, rarely used architectures, slowing down the linux-next merge integration test immensely and wasting quite a bit of electricity. linux-next making it even harder to test for is a step backwards.

Thanks,

Ingo

Stephen Rothwell

author

2016-03-01 09:40 UTC #6 lore raw

In reply to Ingo Molnar

Hi Ingo,

Quoted 5 lines · On Tue, 1 Mar 2016 08:07:50 +0100 Ingo Molnar <[email protected]> wrote:

Quoted 2 lines
This build is done with a PowerPC hosted cross compiler with no glibc.
Ugh, what a rare and weird way to build an x86 kernel, and you made linux-next dependent on it?

It is just the fastest hardware I currently have access to (you remember who I work for, right? ;-)). I have always done at least part of the linux-next building (daily, or overnight) on PowerPC hardware and this is only the 2nd or third time in over 8 years that it has found an issue like this.

Quoted 3 lines

Quoted 1 line
I assume that some things here need to be built with HOSTCC?
I suspect that's the culprit.

Good, hopefully it is not too hard to fix.

Signature · 2 lines

Cheers,
Stephen Rothwell

Josh Poimboeuf

2016-03-01 21:54 UTC #7 lore raw

PATCH 1/1 objtool: Disable stack validation when CROSS_COMPILE is used

Quoted 15 lines · Stephen Rothwell

Hi Ingo,
Quoted 5 lines · On Tue, 1 Mar 2016 08:07:50 +0100 Ingo Molnar <[email protected]> wrote:
Quoted 2 lines
This build is done with a PowerPC hosted cross compiler with no glibc.
Ugh, what a rare and weird way to build an x86 kernel, and you made linux-next dependent on it?
It is just the fastest hardware I currently have access to (you remember who I work for, right? ;-)). I have always done at least part of the linux-next building (daily, or overnight) on PowerPC hardware and this is only the 2nd or third time in over 8 years that it has found an issue like this.

Quoted 5 lines

Quoted 3 lines
Quoted 1 line
I assume that some things here need to be built with HOSTCC?
I suspect that's the culprit.
Good, hopefully it is not too hard to fix.

Changing it to use the host compiler would probably be an easy fix, but that would expose a harder bug related to endianness.

objtool uses the kernel x86 decoder (arch/x86/lib/insn.c) which is endian-ignorant. If it tries to analyze reverse endian binaries it gets confused. And since CONFIG_STACK_VALIDATION causes objtool to analyze every .o file in the build, it'll spit out a ton of false warnings.

How about the below workaround patch to disable objtool and warn when CROSS_COMPILE is used? If anybody complains about lack of cross-compile support later, we could try to fix it then.

Quoted 1 line

From a3c65947011a420743f308b698171c4209105d3f Mon Sep 17 00:00:00 2001

Message-Id: <a3c65947011a420743f308b698171c4209105d3f.1456868910.git.jpoimboe@redhat.com>
From: Josh Poimboeuf <[email protected]>
Date: Tue, 1 Mar 2016 13:35:51 -0600
Subject: [PATCH] objtool: Disable stack validation when CROSS_COMPILE is used

When building with CONFIG_STACK_VALIDATION on a PowerPC host using an x86 cross-compiler, Stephen Rothwell saw the following objtool build errors:

    DESCEND  objtool
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/builtin-check.o
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/special.o
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/elf.o
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/objtool.o
    MKDIR    /home/sfr/next/x86_64_allmodconfig/tools/objtool/arch/x86/insn/
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/libstring.o

The problem is that objtool was compiled with the cross compiler instead of the host compiler. But even if that were fixed, we'd still have a problem with endianness. objtool's x86 decoder (copied from arch/x86/lib/insn.c) assumes that the endianness of the host and target are the same. When analyzing a little-endian x86 binary on a big-endian host, it will get confused and spit out a bunch of false positive warnings during the build.

Eventually we may want to add endian awareness to x86 objtool, but that's generally a rare edge case and it's not clear if anybody would use it. For now, when a cross-compiler is used, just disable CONFIG_STACK_VALIDATION and emit a warning.

Reported-by Stephen Rothwell <[email protected]>

Signed-off-by Josh Poimboeuf <[email protected]>

---
 Makefile               | 12 +++++++++++-
 scripts/Makefile.build |  4 +++-
 2 files changed, 14 insertions(+), 2 deletions(-)

Makefile +11 -1

index 8cc926f..b59007c 100644
@@ -998,8 +998,18 @@ prepare0: archprepare FORCE
 # All the preparing..
 prepare: prepare0 prepare-objtool
 
+
+ifdef CONFIG_STACK_VALIDATION
+  ifeq ($(CROSS_COMPILE),)
+    objtool_target := tools/objtool FORCE
+  else
+    $(warning Stack validation is not supported with cross-compilation.  Disabling CONFIG_STACK_VALIDATION!)
+  endif
+endif
+
 PHONY += prepare-objtool
-prepare-objtool: $(if $(CONFIG_STACK_VALIDATION), tools/objtool FORCE)
+prepare-objtool: $(objtool_target)
+
 
 # Generate some files
 # ---------------------------------------------------------------------------

scripts/Makefile.build +3 -1

index 130a452..973bb26 100644
@@ -242,6 +242,7 @@ cmd_record_mcount =						\
 endif
 
 ifdef CONFIG_STACK_VALIDATION
+ifeq ($(CROSS_COMPILE),)
 
 __objtool_obj := $(objtree)/tools/objtool/objtool

@@ -260,13 +261,14 @@ objtool_obj = $(if $(patsubst y%,, \
 	$(OBJECT_FILES_NON_STANDARD_$(basetarget).o)$(OBJECT_FILES_NON_STANDARD)n), \
 	$(__objtool_obj))
 
+endif # CROSS_COMPILE
 endif # CONFIG_STACK_VALIDATION
 
 define rule_cc_o_c
 	$(call echo-cmd,checksrc) $(cmd_checksrc)			  \
 	$(call echo-cmd,cc_o_c) $(cmd_cc_o_c);				  \
 	$(cmd_modversions)						  \
-	$(cmd_objtool)						  \
+	$(cmd_objtool)							  \
 	$(call echo-cmd,record_mcount)					  \
 	$(cmd_record_mcount)						  \
 	scripts/basic/fixdep $(depfile) $@ '$(call make-cmd,cc_o_c)' >    \

Signature · 1 line

2.4.3

Stephen Rothwell

author

2016-03-02 02:27 UTC #8 lore raw

Re: [PATCH] objtool: Disable stack validation when CROSS_COMPILE is used

Hi Josh,

Quoted 3 lines · On Tue, 1 Mar 2016 15:54:51 -0600 Josh Poimboeuf <[email protected]> wrote:

Changing it to use the host compiler would probably be an easy fix, but that would expose a harder bug related to endianness.

Just by luck, my PowerPC host is little endian :-)

Quoted 3 lines

How about the below workaround patch to disable objtool and warn when CROSS_COMPILE is used? If anybody complains about lack of cross-compile support later, we could try to fix it then.

This seems reasonable.

Quoted 5 lines

From a3c65947011a420743f308b698171c4209105d3f Mon Sep 17 00:00:00 2001
Message-Id: <a3c65947011a420743f308b698171c4209105d3f.1456868910.git.jpoimboe@redhat.com>
From: Josh Poimboeuf <[email protected]>
Date: Tue, 1 Mar 2016 13:35:51 -0600
Subject: [PATCH] objtool: Disable stack validation when CROSS_COMPILE is used

I have applied this to the merge of the tip tree in linux-next today and it compiles fine for me. I will continue applying it until something better comes along or it is applied to the tip tree.

Thanks for that.

Signature · 2 lines

Cheers,
Stephen Rothwell

Josh Poimboeuf

2016-03-02 21:17 UTC #9 lore raw

Re: [PATCH] objtool: Disable stack validation when CROSS_COMPILE is used

Quoted 8 lines · Stephen Rothwell

Hi Josh,
Quoted 3 lines · On Tue, 1 Mar 2016 15:54:51 -0600 Josh Poimboeuf <[email protected]> wrote:
Changing it to use the host compiler would probably be an easy fix, but that would expose a harder bug related to endianness.
Just by luck, my PowerPC host is little endian :-)

Aha! In that case, I think I'll take a different approach and try to add support for CROSS_COMPILE.

We'll need it anyway, later on, when we port objtool to more architectures.

Quoted 12 lines

Quoted 3 lines
How about the below workaround patch to disable objtool and warn when CROSS_COMPILE is used? If anybody complains about lack of cross-compile support later, we could try to fix it then.
This seems reasonable.
Quoted 5 lines
From a3c65947011a420743f308b698171c4209105d3f Mon Sep 17 00:00:00 2001
Message-Id: <a3c65947011a420743f308b698171c4209105d3f.1456868910.git.jpoimboe@redhat.com>
From: Josh Poimboeuf <[email protected]>
Date: Tue, 1 Mar 2016 13:35:51 -0600
Subject: [PATCH] objtool: Disable stack validation when CROSS_COMPILE is used

Quoted 3 lines

I have applied this to the merge of the tip tree in linux-next today and it compiles fine for me. I will continue applying it until something better comes along or it is applied to the tip tree.

Thanks. I should have the patch(es) soon. If/when they get merged, I'll let you know, and then you can drop this one.

Signature · 1 line

Josh

Stephen Rothwell

author

2016-03-02 22:21 UTC #10 lore raw

Re: [PATCH] objtool: Disable stack validation when CROSS_COMPILE is used

Hi Josh,

Quoted 8 lines · On Wed, 2 Mar 2016 15:17:26 -0600 Josh Poimboeuf <[email protected]> wrote:

Quoted 6 lines · Stephen Rothwell
Quoted 3 lines · On Tue, 1 Mar 2016 15:54:51 -0600 Josh Poimboeuf <[email protected]> wrote:
Changing it to use the host compiler would probably be an easy fix, but that would expose a harder bug related to endianness.

Quoted 7 lines

Quoted 1 line
Just by luck, my PowerPC host is little endian :-)
Aha! In that case, I think I'll take a different approach and try to add support for CROSS_COMPILE.
We'll need it anyway, later on, when we port objtool to more architectures.

That would be great. I was discussing this briefly with the powerpc maintainer and he expressed some interest.

Quoted 2 lines

Thanks. I should have the patch(es) soon. If/when they get merged, I'll let you know, and then you can drop this one.

OK, I will keep an eye out - I usually notice when stuff get fixed, but a heads up is always helpful.

Signature · 2 lines

Cheers,
Stephen Rothwell

Josh Poimboeuf

2016-03-03 00:41 UTC #11 lore raw

cover objtool: Cross-compilation support

This adds the ability for CONFIG_STACK_VALIDATION to work in a cross-compiled environment against an x86 kernel. Based on tip/master.

Josh Poimboeuf (2):
x86/asm/decoder: Use explicitly signed chars
objtool: Support CROSS_COMPILE

 arch/x86/lib/insn.c                     |  6 +++---
 tools/lib/subcmd/Makefile               |  6 ++++--
 tools/objtool/Makefile                  | 17 ++++++++++-------
 tools/objtool/arch/x86/insn/insn.c      |  6 +++---
 tools/perf/util/intel-pt-decoder/insn.c |  6 +++---
 5 files changed, 23 insertions(+), 18 deletions(-)

Signature · 1 line

2.4.3

Josh Poimboeuf

2016-03-03 00:41 UTC #12 lore raw

PATCH 1/2 x86/asm/decoder: Use explicitly signed chars

When running objtool on a ppc64le host to analyze x86 binaries, it reports a lot of false warnings like:

ipc/compat_mq.o: warning: objtool: compat_SyS_mq_open()+0x91: can't find jump dest instruction at .text+0x3a5

The warnings are caused by the x86 instruction decoder setting the wrong value for the jump instruction's immediate field because it assumes that "char == signed char", which isn't true for all architectures. When converting char to int, gcc sign-extends on x86 but doesn't sign-extend on ppc64le.

According to the gcc man page, that's a feature, not a bug:

Quoted 6 lines

Each kind of machine has a default for what "char" should be. It is either like "unsigned char" by default or like "signed char" by default.
Ideally, a portable program should always use "signed char" or "unsigned char" when it depends on the signedness of an object.

Conform to the "standards" by changing the "char" casts to "signed char". This results in no actual changes to the object code on x86.

Note: the x86 decoder now lives in three different locations in the kernel tree, which are all kept in sync via makefile checks and
warnings: in-kernel, perf, and objtool. This fixes all three locations. Eventually we should probably try to at least converge the two separate "tools" locations into a single shared location.

Signed-off-by Josh Poimboeuf <[email protected]>

---
 arch/x86/lib/insn.c                     | 6 +++---
 tools/objtool/arch/x86/insn/insn.c      | 6 +++---
 tools/perf/util/intel-pt-decoder/insn.c | 6 +++---
 3 files changed, 9 insertions(+), 9 deletions(-)

arch/x86/lib/insn.c +3 -3

index 8f72b33..1a41693 100644
@@ -374,7 +374,7 @@ void insn_get_displacement(struct insn *insn)
 		if (mod == 3)
 			goto out;
 		if (mod == 1) {
-			insn->displacement.value = get_next(char, insn);
+			insn->displacement.value = get_next(signed char, insn);
 			insn->displacement.nbytes = 1;
 		} else if (insn->addr_bytes == 2) {
 			if ((mod == 0 && rm == 6) || mod == 2) {

@@ -532,7 +532,7 @@ void insn_get_immediate(struct insn *insn)
 
 	switch (inat_immediate_size(insn->attr)) {
 	case INAT_IMM_BYTE:
-		insn->immediate.value = get_next(char, insn);
+		insn->immediate.value = get_next(signed char, insn);
 		insn->immediate.nbytes = 1;
 		break;
 	case INAT_IMM_WORD:

@@ -566,7 +566,7 @@ void insn_get_immediate(struct insn *insn)
 		goto err_out;
 	}
 	if (inat_has_second_immediate(insn->attr)) {
-		insn->immediate2.value = get_next(char, insn);
+		insn->immediate2.value = get_next(signed char, insn);
 		insn->immediate2.nbytes = 1;
 	}
 done:

tools/objtool/arch/x86/insn/insn.c +3 -3

index 47314a6..9f26eae 100644
@@ -374,7 +374,7 @@ void insn_get_displacement(struct insn *insn)
 		if (mod == 3)
 			goto out;
 		if (mod == 1) {
-			insn->displacement.value = get_next(char, insn);
+			insn->displacement.value = get_next(signed char, insn);
 			insn->displacement.nbytes = 1;
 		} else if (insn->addr_bytes == 2) {
 			if ((mod == 0 && rm == 6) || mod == 2) {

@@ -532,7 +532,7 @@ void insn_get_immediate(struct insn *insn)
 
 	switch (inat_immediate_size(insn->attr)) {
 	case INAT_IMM_BYTE:
-		insn->immediate.value = get_next(char, insn);
+		insn->immediate.value = get_next(signed char, insn);
 		insn->immediate.nbytes = 1;
 		break;
 	case INAT_IMM_WORD:

@@ -566,7 +566,7 @@ void insn_get_immediate(struct insn *insn)
 		goto err_out;
 	}
 	if (inat_has_second_immediate(insn->attr)) {
-		insn->immediate2.value = get_next(char, insn);
+		insn->immediate2.value = get_next(signed char, insn);
 		insn->immediate2.nbytes = 1;
 	}
 done:

tools/perf/util/intel-pt-decoder/insn.c +3 -3

index 47314a6..9f26eae 100644
@@ -374,7 +374,7 @@ void insn_get_displacement(struct insn *insn)
 		if (mod == 3)
 			goto out;
 		if (mod == 1) {
-			insn->displacement.value = get_next(char, insn);
+			insn->displacement.value = get_next(signed char, insn);
 			insn->displacement.nbytes = 1;
 		} else if (insn->addr_bytes == 2) {
 			if ((mod == 0 && rm == 6) || mod == 2) {

@@ -532,7 +532,7 @@ void insn_get_immediate(struct insn *insn)
 
 	switch (inat_immediate_size(insn->attr)) {
 	case INAT_IMM_BYTE:
-		insn->immediate.value = get_next(char, insn);
+		insn->immediate.value = get_next(signed char, insn);
 		insn->immediate.nbytes = 1;
 		break;
 	case INAT_IMM_WORD:

@@ -566,7 +566,7 @@ void insn_get_immediate(struct insn *insn)
 		goto err_out;
 	}
 	if (inat_has_second_immediate(insn->attr)) {
-		insn->immediate2.value = get_next(char, insn);
+		insn->immediate2.value = get_next(signed char, insn);
 		insn->immediate2.nbytes = 1;
 	}
 done:

Signature · 1 line

2.4.3

tip-bot for Josh Poimboeuf

merge bot

2016-03-03 16:59 UTC #13 lore raw

[tip:core/objtool] x86/asm/decoder: Use explicitly signed chars

x86/asm/decoder: Use explicitly signed chars

Commit-ID:  19072f23d1d785c093b7f81cb1fb161e7a13ecc0
Gitweb:     http://git.kernel.org/tip/19072f23d1d785c093b7f81cb1fb161e7a13ecc0
Author:     Josh Poimboeuf <[email protected]>
AuthorDate: Wed, 2 Mar 2016 18:39:36 -0600
Committer:  Ingo Molnar <[email protected]>
CommitDate: Thu, 3 Mar 2016 16:13:00 +0100

x86/asm/decoder: Use explicitly signed chars

When running objtool on a ppc64le host to analyze x86 binaries, it reports a lot of false warnings like:

ipc/compat_mq.o: warning: objtool: compat_SyS_mq_open()+0x91: can't find jump dest instruction at .text+0x3a5

According to the gcc man page, that's a feature, not a bug:

Quoted 6 lines

Each kind of machine has a default for what "char" should be. It is either like "unsigned char" by default or like "signed char" by default.
Ideally, a portable program should always use "signed char" or "unsigned char" when it depends on the signedness of an object.

Conform to the "standards" by changing the "char" casts to "signed char". This results in no actual changes to the object code on x86.

Signed-off-by Josh Poimboeuf <[email protected]>

Cc Adrian Hunter <[email protected]>

Cc Linus Torvalds <[email protected]>

Cc Masami Hiramatsu <[email protected]>

Cc Michael Ellerman <[email protected]>

Cc Peter Zijlstra <[email protected]>

Cc Stephen Rothwell <[email protected]>

Cc Thomas Gleixner <[email protected]>

Link http://lkml.kernel.org/r/9dd4161719b20e6def9564646d68bfbe498c549f.1456962210.git.jpoimboe@redhat.com open here

Signed-off-by Ingo Molnar <[email protected]>

---
 arch/x86/lib/insn.c                     | 6 +++---
 tools/objtool/arch/x86/insn/insn.c      | 6 +++---
 tools/perf/util/intel-pt-decoder/insn.c | 6 +++---
 3 files changed, 9 insertions(+), 9 deletions(-)

arch/x86/lib/insn.c +3 -3

index 8f72b33..1a41693 100644
@@ -374,7 +374,7 @@ void insn_get_displacement(struct insn *insn)
 		if (mod == 3)
 			goto out;
 		if (mod == 1) {
-			insn->displacement.value = get_next(char, insn);
+			insn->displacement.value = get_next(signed char, insn);
 			insn->displacement.nbytes = 1;
 		} else if (insn->addr_bytes == 2) {
 			if ((mod == 0 && rm == 6) || mod == 2) {

@@ -532,7 +532,7 @@ void insn_get_immediate(struct insn *insn)
 
 	switch (inat_immediate_size(insn->attr)) {
 	case INAT_IMM_BYTE:
-		insn->immediate.value = get_next(char, insn);
+		insn->immediate.value = get_next(signed char, insn);
 		insn->immediate.nbytes = 1;
 		break;
 	case INAT_IMM_WORD:

@@ -566,7 +566,7 @@ void insn_get_immediate(struct insn *insn)
 		goto err_out;
 	}
 	if (inat_has_second_immediate(insn->attr)) {
-		insn->immediate2.value = get_next(char, insn);
+		insn->immediate2.value = get_next(signed char, insn);
 		insn->immediate2.nbytes = 1;
 	}
 done:

tools/objtool/arch/x86/insn/insn.c +3 -3

index 47314a6..9f26eae 100644
@@ -374,7 +374,7 @@ void insn_get_displacement(struct insn *insn)
 		if (mod == 3)
 			goto out;
 		if (mod == 1) {
-			insn->displacement.value = get_next(char, insn);
+			insn->displacement.value = get_next(signed char, insn);
 			insn->displacement.nbytes = 1;
 		} else if (insn->addr_bytes == 2) {
 			if ((mod == 0 && rm == 6) || mod == 2) {

@@ -532,7 +532,7 @@ void insn_get_immediate(struct insn *insn)
 
 	switch (inat_immediate_size(insn->attr)) {
 	case INAT_IMM_BYTE:
-		insn->immediate.value = get_next(char, insn);
+		insn->immediate.value = get_next(signed char, insn);
 		insn->immediate.nbytes = 1;
 		break;
 	case INAT_IMM_WORD:

@@ -566,7 +566,7 @@ void insn_get_immediate(struct insn *insn)
 		goto err_out;
 	}
 	if (inat_has_second_immediate(insn->attr)) {
-		insn->immediate2.value = get_next(char, insn);
+		insn->immediate2.value = get_next(signed char, insn);
 		insn->immediate2.nbytes = 1;
 	}
 done:

tools/perf/util/intel-pt-decoder/insn.c +3 -3

index 47314a6..9f26eae 100644
@@ -374,7 +374,7 @@ void insn_get_displacement(struct insn *insn)
 		if (mod == 3)
 			goto out;
 		if (mod == 1) {
-			insn->displacement.value = get_next(char, insn);
+			insn->displacement.value = get_next(signed char, insn);
 			insn->displacement.nbytes = 1;
 		} else if (insn->addr_bytes == 2) {
 			if ((mod == 0 && rm == 6) || mod == 2) {

@@ -532,7 +532,7 @@ void insn_get_immediate(struct insn *insn)
 
 	switch (inat_immediate_size(insn->attr)) {
 	case INAT_IMM_BYTE:
-		insn->immediate.value = get_next(char, insn);
+		insn->immediate.value = get_next(signed char, insn);
 		insn->immediate.nbytes = 1;
 		break;
 	case INAT_IMM_WORD:

@@ -566,7 +566,7 @@ void insn_get_immediate(struct insn *insn)
 		goto err_out;
 	}
 	if (inat_has_second_immediate(insn->attr)) {
-		insn->immediate2.value = get_next(char, insn);
+		insn->immediate2.value = get_next(signed char, insn);
 		insn->immediate2.nbytes = 1;
 	}
 done:

H. Peter Anvin

reviewer

2016-03-03 19:01 UTC #14 lore raw

Re: [tip:core/objtool] x86/asm/decoder: Use explicitly signed chars

Quoted 154 lines · tip-bot for Josh Poimboeuf

Commit-ID: 19072f23d1d785c093b7f81cb1fb161e7a13ecc0 Gitweb:
http://git.kernel.org/tip/19072f23d1d785c093b7f81cb1fb161e7a13ecc0
Author:     Josh Poimboeuf <[email protected]>
AuthorDate: Wed, 2 Mar 2016 18:39:36 -0600
Committer:  Ingo Molnar <[email protected]>
CommitDate: Thu, 3 Mar 2016 16:13:00 +0100
x86/asm/decoder: Use explicitly signed chars
When running objtool on a ppc64le host to analyze x86 binaries, it reports a lot of false warnings like:
ipc/compat_mq.o: warning: objtool: compat_SyS_mq_open()+0x91: can't find jump dest instruction at .text+0x3a5
The warnings are caused by the x86 instruction decoder setting the wrong
value for the jump instruction's immediate field because it assumes that
"char == signed char", which isn't true for all architectures. When converting char to int, gcc sign-extends on x86 but doesn't sign-extend on ppc64le.
According to the gcc man page, that's a feature, not a bug:
Quoted 6 lines
Each kind of machine has a default for what "char" should be. It is either like "unsigned char" by default or like "signed char" by default.
Ideally, a portable program should always use "signed char" or "unsigned char" when it depends on the signedness of an object.
Conform to the "standards" by changing the "char" casts to "signed char". This results in no actual changes to the object code on x86.
Note: the x86 decoder now lives in three different locations in the kernel tree, which are all kept in sync via makefile checks and
warnings: in-kernel, perf, and objtool. This fixes all three locations.
Eventually we should probably try to at least converge the two separate "tools" locations into a single shared location.
Signed-off-by Josh Poimboeuf <[email protected]>
Cc Adrian Hunter <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Masami Hiramatsu <[email protected]>
Cc Michael Ellerman <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephen Rothwell <[email protected]>
Cc Thomas Gleixner <[email protected]>
Link:
http://lkml.kernel.org/r/9dd4161719b20e6def9564646d68bfbe498c549f.1456962210.git.jpoimboe@redhat.com
Signed-off-by Ingo Molnar <[email protected]>
---
arch/x86/lib/insn.c                     | 6 +++---
tools/objtool/arch/x86/insn/insn.c      | 6 +++---
tools/perf/util/intel-pt-decoder/insn.c | 6 +++---
3 files changed, 9 insertions(+), 9 deletions(-)
arch/x86/lib/insn.c +3 -3
index 8f72b33..1a41693 100644
@@ -374,7 +374,7 @@ void insn_get_displacement(struct insn *insn)
 		if (mod == 3)
 			goto out;
 		if (mod == 1) {
-			insn->displacement.value = get_next(char, insn);
+			insn->displacement.value = get_next(signed char, insn);
 			insn->displacement.nbytes = 1;
 		} else if (insn->addr_bytes == 2) {
 			if ((mod == 0 && rm == 6) || mod == 2) {
@@ -532,7 +532,7 @@ void insn_get_immediate(struct insn *insn)
 
 	switch (inat_immediate_size(insn->attr)) {
 	case INAT_IMM_BYTE:
-		insn->immediate.value = get_next(char, insn);
+		insn->immediate.value = get_next(signed char, insn);
 		insn->immediate.nbytes = 1;
 		break;
 	case INAT_IMM_WORD:
@@ -566,7 +566,7 @@ void insn_get_immediate(struct insn *insn)
 		goto err_out;
 	}
 	if (inat_has_second_immediate(insn->attr)) {
-		insn->immediate2.value = get_next(char, insn);
+		insn->immediate2.value = get_next(signed char, insn);
 		insn->immediate2.nbytes = 1;
 	}
done:
tools/objtool/arch/x86/insn/insn.c +3 -3
index 47314a6..9f26eae 100644
@@ -374,7 +374,7 @@ void insn_get_displacement(struct insn *insn)
 		if (mod == 3)
 			goto out;
 		if (mod == 1) {
-			insn->displacement.value = get_next(char, insn);
+			insn->displacement.value = get_next(signed char, insn);
 			insn->displacement.nbytes = 1;
 		} else if (insn->addr_bytes == 2) {
 			if ((mod == 0 && rm == 6) || mod == 2) {
@@ -532,7 +532,7 @@ void insn_get_immediate(struct insn *insn)
 
 	switch (inat_immediate_size(insn->attr)) {
 	case INAT_IMM_BYTE:
-		insn->immediate.value = get_next(char, insn);
+		insn->immediate.value = get_next(signed char, insn);
 		insn->immediate.nbytes = 1;
 		break;
 	case INAT_IMM_WORD:
@@ -566,7 +566,7 @@ void insn_get_immediate(struct insn *insn)
 		goto err_out;
 	}
 	if (inat_has_second_immediate(insn->attr)) {
-		insn->immediate2.value = get_next(char, insn);
+		insn->immediate2.value = get_next(signed char, insn);
 		insn->immediate2.nbytes = 1;
 	}
done:
tools/perf/util/intel-pt-decoder/insn.c +3 -3
index 47314a6..9f26eae 100644
@@ -374,7 +374,7 @@ void insn_get_displacement(struct insn *insn)
 		if (mod == 3)
 			goto out;
 		if (mod == 1) {
-			insn->displacement.value = get_next(char, insn);
+			insn->displacement.value = get_next(signed char, insn);
 			insn->displacement.nbytes = 1;
 		} else if (insn->addr_bytes == 2) {
 			if ((mod == 0 && rm == 6) || mod == 2) {
@@ -532,7 +532,7 @@ void insn_get_immediate(struct insn *insn)
 
 	switch (inat_immediate_size(insn->attr)) {
 	case INAT_IMM_BYTE:
-		insn->immediate.value = get_next(char, insn);
+		insn->immediate.value = get_next(signed char, insn);
 		insn->immediate.nbytes = 1;
 		break;
 	case INAT_IMM_WORD:
@@ -566,7 +566,7 @@ void insn_get_immediate(struct insn *insn)
 		goto err_out;
 	}
 	if (inat_has_second_immediate(insn->attr)) {
-		insn->immediate2.value = get_next(char, insn);
+		insn->immediate2.value = get_next(signed char, insn);
 		insn->immediate2.nbytes = 1;
 	}

Quoted 1 line

done:

It ought to be made specific as __s8 (or int8_t) really...

Signature · 1 line

Sent from my Android device with K-9 Mail. Please excuse brevity and formatting.

Josh Poimboeuf

2016-03-03 00:41 UTC #15 lore raw

PATCH 2/2 objtool: Support CROSS_COMPILE

In reply to Josh Poimboeuf

When building with CONFIG_STACK_VALIDATION on a ppc64le host with an x86 cross-compiler, Stephen Rothwell saw the following objtool build errors:

    DESCEND  objtool
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/builtin-check.o
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/special.o
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/elf.o
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/objtool.o
    MKDIR    /home/sfr/next/x86_64_allmodconfig/tools/objtool/arch/x86/insn/
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/libstring.o

It fails to build because it tries to compile objtool with the cross-compiler instead of the host compiler.

Ensure that it always uses the host compiler by ignoring CROSS_COMPILE.

In order to do that properly, the libsubcmd.a library needs to be built in tools/objtool/ rather than tools/lib/subcmd/. The latter directory contains the cross-compiled version which is needed for perf and possibly other tools.

Note that cross-compiling for x86 on a _big_ endian system would result in a bunch of false positive objtool warnings during the kernel build because it isn't endian-aware. But that's generally a rare edge case and there haven't been any reports of anybody needing that.

Reported-by Stephen Rothwell <[email protected]>

Signed-off-by Josh Poimboeuf <[email protected]>

---
 tools/lib/subcmd/Makefile |  6 ++++--
 tools/objtool/Makefile    | 17 ++++++++++-------
 2 files changed, 14 insertions(+), 9 deletions(-)

tools/lib/subcmd/Makefile +4 -2

index 629cf8c..1faecb8 100644
@@ -8,8 +8,10 @@ srctree := $(patsubst %/,%,$(dir $(srctree)))
 #$(info Determined 'srctree' to be $(srctree))
 endif
 
-CC = $(CROSS_COMPILE)gcc
-AR = $(CROSS_COMPILE)ar
+CC ?= $(CROSS_COMPILE)gcc
+LD ?= $(CROSS_COMPILE)ld
+AR ?= $(CROSS_COMPILE)ar
+
 RM = rm -f
 
 MAKEFLAGS += --no-print-directory

tools/objtool/Makefile +10 -7

index c4f0713..e4a6bd5 100644
@@ -7,13 +7,19 @@ ARCH := x86
 endif
 endif
 
+# always use the host compiler
+CC = gcc
+LD = ld
+AR = ar
+
 ifeq ($(srctree),)
 srctree := $(patsubst %/,%,$(dir $(shell pwd)))
 srctree := $(patsubst %/,%,$(dir $(srctree)))
 endif
 
-SUBCMD_SRCDIR	= $(srctree)/tools/lib/subcmd/
-LIBSUBCMD	= $(if $(OUTPUT),$(OUTPUT),$(SUBCMD_SRCDIR))libsubcmd.a
+SUBCMD_SRCDIR		= $(srctree)/tools/lib/subcmd/
+LIBSUBCMD_OUTPUT	= $(if $(OUTPUT),$(OUTPUT),$(PWD)/)
+LIBSUBCMD		= $(LIBSUBCMD_OUTPUT)libsubcmd.a
 
 OBJTOOL    := $(OUTPUT)objtool
 OBJTOOL_IN := $(OBJTOOL)-in.o

@@ -45,12 +51,9 @@ $(OBJTOOL): $(LIBSUBCMD) $(OBJTOOL_IN)
 
 
 $(LIBSUBCMD): fixdep FORCE
-	$(Q)$(MAKE) -C $(SUBCMD_SRCDIR)
-
-$(LIBSUBCMD)-clean:
-	$(Q)$(MAKE) -C $(SUBCMD_SRCDIR) clean > /dev/null
+	$(Q)$(MAKE) -C $(SUBCMD_SRCDIR) OUTPUT=$(LIBSUBCMD_OUTPUT)
 
-clean: $(LIBSUBCMD)-clean
+clean:
 	$(call QUIET_CLEAN, objtool) $(RM) $(OBJTOOL)
 	$(Q)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
 	$(Q)$(RM) $(OUTPUT)arch/x86/insn/inat-tables.c $(OUTPUT)fixdep

Signature · 1 line

2.4.3

Stephen Rothwell

author

2016-03-03 02:43 UTC #16 lore raw

Re: [PATCH 2/2] objtool: Support CROSS_COMPILE

Hi Josh,

Just a couple of quick comments ...

tools/objtool/Makefile

Quoted 5 lines · On Wed, 2 Mar 2016 18:39:37 -0600 Josh Poimboeuf <[email protected]> wrote:

tools/objtool/Makefile +0 -0
index c4f0713..e4a6bd5 100644

I was wondering if this would be more appropriate in scripts/objtool since it is used during the building of the kernel. Or does it have a wider use?

tools/objtool/Makefile ARCH := x86 · line 11

Quoted 6 lines

tools/objtool/Makefile +2 -0

@@ -7,13 +7,19 @@ ARCH := x86
 endif
 endif
 
+# always use the host compiler
+CC = gcc

We have HOSTCC with its associated HOSTCFLAGS etc ... I am not sure if that is more appropriate (but it does take care of people using clang).

Signature · 2 lines

Cheers,
Stephen Rothwell

Josh Poimboeuf

2016-03-03 03:21 UTC #17 lore raw

Re: [PATCH 2/2] objtool: Support CROSS_COMPILE

Quoted 6 lines · Stephen Rothwell

Hi Josh,
Just a couple of quick comments ...
Quoted 1 line · On Wed, 2 Mar 2016 18:39:37 -0600 Josh Poimboeuf <[email protected]> wrote:

Quoted 8 lines

tools/objtool/Makefile
Quoted 4 lines
tools/objtool/Makefile +0 -0
index c4f0713..e4a6bd5 100644
I was wondering if this would be more appropriate in scripts/objtool since it is used during the building of the kernel. Or does it have a wider use?

Yeah, it was actually in the scripts/ dir in earlier revisions of the patch set, for that very reason. However, Ingo pointed out that it could be useful beyond the kernel, so we graduated it to a "tool".

Quoted 9 lines

tools/objtool/Makefile ARCH := x86 · line 11
Quoted 6 lines
tools/objtool/Makefile +2 -0
@@ -7,13 +7,19 @@ ARCH := x86
 endif
 endif
 
+# always use the host compiler
+CC = gcc
We have HOSTCC with its associated HOSTCFLAGS etc ... I am not sure if that is more appropriate (but it does take care of people using clang).

The "tools" are almost completely separate from the rest of the kernel. They have their own scaled-down version of kbuild, which doesn't have HOSTCC.

But yeah, we might eventually need to copy some of the host compilation infrastructure from scripts/Makefile.host over to the tools/ side.

Signature · 1 line

Josh

Stephen Rothwell

author

2016-03-03 03:38 UTC #18 lore raw

Re: [PATCH 2/2] objtool: Support CROSS_COMPILE

Hi Josh,

Quoted 15 lines · On Wed, 2 Mar 2016 21:20:58 -0600 Josh Poimboeuf <[email protected]> wrote:

Quoted 4 lines · Stephen Rothwell
I was wondering if this would be more appropriate in scripts/objtool since it is used during the building of the kernel. Or does it have a wider use?
Yeah, it was actually in the scripts/ dir in earlier revisions of the patch set, for that very reason. However, Ingo pointed out that it could be useful beyond the kernel, so we graduated it to a "tool".
Quoted 3 lines
We have HOSTCC with its associated HOSTCFLAGS etc ... I am not sure if that is more appropriate (but it does take care of people using clang).

Quoted 6 lines

The "tools" are almost completely separate from the rest of the kernel. They have their own scaled-down version of kbuild, which doesn't have HOSTCC.
But yeah, we might eventually need to copy some of the host compilation infrastructure from scripts/Makefile.host over to the tools/ side.

That all sounds sane, thanks.

I did not add this to linux-next today, but may tomorrow if people think it is sensible to do so (for testing on a powerpcle host). If I do, I will just back out to the previous patch if it all goes south (so it won't impact on the rest of the tip tree's testing).

Signature · 2 lines

Cheers,
Stephen Rothwell

Josh Poimboeuf

2016-03-03 03:46 UTC #19 lore raw

Re: [PATCH 2/2] objtool: Support CROSS_COMPILE

Quoted 25 lines · Stephen Rothwell

Hi Josh,
Quoted 15 lines · On Wed, 2 Mar 2016 21:20:58 -0600 Josh Poimboeuf <[email protected]> wrote:
Quoted 4 lines · Stephen Rothwell
I was wondering if this would be more appropriate in scripts/objtool since it is used during the building of the kernel. Or does it have a wider use?
Yeah, it was actually in the scripts/ dir in earlier revisions of the patch set, for that very reason. However, Ingo pointed out that it could be useful beyond the kernel, so we graduated it to a "tool".
Quoted 3 lines
We have HOSTCC with its associated HOSTCFLAGS etc ... I am not sure if that is more appropriate (but it does take care of people using clang).
Quoted 6 lines
The "tools" are almost completely separate from the rest of the kernel. They have their own scaled-down version of kbuild, which doesn't have HOSTCC.
But yeah, we might eventually need to copy some of the host compilation infrastructure from scripts/Makefile.host over to the tools/ side.

Quoted 6 lines

That all sounds sane, thanks.
I did not add this to linux-next today, but may tomorrow if people think it is sensible to do so (for testing on a powerpcle host). If I do, I will just back out to the previous patch if it all goes south (so it won't impact on the rest of the tip tree's testing).

Sounds good, thanks!

FWIW, I did test it on a ppc64le host with an x86 cross-compiler and it worked fine (but more testing is certainly welcome).

Signature · 1 line

Josh

Ingo Molnar

2016-03-03 15:10 UTC #20 lore raw

Re: [PATCH 2/2] objtool: Support CROSS_COMPILE

In reply to Stephen Rothwell

Quoted 25 lines · Stephen Rothwell

Hi Josh,
Quoted 15 lines · On Wed, 2 Mar 2016 21:20:58 -0600 Josh Poimboeuf <[email protected]> wrote:
Quoted 4 lines · Stephen Rothwell
I was wondering if this would be more appropriate in scripts/objtool since it is used during the building of the kernel. Or does it have a wider use?
Yeah, it was actually in the scripts/ dir in earlier revisions of the patch set, for that very reason. However, Ingo pointed out that it could be useful beyond the kernel, so we graduated it to a "tool".
Quoted 3 lines
We have HOSTCC with its associated HOSTCFLAGS etc ... I am not sure if that is more appropriate (but it does take care of people using clang).
Quoted 6 lines
The "tools" are almost completely separate from the rest of the kernel. They have their own scaled-down version of kbuild, which doesn't have HOSTCC.
But yeah, we might eventually need to copy some of the host compilation infrastructure from scripts/Makefile.host over to the tools/ side.

Quoted 6 lines

That all sounds sane, thanks.
I did not add this to linux-next today, but may tomorrow if people think it is sensible to do so (for testing on a powerpcle host). If I do, I will just back out to the previous patch if it all goes south (so it won't impact on the rest of the tip tree's testing).

I'll add Josh's fixes to -tip ASAP as well, so hopefully soon you can drop all linux-next specific patches related to this and it will all work Just Fine (tm).

Sorry about the breakage!

Thanks,

Ingo

Stephen Rothwell

author

2016-03-03 22:59 UTC #21 lore raw

Re: [PATCH 2/2] objtool: Support CROSS_COMPILE

Hi Ingo,

Quoted 3 lines · On Thu, 3 Mar 2016 16:10:21 +0100 Ingo Molnar <[email protected]> wrote:

I'll add Josh's fixes to -tip ASAP as well, so hopefully soon you can drop all linux-next specific patches related to this and it will all work Just Fine (tm).

Thanks for that. I have now dropped these patches.

Quoted 1 line

Sorry about the breakage!

Hey, it happens ..

Signature · 2 lines

Cheers,
Stephen Rothwell

H. Peter Anvin

reviewer

2016-03-03 15:28 UTC #22 lore raw

Re: [PATCH 2/2] objtool: Support CROSS_COMPILE

In reply to Stephen Rothwell

Quoted 20 lines · Stephen Rothwell

Hi Josh,
Just a couple of quick comments ...
tools/objtool/Makefile
Quoted 5 lines · On Wed, 2 Mar 2016 18:39:37 -0600 Josh Poimboeuf <[email protected]> wrote:
tools/objtool/Makefile +0 -0
index c4f0713..e4a6bd5 100644
I was wondering if this would be more appropriate in scripts/objtool since it is used during the building of the kernel. Or does it have a wider use?
Quoted 4 lines
tools/objtool/Makefile +0 -0
@@ -7,13 +7,19 @@ ARCH := x86
 endif
 endif
 

Quoted 5 lines

tools/objtool/Makefile ARCH := x86 · line 11
Quoted 2 lines
tools/objtool/Makefile +2 -0
+# always use the host compiler
+CC = gcc
We have HOSTCC with its associated HOSTCFLAGS etc ... I am not sure if that is more appropriate (but it does take care of people using clang).

This is what HOSTCC is for. And yes, this belongs in scripts.

Signature · 1 line

Sent from my Android device with K-9 Mail. Please excuse brevity and formatting.

tip-bot for Josh Poimboeuf

merge bot

2016-03-03 16:54 UTC #23 lore raw

[tip:core/objtool] objtool: Support CROSS_COMPILE

In reply to Josh Poimboeuf

objtool: Support CROSS_COMPILE

Commit-ID:  c1d45c3abd49b5bf9447e435099c1b000dcde752
Gitweb:     http://git.kernel.org/tip/c1d45c3abd49b5bf9447e435099c1b000dcde752
Author:     Josh Poimboeuf <[email protected]>
AuthorDate: Wed, 2 Mar 2016 18:39:37 -0600
Committer:  Ingo Molnar <[email protected]>
CommitDate: Thu, 3 Mar 2016 16:13:00 +0100

objtool: Support CROSS_COMPILE

When building with CONFIG_STACK_VALIDATION on a ppc64le host with an x86 cross-compiler, Stephen Rothwell saw the following objtool build errors:

    DESCEND  objtool
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/builtin-check.o
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/special.o
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/elf.o
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/objtool.o
    MKDIR    /home/sfr/next/x86_64_allmodconfig/tools/objtool/arch/x86/insn/
    CC       /home/sfr/next/x86_64_allmodconfig/tools/objtool/libstring.o

It fails to build because it tries to compile objtool with the cross-compiler instead of the host compiler.

Ensure that it always uses the host compiler by ignoring CROSS_COMPILE.

Reported-by Stephen Rothwell <[email protected]>

Signed-off-by Josh Poimboeuf <[email protected]>

Cc Adrian Hunter <[email protected]>

Cc Linus Torvalds <[email protected]>

Cc Masami Hiramatsu <[email protected]>

Cc Michael Ellerman <[email protected]>

Cc Peter Zijlstra <[email protected]>

Cc Thomas Gleixner <[email protected]>

Link http://lkml.kernel.org/r/55b63eefc347f1bb28573f972d8d1adbf1f1c31d.1456962210.git.jpoimboe@redhat.com open here

Signed-off-by Ingo Molnar <[email protected]>

---
 tools/lib/subcmd/Makefile |  6 ++++--
 tools/objtool/Makefile    | 17 ++++++++++-------
 2 files changed, 14 insertions(+), 9 deletions(-)

tools/lib/subcmd/Makefile +4 -2

index 629cf8c..1faecb8 100644
@@ -8,8 +8,10 @@ srctree := $(patsubst %/,%,$(dir $(srctree)))
 #$(info Determined 'srctree' to be $(srctree))
 endif
 
-CC = $(CROSS_COMPILE)gcc
-AR = $(CROSS_COMPILE)ar
+CC ?= $(CROSS_COMPILE)gcc
+LD ?= $(CROSS_COMPILE)ld
+AR ?= $(CROSS_COMPILE)ar
+
 RM = rm -f
 
 MAKEFLAGS += --no-print-directory

tools/objtool/Makefile +10 -7

index c4f0713..e4a6bd5 100644
@@ -7,13 +7,19 @@ ARCH := x86
 endif
 endif
 
+# always use the host compiler
+CC = gcc
+LD = ld
+AR = ar
+
 ifeq ($(srctree),)
 srctree := $(patsubst %/,%,$(dir $(shell pwd)))
 srctree := $(patsubst %/,%,$(dir $(srctree)))
 endif
 
-SUBCMD_SRCDIR	= $(srctree)/tools/lib/subcmd/
-LIBSUBCMD	= $(if $(OUTPUT),$(OUTPUT),$(SUBCMD_SRCDIR))libsubcmd.a
+SUBCMD_SRCDIR		= $(srctree)/tools/lib/subcmd/
+LIBSUBCMD_OUTPUT	= $(if $(OUTPUT),$(OUTPUT),$(PWD)/)
+LIBSUBCMD		= $(LIBSUBCMD_OUTPUT)libsubcmd.a
 
 OBJTOOL    := $(OUTPUT)objtool
 OBJTOOL_IN := $(OBJTOOL)-in.o

@@ -45,12 +51,9 @@ $(OBJTOOL): $(LIBSUBCMD) $(OBJTOOL_IN)
 
 
 $(LIBSUBCMD): fixdep FORCE
-	$(Q)$(MAKE) -C $(SUBCMD_SRCDIR)
-
-$(LIBSUBCMD)-clean:
-	$(Q)$(MAKE) -C $(SUBCMD_SRCDIR) clean > /dev/null
+	$(Q)$(MAKE) -C $(SUBCMD_SRCDIR) OUTPUT=$(LIBSUBCMD_OUTPUT)
 
-clean: $(LIBSUBCMD)-clean
+clean:
 	$(call QUIET_CLEAN, objtool) $(RM) $(OBJTOOL)
 	$(Q)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
 	$(Q)$(RM) $(OUTPUT)arch/x86/insn/inat-tables.c $(OUTPUT)fixdep

Sedat Dilek

2016-03-03 07:31 UTC #24 lore raw

Re: [PATCH] objtool: Disable stack validation when CROSS_COMPILE is used

In reply to Stephen Rothwell

Quoted 24 lines · Stephen Rothwell

Hi Josh,
Quoted 3 lines · On Tue, 1 Mar 2016 15:54:51 -0600 Josh Poimboeuf <[email protected]> wrote:
Changing it to use the host compiler would probably be an easy fix, but that would expose a harder bug related to endianness.
Just by luck, my PowerPC host is little endian :-)
Quoted 3 lines
How about the below workaround patch to disable objtool and warn when CROSS_COMPILE is used? If anybody complains about lack of cross-compile support later, we could try to fix it then.
This seems reasonable.
Quoted 7 lines
From a3c65947011a420743f308b698171c4209105d3f Mon Sep 17 00:00:00 2001 Message-Id:
<a3c65947011a420743f308b698171c4209105d3f.1456868910.git.jpoimboe@redhat.com>
From: Josh Poimboeuf <[email protected]>
Date: Tue, 1 Mar 2016 13:35:51 -0600
Subject: [PATCH] objtool: Disable stack validation when CROSS_COMPILE is used

Quoted 4 lines

I have applied this to the merge of the tip tree in linux-next today and it compiles fine for me. I will continue applying it until something better comes along or it is applied to the tip tree.

Does Linux next-20160303 has this patch?
On a quick view I could not find it.

- Sedat -

Quoted 9 lines

Thanks for that.
--
Cheers,
Stephen Rothwell
--
To unsubscribe from this list: send the line "unsubscribe linux-next" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Stephen Rothwell

author

2016-03-03 07:57 UTC #25 lore raw

Re: [PATCH] objtool: Disable stack validation when CROSS_COMPILE is used

Hi Sedat,

Quoted 3 lines · On Thu, 3 Mar 2016 08:31:57 +0100 Sedat Dilek <[email protected]> wrote:

Does Linux next-20160303 has this patch?
On a quick view I could not find it.

It is applied as part of the merge commit that merges the tip tree, so there is not a separate commit for it.

Signature · 2 lines

Cheers,
Stephen Rothwell