Link time optimization for LTO/x86

PATCH In review Andi Kleen 6 participants 31 messages Nov 27, 2017 – May 1, 2018

Subsystems (19)

Documentation Media (V4L/DVB) x86 VFS +15 more subsystems

lore

Share & tools

lore

Andi Kleen

author

2017-11-27 21:34 UTC #0 lore raw

This is an updated version of my older LTO patchkit for gcc/x86 This version doesn't need special binutils, but requires gcc 5+. It also is compatible with near all options (except MODVERSIONS)

This allows the compiler to optimize over source files and throw away unnecessary functions.

It also found various problems in source files, these are fixed in the first few patches.

There are still some minor issues, see the individual files. Also it still does double/triple link for KALLSYMS, which increases the build time

Available at

git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc lto-415-2

-Andi

Andi Kleen

author

2017-11-27 21:34 UTC #1 lore raw

PATCH 1/21 x86/xen: Mark pv stub assembler symbol visible

From: Andi Kleen <[email protected]>

With LTO any external assembler symbol has to be marked __visible. Mark the generated asm PV stubs __visible to prevent a linker error.

Signed-off-by Andi Kleen <[email protected]>

---
 arch/x86/include/asm/paravirt.h | 3 ++-
 drivers/xen/time.c              | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

arch/x86/include/asm/paravirt.h +2 -1

index 283efcaac8af..cf2861aa4df0 100644
@@ -745,7 +745,8 @@ static __always_inline bool pv_vcpu_is_preempted(long cpu)
  */
 #define PV_THUNK_NAME(func) "__raw_callee_save_" #func
 #define PV_CALLEE_SAVE_REGS_THUNK(func)					\
-	extern typeof(func) __raw_callee_save_##func;			\
+	extern __visible typeof(func) __raw_callee_save_##func;		\
+	extern __visible typeof(func) func;				\
 									\
 	asm(".pushsection .text;"					\
 	    ".globl " PV_THUNK_NAME(func) ";"				\

drivers/xen/time.c +1 -1

index 3e741cd1409c..708a00c337d7 100644
@@ -144,7 +144,7 @@ void xen_get_runstate_snapshot(struct vcpu_runstate_info *res)
 }
 
 /* return true when a vcpu could run but has no real cpu to run on */
-bool xen_vcpu_stolen(int vcpu)
+__visible bool xen_vcpu_stolen(int vcpu)
 {
 	return per_cpu(xen_runstate, vcpu).state == RUNSTATE_runnable;
 }

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #2 lore raw

PATCH 2/21 afs: Fix const confusion in AFS

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

A trace point string cannot be const because the underlying special section is not marked const. An LTO build complains about the section attribute mismatch. Fix it by not marking the trace point string in afs const.

Cc [email protected]

Signed-off-by Andi Kleen <[email protected]>

---
 fs/afs/cmservice.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

fs/afs/cmservice.c +1 -1

index 41e277f57b20..0e9ea0f8d620 100644
@@ -31,7 +31,7 @@ static void SRXAFSCB_ProbeUuid(struct work_struct *);
 static void SRXAFSCB_TellMeAboutYourself(struct work_struct *);
 
 #define CM_NAME(name) \
-	const char afs_SRXCB##name##_name[] __tracepoint_string =	\
+	char afs_SRXCB##name##_name[] __tracepoint_string =	\
 		"CB." #name
 
 /*

Signature · 1 line

2.13.6

David Howells

maintainer

2017-11-28 16:04 UTC #3 lore raw

Re: [PATCH 02/21] afs: Fix const confusion in AFS

Quoted 4 lines · Andi Kleen

A trace point string cannot be const because the underlying special section is not marked const. An LTO build complains about the section attribute mismatch. Fix it by not marking the trace point string in afs const.

Do you want this to go through my tree?

David

Andi Kleen

author

2017-11-28 16:50 UTC #4 lore raw

Re: [PATCH 02/21] afs: Fix const confusion in AFS

Quoted 8 lines · David Howells

Quoted 4 lines · Andi Kleen
A trace point string cannot be const because the underlying special section is not marked const. An LTO build complains about the section attribute mismatch. Fix it by not marking the trace point string in afs const.
Do you want this to go through my tree?

Yes please.

-Andi

Andi Kleen

author

2017-11-27 21:34 UTC #5 lore raw

PATCH 3/21 x86/timer: Don't inline __const_udelay

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

__const_udelay is marked inline, and LTO will happily inline it everywhere Dropping the inline saves ~44k text in a LTO build.

13999560        1740864 1499136 17239560        1070e08 vmlinux-with-udelay-inline
13954764        1736768 1499136 17190668        1064f0c vmlinux-wo-udelay-inline

Even without LTO I believe marking it noinline documents it correctly.

Signed-off-by Andi Kleen <[email protected]>

---
 arch/x86/lib/delay.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

arch/x86/lib/delay.c +1 -1

index 553f8fd23cc4..09c83b2f80d2 100644
@@ -162,7 +162,7 @@ void __delay(unsigned long loops)
 }
 EXPORT_SYMBOL(__delay);
 
-inline void __const_udelay(unsigned long xloops)
+void __const_udelay(unsigned long xloops)
 {
 	unsigned long lpj = this_cpu_read(cpu_info.loops_per_jiffy) ? : loops_per_jiffy;
 	int d0;

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #6 lore raw

PATCH 4/21 locking/spinlocks: Mark spinlocks noinline when inline spinlocks are disabled

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

Otherwise LTO will inline them anyways and cause a large kernel text increase.

Since the explicit intention here is to not inline them marking them noinline is good documentation even for the non LTO case.

Signed-off-by Andi Kleen <[email protected]>

---
 kernel/locking/spinlock.c | 56 +++++++++++++++++++++++------------------------
 1 file changed, 28 insertions(+), 28 deletions(-)

kernel/locking/spinlock.c +28 ~~-28~~

index 1fd1a7543cdd..09d0dce7fa5d 100644
@@ -130,7 +130,7 @@ BUILD_LOCK_OPS(write, rwlock);
 #endif
 
 #ifndef CONFIG_INLINE_SPIN_TRYLOCK
-int __lockfunc _raw_spin_trylock(raw_spinlock_t *lock)
+noinline int __lockfunc _raw_spin_trylock(raw_spinlock_t *lock)
 {
 	return __raw_spin_trylock(lock);
 }

@@ -138,7 +138,7 @@ EXPORT_SYMBOL(_raw_spin_trylock);
 #endif
 
 #ifndef CONFIG_INLINE_SPIN_TRYLOCK_BH
-int __lockfunc _raw_spin_trylock_bh(raw_spinlock_t *lock)
+noinline int __lockfunc _raw_spin_trylock_bh(raw_spinlock_t *lock)
 {
 	return __raw_spin_trylock_bh(lock);
 }

@@ -146,7 +146,7 @@ EXPORT_SYMBOL(_raw_spin_trylock_bh);
 #endif
 
 #ifndef CONFIG_INLINE_SPIN_LOCK
-void __lockfunc _raw_spin_lock(raw_spinlock_t *lock)
+noinline void __lockfunc _raw_spin_lock(raw_spinlock_t *lock)
 {
 	__raw_spin_lock(lock);
 }

@@ -154,7 +154,7 @@ EXPORT_SYMBOL(_raw_spin_lock);
 #endif
 
 #ifndef CONFIG_INLINE_SPIN_LOCK_IRQSAVE
-unsigned long __lockfunc _raw_spin_lock_irqsave(raw_spinlock_t *lock)
+noinline unsigned long __lockfunc _raw_spin_lock_irqsave(raw_spinlock_t *lock)
 {
 	return __raw_spin_lock_irqsave(lock);
 }

@@ -162,7 +162,7 @@ EXPORT_SYMBOL(_raw_spin_lock_irqsave);
 #endif
 
 #ifndef CONFIG_INLINE_SPIN_LOCK_IRQ
-void __lockfunc _raw_spin_lock_irq(raw_spinlock_t *lock)
+noinline void __lockfunc _raw_spin_lock_irq(raw_spinlock_t *lock)
 {
 	__raw_spin_lock_irq(lock);
 }

@@ -170,7 +170,7 @@ EXPORT_SYMBOL(_raw_spin_lock_irq);
 #endif
 
 #ifndef CONFIG_INLINE_SPIN_LOCK_BH
-void __lockfunc _raw_spin_lock_bh(raw_spinlock_t *lock)
+noinline void __lockfunc _raw_spin_lock_bh(raw_spinlock_t *lock)
 {
 	__raw_spin_lock_bh(lock);
 }

@@ -178,7 +178,7 @@ EXPORT_SYMBOL(_raw_spin_lock_bh);
 #endif
 
 #ifdef CONFIG_UNINLINE_SPIN_UNLOCK
-void __lockfunc _raw_spin_unlock(raw_spinlock_t *lock)
+noinline void __lockfunc _raw_spin_unlock(raw_spinlock_t *lock)
 {
 	__raw_spin_unlock(lock);
 }

@@ -186,7 +186,7 @@ EXPORT_SYMBOL(_raw_spin_unlock);
 #endif
 
 #ifndef CONFIG_INLINE_SPIN_UNLOCK_IRQRESTORE
-void __lockfunc _raw_spin_unlock_irqrestore(raw_spinlock_t *lock, unsigned long flags)
+noinline void __lockfunc _raw_spin_unlock_irqrestore(raw_spinlock_t *lock, unsigned long flags)
 {
 	__raw_spin_unlock_irqrestore(lock, flags);
 }

@@ -194,7 +194,7 @@ EXPORT_SYMBOL(_raw_spin_unlock_irqrestore);
 #endif
 
 #ifndef CONFIG_INLINE_SPIN_UNLOCK_IRQ
-void __lockfunc _raw_spin_unlock_irq(raw_spinlock_t *lock)
+noinline void __lockfunc _raw_spin_unlock_irq(raw_spinlock_t *lock)
 {
 	__raw_spin_unlock_irq(lock);
 }

@@ -202,7 +202,7 @@ EXPORT_SYMBOL(_raw_spin_unlock_irq);
 #endif
 
 #ifndef CONFIG_INLINE_SPIN_UNLOCK_BH
-void __lockfunc _raw_spin_unlock_bh(raw_spinlock_t *lock)
+noinline void __lockfunc _raw_spin_unlock_bh(raw_spinlock_t *lock)
 {
 	__raw_spin_unlock_bh(lock);
 }

@@ -210,7 +210,7 @@ EXPORT_SYMBOL(_raw_spin_unlock_bh);
 #endif
 
 #ifndef CONFIG_INLINE_READ_TRYLOCK
-int __lockfunc _raw_read_trylock(rwlock_t *lock)
+noinline int __lockfunc _raw_read_trylock(rwlock_t *lock)
 {
 	return __raw_read_trylock(lock);
 }

@@ -218,7 +218,7 @@ EXPORT_SYMBOL(_raw_read_trylock);
 #endif
 
 #ifndef CONFIG_INLINE_READ_LOCK
-void __lockfunc _raw_read_lock(rwlock_t *lock)
+noinline void __lockfunc _raw_read_lock(rwlock_t *lock)
 {
 	__raw_read_lock(lock);
 }

@@ -226,7 +226,7 @@ EXPORT_SYMBOL(_raw_read_lock);
 #endif
 
 #ifndef CONFIG_INLINE_READ_LOCK_IRQSAVE
-unsigned long __lockfunc _raw_read_lock_irqsave(rwlock_t *lock)
+noinline unsigned long __lockfunc _raw_read_lock_irqsave(rwlock_t *lock)
 {
 	return __raw_read_lock_irqsave(lock);
 }

@@ -234,7 +234,7 @@ EXPORT_SYMBOL(_raw_read_lock_irqsave);
 #endif
 
 #ifndef CONFIG_INLINE_READ_LOCK_IRQ
-void __lockfunc _raw_read_lock_irq(rwlock_t *lock)
+noinline void __lockfunc _raw_read_lock_irq(rwlock_t *lock)
 {
 	__raw_read_lock_irq(lock);
 }

@@ -242,7 +242,7 @@ EXPORT_SYMBOL(_raw_read_lock_irq);
 #endif
 
 #ifndef CONFIG_INLINE_READ_LOCK_BH
-void __lockfunc _raw_read_lock_bh(rwlock_t *lock)
+noinline void __lockfunc _raw_read_lock_bh(rwlock_t *lock)
 {
 	__raw_read_lock_bh(lock);
 }

@@ -250,7 +250,7 @@ EXPORT_SYMBOL(_raw_read_lock_bh);
 #endif
 
 #ifndef CONFIG_INLINE_READ_UNLOCK
-void __lockfunc _raw_read_unlock(rwlock_t *lock)
+noinline void __lockfunc _raw_read_unlock(rwlock_t *lock)
 {
 	__raw_read_unlock(lock);
 }

@@ -258,7 +258,7 @@ EXPORT_SYMBOL(_raw_read_unlock);
 #endif
 
 #ifndef CONFIG_INLINE_READ_UNLOCK_IRQRESTORE
-void __lockfunc _raw_read_unlock_irqrestore(rwlock_t *lock, unsigned long flags)
+noinline void __lockfunc _raw_read_unlock_irqrestore(rwlock_t *lock, unsigned long flags)
 {
 	__raw_read_unlock_irqrestore(lock, flags);
 }

@@ -266,7 +266,7 @@ EXPORT_SYMBOL(_raw_read_unlock_irqrestore);
 #endif
 
 #ifndef CONFIG_INLINE_READ_UNLOCK_IRQ
-void __lockfunc _raw_read_unlock_irq(rwlock_t *lock)
+noinline void __lockfunc _raw_read_unlock_irq(rwlock_t *lock)
 {
 	__raw_read_unlock_irq(lock);
 }

@@ -274,7 +274,7 @@ EXPORT_SYMBOL(_raw_read_unlock_irq);
 #endif
 
 #ifndef CONFIG_INLINE_READ_UNLOCK_BH
-void __lockfunc _raw_read_unlock_bh(rwlock_t *lock)
+noinline void __lockfunc _raw_read_unlock_bh(rwlock_t *lock)
 {
 	__raw_read_unlock_bh(lock);
 }

@@ -282,7 +282,7 @@ EXPORT_SYMBOL(_raw_read_unlock_bh);
 #endif
 
 #ifndef CONFIG_INLINE_WRITE_TRYLOCK
-int __lockfunc _raw_write_trylock(rwlock_t *lock)
+noinline int __lockfunc _raw_write_trylock(rwlock_t *lock)
 {
 	return __raw_write_trylock(lock);
 }

@@ -290,7 +290,7 @@ EXPORT_SYMBOL(_raw_write_trylock);
 #endif
 
 #ifndef CONFIG_INLINE_WRITE_LOCK
-void __lockfunc _raw_write_lock(rwlock_t *lock)
+noinline void __lockfunc _raw_write_lock(rwlock_t *lock)
 {
 	__raw_write_lock(lock);
 }

@@ -298,7 +298,7 @@ EXPORT_SYMBOL(_raw_write_lock);
 #endif
 
 #ifndef CONFIG_INLINE_WRITE_LOCK_IRQSAVE
-unsigned long __lockfunc _raw_write_lock_irqsave(rwlock_t *lock)
+noinline unsigned long __lockfunc _raw_write_lock_irqsave(rwlock_t *lock)
 {
 	return __raw_write_lock_irqsave(lock);
 }

@@ -306,7 +306,7 @@ EXPORT_SYMBOL(_raw_write_lock_irqsave);
 #endif
 
 #ifndef CONFIG_INLINE_WRITE_LOCK_IRQ
-void __lockfunc _raw_write_lock_irq(rwlock_t *lock)
+noinline void __lockfunc _raw_write_lock_irq(rwlock_t *lock)
 {
 	__raw_write_lock_irq(lock);
 }

@@ -314,7 +314,7 @@ EXPORT_SYMBOL(_raw_write_lock_irq);
 #endif
 
 #ifndef CONFIG_INLINE_WRITE_LOCK_BH
-void __lockfunc _raw_write_lock_bh(rwlock_t *lock)
+noinline void __lockfunc _raw_write_lock_bh(rwlock_t *lock)
 {
 	__raw_write_lock_bh(lock);
 }

@@ -322,7 +322,7 @@ EXPORT_SYMBOL(_raw_write_lock_bh);
 #endif
 
 #ifndef CONFIG_INLINE_WRITE_UNLOCK
-void __lockfunc _raw_write_unlock(rwlock_t *lock)
+noinline void __lockfunc _raw_write_unlock(rwlock_t *lock)
 {
 	__raw_write_unlock(lock);
 }

@@ -330,7 +330,7 @@ EXPORT_SYMBOL(_raw_write_unlock);
 #endif
 
 #ifndef CONFIG_INLINE_WRITE_UNLOCK_IRQRESTORE
-void __lockfunc _raw_write_unlock_irqrestore(rwlock_t *lock, unsigned long flags)
+noinline void __lockfunc _raw_write_unlock_irqrestore(rwlock_t *lock, unsigned long flags)
 {
 	__raw_write_unlock_irqrestore(lock, flags);
 }

@@ -338,7 +338,7 @@ EXPORT_SYMBOL(_raw_write_unlock_irqrestore);
 #endif
 
 #ifndef CONFIG_INLINE_WRITE_UNLOCK_IRQ
-void __lockfunc _raw_write_unlock_irq(rwlock_t *lock)
+noinline void __lockfunc _raw_write_unlock_irq(rwlock_t *lock)
 {
 	__raw_write_unlock_irq(lock);
 }

@@ -346,7 +346,7 @@ EXPORT_SYMBOL(_raw_write_unlock_irq);
 #endif
 
 #ifndef CONFIG_INLINE_WRITE_UNLOCK_BH
-void __lockfunc _raw_write_unlock_bh(rwlock_t *lock)
+noinline void __lockfunc _raw_write_unlock_bh(rwlock_t *lock)
 {
 	__raw_write_unlock_bh(lock);
 }

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #7 lore raw

PATCH 5/21 x86/kvm: Make steal_time visible

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

This per cpu variable is accessed from assembler code, so needs to be visible.

Signed-off-by Andi Kleen <[email protected]>

---
 arch/x86/kernel/kvm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

arch/x86/kernel/kvm.c +1 -1

index b40ffbf156c1..8484e3e41d36 100644
@@ -76,7 +76,7 @@ static int parse_no_kvmclock_vsyscall(char *arg)
 early_param("no-kvmclock-vsyscall", parse_no_kvmclock_vsyscall);
 
 static DEFINE_PER_CPU_DECRYPTED(struct kvm_vcpu_pv_apf_data, apf_reason) __aligned(64);
-static DEFINE_PER_CPU_DECRYPTED(struct kvm_steal_time, steal_time) __aligned(64);
+DEFINE_PER_CPU_DECRYPTED(struct kvm_steal_time, steal_time) __aligned(64) __visible;
 static int has_steal_clock = 0;
 
 /*

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #8 lore raw

PATCH 6/21 x86/syscalls: Make x86 syscalls use real prototypes

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

LTO complains very loudly that the x86 syscalls use their own different prototypes. Switch it to use the real prototypes instead. This requires adding a few extra prototypes to asm/syscalls.h.

This is a generic cleanup, useful even without LTO.

Signed-off-by Andi Kleen <[email protected]>

---
 arch/x86/entry/syscall_32.c     | 13 +++++--------
 arch/x86/entry/syscall_64.c     | 13 +++++--------
 arch/x86/include/asm/syscalls.h | 42 ++++++++++++++++++++++++++++++++++++-----
 3 files changed, 47 insertions(+), 21 deletions(-)

arch/x86/entry/syscall_32.c +5 -8

index 95c294963612..f7ffd4b100fd 100644
@@ -6,20 +6,17 @@
 #include <linux/cache.h>
 #include <asm/asm-offsets.h>
 #include <asm/syscall.h>
+#include <linux/syscalls.h>
+#include <asm/syscalls.h>
+#include <asm/sys_ia32.h>
 
-#define __SYSCALL_I386(nr, sym, qual) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;
-#include <asm/syscalls_32.h>
-#undef __SYSCALL_I386
-
-#define __SYSCALL_I386(nr, sym, qual) [nr] = sym,
-
-extern asmlinkage long sys_ni_syscall(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long);
+#define __SYSCALL_I386(nr, sym, qual) [nr] = (sys_call_ptr_t)sym,
 
 __visible const sys_call_ptr_t ia32_sys_call_table[__NR_syscall_compat_max+1] = {
 	/*
 	 * Smells like a compiler bug -- it doesn't work
 	 * when the & below is removed.
 	 */
-	[0 ... __NR_syscall_compat_max] = &sys_ni_syscall,
+	[0 ... __NR_syscall_compat_max] = (sys_call_ptr_t)&sys_ni_syscall,
 #include <asm/syscalls_32.h>
 };

arch/x86/entry/syscall_64.c +5 -8

index 9c09775e589d..1ccf4cd300a6 100644
@@ -4,25 +4,22 @@
 #include <linux/linkage.h>
 #include <linux/sys.h>
 #include <linux/cache.h>
+#include <linux/syscalls.h>
+#include <linux/compat.h>
 #include <asm/asm-offsets.h>
 #include <asm/syscall.h>
+#include <asm/syscalls.h>
 
 #define __SYSCALL_64_QUAL_(sym) sym
 #define __SYSCALL_64_QUAL_ptregs(sym) ptregs_##sym
 
-#define __SYSCALL_64(nr, sym, qual) extern asmlinkage long __SYSCALL_64_QUAL_##qual(sym)(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long);
-#include <asm/syscalls_64.h>
-#undef __SYSCALL_64
-
-#define __SYSCALL_64(nr, sym, qual) [nr] = __SYSCALL_64_QUAL_##qual(sym),
-
-extern long sys_ni_syscall(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long);
+#define __SYSCALL_64(nr, sym, qual) [nr] = (sys_call_ptr_t)__SYSCALL_64_QUAL_##qual(sym),
 
 asmlinkage const sys_call_ptr_t sys_call_table[__NR_syscall_max+1] = {
 	/*
 	 * Smells like a compiler bug -- it doesn't work
 	 * when the & below is removed.
 	 */
-	[0 ... __NR_syscall_max] = &sys_ni_syscall,
+	[0 ... __NR_syscall_max] = (sys_call_ptr_t)&sys_ni_syscall,
 #include <asm/syscalls_64.h>
 };

arch/x86/include/asm/syscalls.h +37 -5

62 lines not on this page · raw message

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #9 lore raw

PATCH 7/21 x86: Make exception handler functions visible

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

Make the C exception handler functions that are directly called through exception tables visible. LTO needs to know they are accessed from assembler.

Signed-off-by Andi Kleen <[email protected]>

---
 arch/x86/mm/extable.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

arch/x86/mm/extable.c +9 -8

index 3321b446b66c..abe60607e8b9 100644
@@ -20,7 +20,7 @@ ex_fixup_handler(const struct exception_table_entry *x)
 	return (ex_handler_t)((unsigned long)&x->handler + x->handler);
 }
 
-bool ex_handler_default(const struct exception_table_entry *fixup,
+__visible bool ex_handler_default(const struct exception_table_entry *fixup,
 		       struct pt_regs *regs, int trapnr)
 {
 	regs->ip = ex_fixup_addr(fixup);

@@ -28,7 +28,7 @@ bool ex_handler_default(const struct exception_table_entry *fixup,
 }
 EXPORT_SYMBOL(ex_handler_default);
 
-bool ex_handler_fault(const struct exception_table_entry *fixup,
+__visible bool ex_handler_fault(const struct exception_table_entry *fixup,
 		     struct pt_regs *regs, int trapnr)
 {
 	regs->ip = ex_fixup_addr(fixup);

@@ -41,7 +41,7 @@ EXPORT_SYMBOL_GPL(ex_handler_fault);
  * Handler for UD0 exception following a failed test against the
  * result of a refcount inc/dec/add/sub.
  */
-bool ex_handler_refcount(const struct exception_table_entry *fixup,
+__visible bool ex_handler_refcount(const struct exception_table_entry *fixup,
 			 struct pt_regs *regs, int trapnr)
 {
 	/* First unconditionally saturate the refcount. */

@@ -94,6 +94,7 @@ EXPORT_SYMBOL_GPL(ex_handler_refcount);
  * of vulnerability by restoring from the initial state (essentially, zeroing
  * out all the FPU registers) if we can't restore from the task's FPU state.
  */
+__visible
 bool ex_handler_fprestore(const struct exception_table_entry *fixup,
 			  struct pt_regs *regs, int trapnr)
 {

@@ -107,7 +108,7 @@ bool ex_handler_fprestore(const struct exception_table_entry *fixup,
 }
 EXPORT_SYMBOL_GPL(ex_handler_fprestore);
 
-bool ex_handler_ext(const struct exception_table_entry *fixup,
+__visible bool ex_handler_ext(const struct exception_table_entry *fixup,
 		   struct pt_regs *regs, int trapnr)
 {
 	/* Special hack for uaccess_err */

@@ -117,7 +118,7 @@ bool ex_handler_ext(const struct exception_table_entry *fixup,
 }
 EXPORT_SYMBOL(ex_handler_ext);
 
-bool ex_handler_rdmsr_unsafe(const struct exception_table_entry *fixup,
+__visible bool ex_handler_rdmsr_unsafe(const struct exception_table_entry *fixup,
 			     struct pt_regs *regs, int trapnr)
 {
 	if (pr_warn_once("unchecked MSR access error: RDMSR from 0x%x at rIP: 0x%lx (%pF)\n",

@@ -132,7 +133,7 @@ bool ex_handler_rdmsr_unsafe(const struct exception_table_entry *fixup,
 }
 EXPORT_SYMBOL(ex_handler_rdmsr_unsafe);
 
-bool ex_handler_wrmsr_unsafe(const struct exception_table_entry *fixup,
+__visible bool ex_handler_wrmsr_unsafe(const struct exception_table_entry *fixup,
 			     struct pt_regs *regs, int trapnr)
 {
 	if (pr_warn_once("unchecked MSR access error: WRMSR to 0x%x (tried to write 0x%08x%08x) at rIP: 0x%lx (%pF)\n",

@@ -146,7 +147,7 @@ bool ex_handler_wrmsr_unsafe(const struct exception_table_entry *fixup,
 }
 EXPORT_SYMBOL(ex_handler_wrmsr_unsafe);
 
-bool ex_handler_clear_fs(const struct exception_table_entry *fixup,
+__visible bool ex_handler_clear_fs(const struct exception_table_entry *fixup,
 			 struct pt_regs *regs, int trapnr)
 {
 	if (static_cpu_has(X86_BUG_NULL_SEG))

@@ -156,7 +157,7 @@ bool ex_handler_clear_fs(const struct exception_table_entry *fixup,
 }
 EXPORT_SYMBOL(ex_handler_clear_fs);
 
-bool ex_has_fault_handler(unsigned long ip)
+__visible bool ex_has_fault_handler(unsigned long ip)
 {
 	const struct exception_table_entry *e;
 	ex_handler_t handler;

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #10 lore raw

PATCH 8/21 x86/idt: Make const __initconst

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

const variables must use __initconst, not __initdata. Fix this up for the new IDT tables recently added, which got it consistently wrong.

Fixes a whole range of commits between
16bc18d895ce x86/idt: Move 32-bit idt_descr to C code
and
dc20b2d52653 x86/idt: Move interrupt gate initialization to IDT code

Signed-off-by Andi Kleen <[email protected]>

---
 arch/x86/kernel/idt.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

arch/x86/kernel/idt.c +6 -6

index d985cef3984f..56d99be3706a 100644
@@ -56,7 +56,7 @@ struct idt_data {
  * Early traps running on the DEFAULT_STACK because the other interrupt
  * stacks work only after cpu_init().
  */
-static const __initdata struct idt_data early_idts[] = {
+static const __initconst struct idt_data early_idts[] = {
 	INTG(X86_TRAP_DB,		debug),
 	SYSG(X86_TRAP_BP,		int3),
 #ifdef CONFIG_X86_32

@@ -70,7 +70,7 @@ static const __initdata struct idt_data early_idts[] = {
  * the traps which use them are reinitialized with IST after cpu_init() has
  * set up TSS.
  */
-static const __initdata struct idt_data def_idts[] = {
+static const __initconst struct idt_data def_idts[] = {
 	INTG(X86_TRAP_DE,		divide_error),
 	INTG(X86_TRAP_NMI,		nmi),
 	INTG(X86_TRAP_BR,		bounds),

@@ -108,7 +108,7 @@ static const __initdata struct idt_data def_idts[] = {
 /*
  * The APIC and SMP idt entries
  */
-static const __initdata struct idt_data apic_idts[] = {
+static const __initconst struct idt_data apic_idts[] = {
 #ifdef CONFIG_SMP
 	INTG(RESCHEDULE_VECTOR,		reschedule_interrupt),
 	INTG(CALL_FUNCTION_VECTOR,	call_function_interrupt),

@@ -150,7 +150,7 @@ static const __initdata struct idt_data apic_idts[] = {
  * Early traps running on the DEFAULT_STACK because the other interrupt
  * stacks work only after cpu_init().
  */
-static const __initdata struct idt_data early_pf_idts[] = {
+static const __initconst struct idt_data early_pf_idts[] = {
 	INTG(X86_TRAP_PF,		page_fault),
 };

@@ -158,7 +158,7 @@ static const __initdata struct idt_data early_pf_idts[] = {
  * Override for the debug_idt. Same as the default, but with interrupt
  * stack set to DEFAULT_STACK (0). Required for NMI trap handling.
  */
-static const __initdata struct idt_data dbg_idts[] = {
+static const __initconst struct idt_data dbg_idts[] = {
 	INTG(X86_TRAP_DB,	debug),
 	INTG(X86_TRAP_BP,	int3),
 };

@@ -180,7 +180,7 @@ gate_desc debug_idt_table[IDT_ENTRIES] __page_aligned_bss;
  * The exceptions which use Interrupt stacks. They are setup after
  * cpu_init() when the TSS has been initialized.
  */
-static const __initdata struct idt_data ist_idts[] = {
+static const __initconst struct idt_data ist_idts[] = {
 	ISTG(X86_TRAP_DB,	debug,		DEBUG_STACK),
 	ISTG(X86_TRAP_NMI,	nmi,		NMI_STACK),
 	SISTG(X86_TRAP_BP,	int3,		DEBUG_STACK),

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #11 lore raw

PATCH 9/21 lto: Use C version for SYSCALL_ALIAS

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

LTO doesn't like the assembler aliasing used for SYSCALL_ALIAS. Replace it with C aliasing. Also mark the only user visible.

Signed-off-by Andi Kleen <[email protected]>

---
 include/linux/linkage.h   | 6 ++----
 kernel/time/posix-stubs.c | 2 +-
 2 files changed, 3 insertions(+), 5 deletions(-)

include/linux/linkage.h +2 -4

index f68db9e450eb..24fb6468b1a5 100644
@@ -30,10 +30,8 @@
 #endif
 
 #ifndef SYSCALL_ALIAS
-#define SYSCALL_ALIAS(alias, name) asm(			\
-	".globl " VMLINUX_SYMBOL_STR(alias) "\n\t"	\
-	".set   " VMLINUX_SYMBOL_STR(alias) ","		\
-		  VMLINUX_SYMBOL_STR(name))
+#define SYSCALL_ALIAS(a, name) \
+	__visible typeof(a) a __attribute__((alias(__stringify(name))))
 #endif
 
 #define __page_aligned_data	__section(.data..page_aligned) __aligned(PAGE_SIZE)

kernel/time/posix-stubs.c +1 -1

index b258bee13b02..5e36cb75f3be 100644
@@ -19,7 +19,7 @@
 #include <linux/posix-timers.h>
 #include <linux/compat.h>
 
-asmlinkage long sys_ni_posix_timers(void)
+__visible asmlinkage long sys_ni_posix_timers(void)
 {
 	pr_err_once("process %d (%s) attempted a POSIX timer syscall "
 		    "while CONFIG_POSIX_TIMERS is not set\n",

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #12 lore raw

PATCH 10/21 Fix read buffer overflow in delta-ipc

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

The single caller passes a string to delta_ipc_open, which copies with a fixed size larger than the string. So it copies some random data after the original string the ro segment.

If the string was at the end of a page it may fault.

Just copy the string with a normal strcpy after clearing the field.

Found by a LTO build (which errors out)
because the compiler inlines the functions and can resolve the string sizes and triggers the compile time checks in memcpy.

In function ‘memcpy’,
inlined from ‘delta_ipc_open.constprop’ at linux/drivers/media/platform/sti/delta/delta-ipc.c:178:0,
inlined from ‘delta_mjpeg_ipc_open’ at linux/drivers/media/platform/sti/delta/delta-mjpeg-dec.c:227:0,
inlined from ‘delta_mjpeg_decode’ at linux/drivers/media/platform/sti/delta/delta-mjpeg-dec.c:403:0:
/home/andi/lsrc/linux/include/linux/string.h:337:0: error: call to ‘__read_overflow2’ declared with attribute error: detected read beyond size of object passed as 2nd parameter

    __read_overflow2();

Cc [email protected]

Signed-off-by Andi Kleen <[email protected]>

---
 arch/x86/platform/intel-mid/device_libs/platform_bt.c | 2 +-
 certs/blacklist_nohashes.c                            | 2 +-
 drivers/media/platform/sti/delta/delta-ipc.c          | 4 ++--
 3 files changed, 4 insertions(+), 4 deletions(-)

arch/x86/platform/intel-mid/device_libs/platform_bt.c +1 -1

index dc036e511f48..2b5d86ce24c2 100644
@@ -60,7 +60,7 @@ static int __init tng_bt_sfi_setup(struct bt_sfi_data *ddata)
 	return 0;
 }
 
-static const struct bt_sfi_data tng_bt_sfi_data __initdata = {
+static const struct bt_sfi_data tng_bt_sfi_data __initconst = {
 	.setup	= tng_bt_sfi_setup,
 };

certs/blacklist_nohashes.c +1 -1

index 73fd99098ad7..753b703ef0ef 100644
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 #include "blacklist.h"
 
-const char __initdata *const blacklist_hashes[] = {
+const char __initconst *const blacklist_hashes[] = {
 	NULL
 };

drivers/media/platform/sti/delta/delta-ipc.c +2 -2

index 41e4a4c259b3..b6c256e3ceb6 100644
@@ -175,8 +175,8 @@ int delta_ipc_open(struct delta_ctx *pctx, const char *name,
 	msg.ipc_buf_size = ipc_buf_size;
 	msg.ipc_buf_paddr = ctx->ipc_buf->paddr;
 
-	memcpy(msg.name, name, sizeof(msg.name));
-	msg.name[sizeof(msg.name) - 1] = 0;
+	memset(msg.name, 0, sizeof(msg.name));
+	strcpy(msg.name, name);
 
 	msg.param_size = param->size;
 	memcpy(ctx->ipc_buf->vaddr, param->data, msg.param_size);

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #13 lore raw

PATCH 11/21 trace: Use -mcount-record for dynamic ftrace

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

gcc 5 supports a new -mcount-record option to generate ftrace tables directly. This avoids the need to run record_mcount manually.

Use this option when available.

So far doesn't use -mcount-nop, which also exists now.

This is needed to make ftrace work with LTO because the normal record-mcount script doesn't run over the link time output.

It should also improve build times slightly in the general case.

Cc [email protected]

Signed-off-by Andi Kleen <[email protected]>

---
 scripts/Makefile.build | 6 ++++++
 1 file changed, 6 insertions(+)

scripts/Makefile.build +6 -0

index cb8997ed0149..8179563bcd85 100644
@@ -219,6 +219,11 @@ cmd_modversions_c =								\
 endif
 
 ifdef CONFIG_FTRACE_MCOUNT_RECORD
+# gcc 5 supports generating the mcount tables directly
+ifneq ($(call cc-option,-mrecord-mcount,y),y)
+KBUILD_CFLAGS += -mrecord-mcount
+else
+# else do it all manually
 ifdef BUILD_C_RECORDMCOUNT
 ifeq ("$(origin RECORDMCOUNT_WARN)", "command line")
   RECORDMCOUNT_FLAGS = -w

@@ -264,6 +269,7 @@ objtool_args += --no-unreachable
 else
 objtool_args += $(call cc-ifversion, -lt, 0405, --no-unreachable)
 endif
+endif
 
 # 'OBJECT_FILES_NON_STANDARD := y': skip objtool checking for a directory
 # 'OBJECT_FILES_NON_STANDARD_foo.o := 'y': skip objtool checking for a file

Signature · 1 line

2.13.6

Steven Rostedt

maintainer

2017-12-01 00:22 UTC #14 lore raw

Re: [PATCH 11/21] trace: Use -mcount-record for dynamic ftrace

Quoted 14 lines · On Mon, 27 Nov 2017 13:34:13 -0800 Andi Kleen <[email protected]> wrote:

From: Andi Kleen <[email protected]>
gcc 5 supports a new -mcount-record option to generate ftrace tables directly. This avoids the need to run record_mcount manually.
Use this option when available.
So far doesn't use -mcount-nop, which also exists now.
This is needed to make ftrace work with LTO because the normal record-mcount script doesn't run over the link time output.

Quoted 6 lines

It should also improve build times slightly in the general case.
Cc [email protected]
Signed-off-by Andi Kleen <[email protected]>
---

Hi Andi,

Thanks for sending this. I'm currently trying to catch up on all the changes that need to get into 4.15 and will be doing a bit of traveling.

I plan on looking at this in a week or two, and add it to the 4.16 queue.

-- Steve

Steven Rostedt

maintainer

2018-05-01 18:42 UTC #15 lore raw

Re: [PATCH 11/21] trace: Use -mcount-record for dynamic ftrace

Quoted 24 lines · On Thu, 30 Nov 2017 19:22:33 -0500 Steven Rostedt <[email protected]> wrote:

Quoted 14 lines · On Mon, 27 Nov 2017 13:34:13 -0800 Andi Kleen <[email protected]> wrote:
From: Andi Kleen <[email protected]>
gcc 5 supports a new -mcount-record option to generate ftrace tables directly. This avoids the need to run record_mcount manually.
Use this option when available.
So far doesn't use -mcount-nop, which also exists now.
This is needed to make ftrace work with LTO because the normal record-mcount script doesn't run over the link time output.
Quoted 6 lines
It should also improve build times slightly in the general case.
Cc [email protected]
Signed-off-by Andi Kleen <[email protected]>
---

Quoted 9 lines

Hi Andi,
Thanks for sending this. I'm currently trying to catch up on all the changes that need to get into 4.15 and will be doing a bit of traveling.
I plan on looking at this in a week or two, and add it to the 4.16 queue.

Looks like it I forgot to add it to the queue (actually, I remember having issues with that queue and restarted it), and this was dropped.

Adding it this time for 4.18

Sorry about that. :-/

-- Steve

Andi Kleen

author

2017-11-27 21:34 UTC #16 lore raw

PATCH 12/21 ftrace: Mark function tracer test functions noinline/noclone

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

The ftrace function tracer self tests calls some functions to verify the get traced. This relies on them not being inlined. Previously this was ensured by putting them into another file, but with LTO the compiler can inline across files, which makes the tests fail.

Mark these functions as noinline and noclone.

Signed-off-by Andi Kleen <[email protected]>

---
 kernel/trace/trace_selftest_dynamic.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

kernel/trace/trace_selftest_dynamic.c +3 -2

index 8cda06a10d66..c364cf777e1a 100644
@@ -1,13 +1,14 @@
 // SPDX-License-Identifier: GPL-2.0
+#include <linux/compiler.h>
 #include "trace.h"
 
-int DYN_FTRACE_TEST_NAME(void)
+noinline __noclone int DYN_FTRACE_TEST_NAME(void)
 {
 	/* used to call mcount */
 	return 0;
 }
 
-int DYN_FTRACE_TEST_NAME2(void)
+noinline __noclone int DYN_FTRACE_TEST_NAME2(void)
 {
 	/* used to call mcount */
 	return 0;

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #17 lore raw

PATCH 13/21 ftrace: Disable LTO for ftrace self tests

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

Even when the test functions are not inlined something makes the ftrace self tests fail with LTO. ftrace manually tests seems to work fine. Disable LTO for the self test file, which makes the self tests work again.

Signed-off-by Andi Kleen <[email protected]>

---
 kernel/trace/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

kernel/trace/Makefile +1 -1

index e2538c7638d4..a471a08305e9 100644
@@ -8,7 +8,7 @@ KBUILD_CFLAGS = $(subst $(CC_FLAGS_FTRACE),,$(ORIG_CFLAGS))
 
 ifdef CONFIG_FTRACE_SELFTEST
 # selftest needs instrumentation
-CFLAGS_trace_selftest_dynamic.o = $(CC_FLAGS_FTRACE)
+CFLAGS_trace_selftest_dynamic.o = $(CC_FLAGS_FTRACE) ${DISABLE_LTO}
 obj-y += trace_selftest_dynamic.o
 endif
 endif

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #18 lore raw

PATCH 14/21 lto, fs: Avoid static variable in linux/fs.h

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

linux/fs.h has a initialized static variable kernel_read_file_str. It doesn't make much sense to have a static variable in a frequently included header file. With LTO -fno-toplevel-reorder gcc is unable to eliminate it, which leads to a lot of unnecessary duplicated copies.

Move the static into the scope of the only inline that uses it, this tells the compiler enough to not duplicate it. Right now the inline is only called from one place, so that is ok. If it was called from more places would need to move it somewhere else to avoid unnecessary copies.

With LTO this avoids ~100k unnecessary data segment for a x86 defconfig build. Even without LTO it doesn't make any sense.

Cc [email protected]

Signed-off-by Andi Kleen <[email protected]>

---
 include/linux/fs.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

include/linux/fs.h +4 -4

index 2995a271ec46..2f02f1c991c9 100644
@@ -2821,12 +2821,12 @@ enum kernel_read_file_id {
 	__kernel_read_file_id(__fid_enumify)
 };
 
-static const char * const kernel_read_file_str[] = {
-	__kernel_read_file_id(__fid_stringify)
-};
-
 static inline const char *kernel_read_file_id_str(enum kernel_read_file_id id)
 {
+	static const char * const kernel_read_file_str[] = {
+		__kernel_read_file_id(__fid_stringify)
+	};
+
 	if ((unsigned)id >= READING_MAX_ID)
 		return kernel_read_file_str[READING_UNKNOWN];

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #19 lore raw

PATCH 15/21 lto, x86, mm: Disable vmalloc BUILD_BUG_ON for LTO

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

On 32bit builds this BUILD_BUG_ON often fires with LTO for unknown reasons. As far as I can tell it's a false positive. So disable it for LTO.

Signed-off-by Andi Kleen <[email protected]>

---
 arch/x86/mm/init_32.c | 2 ++
 1 file changed, 2 insertions(+)

arch/x86/mm/init_32.c +2 -0

index 8a64a6f2848d..8187d8ee98ee 100644
@@ -808,7 +808,9 @@ void __init mem_init(void)
 	BUILD_BUG_ON(VMALLOC_END			> PKMAP_BASE);
 #endif
 #define high_memory (-128UL << 20)
+#ifndef CONFIG_LTO
 	BUILD_BUG_ON(VMALLOC_START			>= VMALLOC_END);
+#endif
 #undef high_memory
 #undef __FIXADDR_TOP

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #20 lore raw

PATCH 16/21 lto: Add __noreorder and mark initcalls __noreorder

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

gcc 5 has a new no_reorder attribute that prevents top level reordering only for that symbol.

Kernels don't like any reordering of initcalls between files, as several initcalls depend on each other. LTO previously needed to use -fno-toplevel-reordering to prevent boot failures.

Add a __noreorder wrapper for the no_reorder attribute and use it for initcalls.

Signed-off-by Andi Kleen <[email protected]>

---
 include/linux/compiler-gcc.h   | 5 +++++
 include/linux/compiler_types.h | 3 +++
 include/linux/init.h           | 2 +-
 3 files changed, 9 insertions(+), 1 deletion(-)

include/linux/compiler-gcc.h +5 -0

index 2272ded07496..0d562baeb379 100644
@@ -324,6 +324,11 @@
 #define __no_sanitize_address
 #endif
 
+#if __GNUC__ >= 5
+/* Avoid reordering a top level statement */
+#define __noreorder    __attribute__((no_reorder))
+#endif
+
 /*
  * A trick to suppress uninitialized variable warning without generating any
  * code

include/linux/compiler_types.h +3 -0

index 6b79a9bba9a7..fe4604603f5e 100644
@@ -260,6 +260,9 @@ struct ftrace_likely_data {
 #define __assume_aligned(a, ...)
 #endif
 
+#ifndef __noreorder
+#define __noreorder
+#endif
 
 /* Are two types/vars the same type (ignoring qualifiers)? */
 #ifndef __same_type

include/linux/init.h +1 -1

index ea1b31101d9e..abf1980367b7 100644
@@ -161,7 +161,7 @@ extern bool initcall_debug;
  */
 
 #define __define_initcall(fn, id) \
-	static initcall_t __initcall_##fn##id __used \
+	static initcall_t __initcall_##fn##id __used __noreorder \
 	__attribute__((__section__(".initcall" #id ".init"))) = fn;
 
 /*

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #21 lore raw

PATCH 17/21 lto, workaround: Disable LTO for BPF

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

Disable LTO for the BPF interpreter. This works around a gcc bug in the LTO partitioner that partitions the jumptable used the BPF interpreter into a different LTO unit. This in term causes assembler
errors because the jump table contains references to the
code labels in the original file.

gcc problem tracked in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=50676

Signed-off-by Andi Kleen <[email protected]>

---
 kernel/bpf/Makefile | 5 +++++
 1 file changed, 5 insertions(+)

kernel/bpf/Makefile +5 -0

index e691da0b3bab..409d4b6762ee 100644
@@ -16,3 +16,8 @@ ifeq ($(CONFIG_PERF_EVENTS),y)
 obj-$(CONFIG_BPF_SYSCALL) += stackmap.o
 endif
 obj-$(CONFIG_CGROUP_BPF) += cgroup.o
+
+# various version of gcc have a LTO bug where the &&labels used in the
+# BPF interpreter can cause linker errors when spread incorrectly over
+# partitions. Disable LTO for BPF for now
+CFLAGS_core.o = $(DISABLE_LTO)

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #22 lore raw

PATCH 18/21 lto, crypto: Disable LTO for camelia glue

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

The camelia assembler glue functions don't like LTO and cause missing symbols. Just disable LTO for them. I tried to add some visibles, but it's good enough and this works fine.

Signed-off-by Andi Kleen <[email protected]>

---
 arch/x86/crypto/Makefile | 1 +
 crypto/Makefile          | 1 +
 2 files changed, 2 insertions(+)

arch/x86/crypto/Makefile +1 -0

index 5f07333bb224..9dd30dd062f1 100644
@@ -64,6 +64,7 @@ serpent-sse2-i586-y := serpent-sse2-i586-asm_32.o serpent_sse2_glue.o
 
 aes-x86_64-y := aes-x86_64-asm_64.o aes_glue.o
 des3_ede-x86_64-y := des3_ede-asm_64.o des3_ede_glue.o
+CFLAGS_camellia_glue.o += $(DISABLE_LTO)
 camellia-x86_64-y := camellia-x86_64-asm_64.o camellia_glue.o
 blowfish-x86_64-y := blowfish-x86_64-asm_64.o blowfish_glue.o
 twofish-x86_64-y := twofish-x86_64-asm_64.o twofish_glue.o

crypto/Makefile +1 -0

index d674884b2d51..86ae10bb468b 100644
@@ -101,6 +101,7 @@ CFLAGS_serpent_generic.o := $(call cc-option,-fsched-pressure)  # https://gcc.gn
 obj-$(CONFIG_CRYPTO_AES) += aes_generic.o
 obj-$(CONFIG_CRYPTO_AES_TI) += aes_ti.o
 obj-$(CONFIG_CRYPTO_CAMELLIA) += camellia_generic.o
+CFLAGS_cast_common.o += $(DISABLE_LTO)
 obj-$(CONFIG_CRYPTO_CAST_COMMON) += cast_common.o
 obj-$(CONFIG_CRYPTO_CAST5) += cast5_generic.o
 obj-$(CONFIG_CRYPTO_CAST6) += cast6_generic.o

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #23 lore raw

PATCH 19/21 lto, x86: Disable LTO for realmode / vDSO / head64

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

These files all don't like being compiled with LTO. Disable it for them.

Signed-off-by Andi Kleen <[email protected]>

---
 arch/x86/entry/vdso/Makefile | 3 +--
 arch/x86/kernel/Makefile     | 2 ++
 arch/x86/realmode/Makefile   | 1 +
 3 files changed, 4 insertions(+), 2 deletions(-)

arch/x86/entry/vdso/Makefile +1 -2

index 1943aebadede..76caa1159ce1 100644
@@ -3,7 +3,6 @@
 # Building vDSO images for x86.
 #
 
-KBUILD_CFLAGS += $(DISABLE_LTO)
 KASAN_SANITIZE			:= n
 UBSAN_SANITIZE			:= n
 OBJECT_FILES_NON_STANDARD	:= y

@@ -74,7 +73,7 @@ $(obj)/vdso-image-%.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(obj)/vdso2c FORCE
 CFL := $(PROFILING) -mcmodel=small -fPIC -O2 -fasynchronous-unwind-tables -m64 \
        $(filter -g%,$(KBUILD_CFLAGS)) $(call cc-option, -fno-stack-protector) \
        -fno-omit-frame-pointer -foptimize-sibling-calls \
-       -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO
+       -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO $(DISABLE_LTO)
 
 $(vobjs): KBUILD_CFLAGS := $(filter-out $(GCC_PLUGINS_CFLAGS),$(KBUILD_CFLAGS)) $(CFL)

arch/x86/kernel/Makefile +2 -0

index 81bb565f4497..18106858d4c3 100644
@@ -22,6 +22,8 @@ CFLAGS_REMOVE_early_printk.o = -pg
 CFLAGS_REMOVE_head64.o = -pg
 endif
 
+CFLAGS_head64.o += $(DISABLE_LTO)
+
 KASAN_SANITIZE_head$(BITS).o				:= n
 KASAN_SANITIZE_dumpstack.o				:= n
 KASAN_SANITIZE_dumpstack_$(BITS).o			:= n

arch/x86/realmode/Makefile +1 -0

index 682c895753d9..719f423da3a0 100644
@@ -6,6 +6,7 @@
 # for more details.
 #
 #
+KBUILD_CFLAGS += $(DISABLE_LTO)
 KASAN_SANITIZE			:= n
 OBJECT_FILES_NON_STANDARD	:= y

Signature · 1 line

2.13.6

Andi Kleen

author

2017-11-27 21:34 UTC #24 lore raw

PATCH 20/21 Kbuild, lto: Add Link Time Optimization support

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

With LTO gcc will do whole program optimizations for the whole kernel and each module. This increases compile time, and makes incremential builds slower, but can generate faster and smaller code and allows the compiler to do some global checking.

gcc can complain now about type mismatches for symbols between different files.

The main advantage is that it allows cross file inlining, which enables a range of new optimizations. It also allows the compiler to throw away unused functions, which typically shrinks the kernel somewhat.

It also enables a range of advanced and future optimizations in the compiler.

Unlike earlier, this version doesn't require special binutils, but relies on THIN_ARCHIVES instead.

This adds the basic Kbuild plumbing for LTO:

- In Kbuild add a new scripts/Makefile.lto that checks the tool chain and when the tests pass sets the LTO options We enable it only for gcc 5.0+ and reasonable new binutils

- Add a new LDFINAL variable that controls the final link for vmlinux or module. In this case we call gcc-ld instead of ld, to run the LTO step.

- Kconfigs:
Since LTO with allyesconfig needs more than 4G of memory (~8G) and has the potential to makes people's system swap to death. Smaller configs typically work with 4G.
I used a nested config that ensures that a simple allyesconfig disables LTO. It has to be explicitely enabled.

- This version runs modpost on the LTO object files. This currently breaks MODVERSIONS and causes some warnings and requires disabling the module resolution checks. MODVERSIONS is excluded with LTO here. Solution would be to reorganize the linking step to do a LDFINAL -r link on all modules before running modpost

- Since this kernel version links the final kernel two-three times for kallsyms all optimization steps are done multiple times.

Thanks to HJ Lu, Joe Mario, Honza Hubicka, Richard Guenther, Don Zickus, Changlong Xie, Gleb Schukin who helped with this project (and probably some more who I forgot, sorry)

Signed-off-by Andi Kleen <[email protected]>

---
 Documentation/lto-build  | 76 ++++++++++++++++++++++++++++++++++++++
 Makefile                 |  6 ++-
 init/Kconfig             | 68 ++++++++++++++++++++++++++++++++++
 scripts/Makefile.lto     | 95 ++++++++++++++++++++++++++++++++++++++++++++++++
 scripts/Makefile.modpost |  7 ++--
 scripts/gcc-ld           |  4 +-
 scripts/link-vmlinux.sh  |  6 +--
 7 files changed, 252 insertions(+), 10 deletions(-)
 create mode 100644 Documentation/lto-build
 create mode 100644 scripts/Makefile.lto

Documentation/lto-build new file +76 -0

new file mode 100644
index 000000000000..f33f008b23db
@@ -0,0 +1,76 @@
+Link time optimization (LTO) for the Linux kernel
+
+This is an experimental feature.
+
+Link Time Optimization allows the compiler to optimize the complete program
+instead of just each file.
+
+The compiler can inline functions between files and do various other global
+optimizations, like specializing functions for common parameters,
+determing when global variables are clobbered, making functions pure/const,
+propagating constants globally, removing unneeded data and others.
+
+It will also drop unused functions which can make the kernel
+image smaller in some circumstances, in particular for small kernel
+configurations.
+
+For small monolithic kernels it can throw away unused code very effectively
+(especially when modules are disabled) and usually shrinks
+the code size.
+
+Build time and memory consumption at build time will increase, depending
+on the size of the largest binary. Modular kernels are less affected.
+With LTO incremental builds are less incremental, as always the whole
+binary needs to be re-optimized (but not re-parsed)
+
+Oops can be somewhat more difficult to read, due to the more aggressive
+inlining (it helps to use scripts/faddr2line)
+
+Normal "reasonable" builds work with less than 4GB of RAM, but very large
+configurations like allyesconfig typically need more memory. The actual
+memory needed depends on the available memory (gcc sizes its garbage
+collector pools based on that or on the ulimit -m limits) and
+the compiler version.
+
+Configuration:
+- Enable CONFIG_LTO_MENU and then disable CONFIG_LTO_DISABLE.
+This is mainly to not have allyesconfig default to LTO.
+
+Requirements:
+- Enough memory: 4GB for a standard build, more for allyesconfig
+The peak memory usage happens single threaded (when lto-wpa merges types),
+so dialing back -j options will not help much.
+
+A 32bit compiler is unlikely to work due to the memory requirements.
+You can however build a kernel targeted at 32bit on a 64bit host.
+
+FAQs:
+
+Q: I get a section type attribute conflict
+A: Usually because of someone doing
+const __initdata (should be const __initconst) or const __read_mostly
+(should be just const). Check both symbols reported by gcc.
+
+Q: What's up with .XXXXX numeric post fixes
+A: This is due LTO turning (near) all symbols to static
+Use gcc 4.9, it avoids them in most cases. They are also filtered out
+in kallsyms. There are still some .lto_priv left.
+
+References:
+
+Presentation on Kernel LTO
+(note, performance numbers/details outdated.  In particular gcc 4.9 fixed
+most of the build time problems):
+http://halobates.de/kernel-lto.pdf
+
+Generic gcc LTO:
+http://www.ucw.cz/~hubicka/slides/labs2013.pdf
+http://www.hipeac.net/system/files/barcelona.pdf
+
+Somewhat outdated too:
+http://gcc.gnu.org/projects/lto/lto.pdf
+http://gcc.gnu.org/projects/lto/whopr.pdf
+
+Happy Link-Time-Optimizing!
+
+Andi Kleen

Makefile +4 -2

index f761bf475ba5..685a638bc3cd 100644
@@ -370,6 +370,7 @@ HOST_LOADLIBES := $(HOST_LFS_LIBS)
 # Make variables (CC, etc...)
 AS		= $(CROSS_COMPILE)as
 LD		= $(CROSS_COMPILE)ld
+LDFINAL		= $(LD)
 CC		= $(CROSS_COMPILE)gcc
 CPP		= $(CC) -E
 AR		= $(CROSS_COMPILE)ar

@@ -427,7 +428,7 @@ KBUILD_LDFLAGS_MODULE := -T $(srctree)/scripts/module-common.lds
 GCC_PLUGINS_CFLAGS :=
 
 export ARCH SRCARCH CONFIG_SHELL HOSTCC HOSTCFLAGS CROSS_COMPILE AS LD CC
-export CPP AR NM STRIP OBJCOPY OBJDUMP HOSTLDFLAGS HOST_LOADLIBES
+export CPP AR NM STRIP OBJCOPY OBJDUMP HOSTLDFLAGS HOST_LOADLIBES LDFINAL
 export MAKE AWK GENKSYMS INSTALLKERNEL PERL PYTHON UTS_MACHINE
 export HOSTCXX HOSTCXXFLAGS LDFLAGS_MODULE CHECK CHECKFLAGS

@@ -813,6 +814,7 @@ KBUILD_ARFLAGS := $(call ar-option,D)
 include scripts/Makefile.kasan
 include scripts/Makefile.extrawarn
 include scripts/Makefile.ubsan
+include scripts/Makefile.lto
 
 # Add any arch overrides and user supplied CPPFLAGS, AFLAGS and CFLAGS as the
 # last assignments

@@ -986,7 +988,7 @@ ARCH_POSTLINK := $(wildcard $(srctree)/arch/$(SRCARCH)/Makefile.postlink)
 
 # Final link of vmlinux with optional arch pass after final link
 cmd_link-vmlinux =                                                 \
-	$(CONFIG_SHELL) $< $(LD) $(LDFLAGS) $(LDFLAGS_vmlinux) ;    \
+	$(CONFIG_SHELL) $< $(LDFINAL) $(LDFLAGS) $(LDFLAGS_vmlinux) ;    \
 	$(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) $@, true)
 
 vmlinux: scripts/link-vmlinux.sh vmlinux_prereq $(vmlinux-deps) FORCE

init/Kconfig +68 -0

80 lines not on this page · raw message

scripts/Makefile.lto new file +95 -0

95 lines not on this page · raw message

scripts/Makefile.modpost +4 -3

index df4174405feb..d1d3b2cfc9ce 100644
@@ -79,7 +79,8 @@ modpost = scripts/mod/modpost                    \
  $(if $(KBUILD_EXTMOD),-o $(modulesymfile))      \
  $(if $(CONFIG_DEBUG_SECTION_MISMATCH),,-S)      \
  $(if $(CONFIG_SECTION_MISMATCH_WARN_ONLY),,-E)  \
- $(if $(KBUILD_EXTMOD)$(KBUILD_MODPOST_WARN),-w)
+ $(if $(KBUILD_EXTMOD)$(KBUILD_MODPOST_WARN),-w) \
+ $(if $(CONFIG_LTO),-w)
 
 MODPOST_OPT=$(subst -i,-n,$(filter -i,$(MAKEFLAGS)))

@@ -118,9 +119,9 @@ targets += $(modules:.ko=.mod.o)
 ARCH_POSTLINK := $(wildcard $(srctree)/arch/$(SRCARCH)/Makefile.postlink)
 
 # Step 6), final link of the modules with optional arch pass after final link
-quiet_cmd_ld_ko_o = LD [M]  $@
+quiet_cmd_ld_ko_o = LDFINAL [M]  $@
       cmd_ld_ko_o =                                                     \
-	$(LD) -r $(LDFLAGS)                                             \
+	$(LDFINAL) -r $(LDFLAGS)                                             \
                  $(KBUILD_LDFLAGS_MODULE) $(LDFLAGS_MODULE)             \
                  -o $@ $(filter-out FORCE,$^) ;                         \
 	$(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) $@, true)

scripts/gcc-ld +2 -2

index 997b818c3962..d95dd0be38e7 100755
@@ -8,7 +8,7 @@ ARGS="-nostdlib"
 
 while [ "$1" != "" ] ; do
 	case "$1" in
-	-save-temps|-m32|-m64) N="$1" ;;
+	-save-temps*|-m32|-m64) N="$1" ;;
 	-r) N="$1" ;;
 	-[Wg]*) N="$1" ;;
 	-[olv]|-[Ofd]*|-nostdlib) N="$1" ;;

@@ -19,7 +19,7 @@ while [ "$1" != "" ] ; do
 -rpath-link|--sort-section|--section-start|-Tbss|-Tdata|-Ttext|\
 --version-script|--dynamic-list|--version-exports-symbol|--wrap|-m)
 		A="$1" ; shift ; N="-Wl,$A,$1" ;;
-	-[m]*) N="$1" ;;
+	-[mp]*) N="$1" ;;
 	-*) N="-Wl,$1" ;;
 	*)  N="$1" ;;
 	esac

scripts/link-vmlinux.sh +3 -3

index c0d129d7f430..964b2ee855dd 100755
@@ -84,7 +84,7 @@ modpost_link()
 			${KBUILD_VMLINUX_LIBS}				\
 			--end-group"
 	fi
-	${LD} ${LDFLAGS} -r -o ${1} ${objects}
+	${LDFINAL} ${LDFLAGS} -r -o ${1} ${objects}
 }
 
 # Link of vmlinux

@@ -113,7 +113,7 @@ vmlinux_link()
 				${1}"
 		fi
 
-		${LD} ${LDFLAGS} ${LDFLAGS_vmlinux} -o ${2}		\
+		${LDFINAL} ${LDFLAGS} ${LDFLAGS_vmlinux} -o ${2}		\
 			-T ${lds} ${objects}
 	else
 		if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then

@@ -309,7 +309,7 @@ if [ -n "${CONFIG_KALLSYMS}" ]; then
 	fi
 fi
 
-info LD vmlinux
+info LDFINAL vmlinux
 vmlinux_link "${kallsymso}" vmlinux
 
 if [ -n "${CONFIG_BUILDTIME_EXTABLE_SORT}" ]; then

Signature · 1 line

2.13.6

Arnd Bergmann

2018-01-27 00:15 UTC #25 lore raw

Re: [PATCH 20/21] Kbuild, lto: Add Link Time Optimization support

Quoted 4 lines · Andi Kleen

From: Andi Kleen <[email protected]>
- Add a new LDFINAL variable that controls the final link for vmlinux or module. In this case we call gcc-ld instead of ld, to run the LTO step.

When I tried this out on allmodconfig (following the lwn article), I ran into a number of warnings:

WARNING: modpost: missing MODULE_LICENSE() in drivers/isdn/hardware/eicon/diva_mnt.o
see include/linux/module.h for more information
WARNING: modpost: missing MODULE_LICENSE() in drivers/isdn/hysdn/hysdn.o see include/linux/module.h for more information
WARNING: modpost: missing MODULE_LICENSE() in drivers/vhost/vhost_scsi.o see include/linux/module.h for more information
WARNING: modpost: missing MODULE_LICENSE() in drivers/scsi/pcmcia/qlogic_cs.o see include/linux/module.h for more information
WARNING: modpost: missing MODULE_LICENSE() in drivers/scsi/pcmcia/fdomain_cs.o
...

These are apparently all compound modules that do have a valid license, and modinfo
shows it correctly:

modinfo build/tmp/drivers/isdn/hardware/eicon/diva_mnt.ko filename:
/home/arnd/arm-soc/build/tmp/drivers/isdn/hardware/eicon/diva_mnt.ko

license:        GPL
author:         Cytronics & Melware, Eicon Networks
description:    Maint driver for Eicon DIVA Server cards

...

but something in the LTO build conuses modpost to the point that it doesn't see it.

I also get 131 errors in x86 allmodconfig along the lines of

/git/arm-soc/include/linux/slab.h:298:0: error: type of 'kmalloc_caches' does not match original declaration [-Werror=lto-type-mismatch]
extern struct kmem_cache *kmalloc_caches[KMALLOC_SHIFT_HIGH + 1]; /git/arm-soc/mm/slab_common.c:958:1: note: 'kmalloc_caches' was
previously declared here
EXPORT_SYMBOL(kmalloc_caches);

/git/arm-soc/include/linux/seq_file.h:109:5: error: type of 'seq_open' does not match original declaration [-Werror=lto-type-mismatc h]

 int seq_open(struct file *, const struct seq_operations *);
     ^

/git/arm-soc/fs/seq_file.c:48:0: note: 'seq_open' was previously declared here
int seq_open(struct file *file, const struct seq_operations *op)

I found a fix for some of these issues attached to
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83954
but am still doing more randconfig build testing.

Arnd

Andi Kleen

2018-01-27 00:56 UTC #26 lore raw

Re: [PATCH 20/21] Kbuild, lto: Add Link Time Optimization support

Quoted 8 lines · Arnd Bergmann

Quoted 4 lines · Andi Kleen
From: Andi Kleen <[email protected]>
- Add a new LDFINAL variable that controls the final link for vmlinux or module. In this case we call gcc-ld instead of ld, to run the LTO step.
When I tried this out on allmodconfig (following the lwn article), I ran into a number of warnings:

Thanks for testing. Yes it's a known issue: during one module build when modpost looks at the file it is still in LTO format, and modpost doesn't understand the LTO symbol table. I had a patch to teach it to it at some point, but it got lost somewhere. The LLVM LTO patchkit has a different solution that actually fixes the sequence to run modpost only after a LTO final link, but I haven't gotten around to port that one.

It seems to work already for the single file modules.

-Andi

Arnd Bergmann

2018-01-27 14:26 UTC #27 lore raw

Re: [PATCH 20/21] Kbuild, lto: Add Link Time Optimization support

Quoted 18 lines · Andi Kleen

Quoted 8 lines · Arnd Bergmann
Quoted 4 lines · Andi Kleen
From: Andi Kleen <[email protected]>
- Add a new LDFINAL variable that controls the final link for vmlinux or module. In this case we call gcc-ld instead of ld, to run the LTO step.
When I tried this out on allmodconfig (following the lwn article), I ran into a number of warnings:
Thanks for testing. Yes it's a known issue: during one module build when modpost looks at the file it is still in LTO format, and modpost doesn't understand the LTO symbol table. I had a patch to teach it to it at some point, but it got lost somewhere. The LLVM LTO patchkit has a different solution that actually fixes the sequence to run modpost only after a LTO final link, but I haven't gotten around to port that one.

Quoted 1 line

It seems to work already for the single file modules.

Ok, I see. I've turned off that warning now for testing.

What about this one:

kallsyms failure: relative symbol value 0xffffffff81000000 out of range in relative mode

I seem to get that all the time here, and have also disabled it for now, but it sounds important (and breaks the build).

Arnd

Andi Kleen

2018-01-28 18:33 UTC #28 lore raw

Re: [PATCH 20/21] Kbuild, lto: Add Link Time Optimization support

Quoted 6 lines

kallsyms failure: relative symbol value 0xffffffff81000000 out of range in relative mode
I seem to get that all the time here, and have also disabled it for now, but it sounds important (and breaks the build).

Need to take a look at it.

I had some patches that completely revamped kallsyms as part of the single pass linking, but I dropped them again because they still had too many issues.

-Andi

Andi Kleen

author

2017-11-27 21:34 UTC #29 lore raw

PATCH 21/21 x86: Enable Link Time Optimization

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

LTO is opt-in per architecture because it usually needs some fixes.

LTO needs THIN_ARCHIVES because standard binutils doesn't like mixing assembler and LTO code with ld -r.

Enable LTO and THIN_ARCHIVES for x86

Signed-off-by Andi Kleen <[email protected]>

---
 arch/x86/Kconfig | 2 ++
 1 file changed, 2 insertions(+)

arch/x86/Kconfig +2 -0

index 8eed3f94bfc7..92650726f908 100644
@@ -122,6 +122,8 @@ config X86
 	select HAVE_ARCH_VMAP_STACK		if X86_64
 	select HAVE_ARCH_WITHIN_STACK_FRAMES
 	select HAVE_CC_STACKPROTECTOR
+	select THIN_ARCHIVES			if LTO
+	select ARCH_SUPPORTS_LTO
 	select HAVE_CMPXCHG_DOUBLE
 	select HAVE_CMPXCHG_LOCAL
 	select HAVE_CONTEXT_TRACKING		if X86_64

Signature · 1 line

2.13.6

Sami Tolvanen

2017-11-29 23:09 UTC #30 lore raw

In reply to Andi Kleen

Hi Andi,

Quoted 3 lines · Andi Kleen

This is an updated version of my older LTO patchkit for gcc/x86 This version doesn't need special binutils, but requires gcc 5+. It also is compatible with near all options (except MODVERSIONS)

Would you mind changing the config option prefix to LTO_GCC, for example? It looks like there are a few places where gcc LTO needs to be disabled, but clang LTO works fine, and vice versa.

My earlier clang LTO patches also had a solution for MODVERSIONS, which might work with gcc as well.

Sami