re-add -ffreestanding

I got the following compile error with gcc 4.1.1 when trying to compile kernel 2.6.18-rc4-mm2 for m68k:

<-- snip -->

...
LD .tmp_vmlinux1
arch/m68k/amiga/built-in.o: In function `amiga_get_hardware_list':
config.c:(.text+0x882): undefined reference to strcpy' kernel/built-in.o: In function prof_cpu_mask_read_proc':
profile.c:(.text+0x41ac): undefined reference to strcpy' kernel/built-in.o: In function sysfs_show_available_clocksources':
clocksource.c:(.text+0x1434e): undefined reference to strcpy' kernel/built-in.o: In function sysfs_show_current_clocksources':
clocksource.c:(.text+0x143a4): undefined reference to strcpy' fs/built-in.o: In function lock_get_status':
locks.c:(.text+0x1198a): undefined reference to strcpy' fs/built-in.o:locks.c:(.text+0x119ae): more undefined references to strcpy' follow make[1]: *** [.tmp_vmlinux1] Error 1

<-- snip -->

The Linux kernel is a freestanding environment, not a hosted one providing a full standard C library gcc can consider being present.

The fact that -ffreestanding had already been re-added on three architectures also reflects this fact.

If we want to use all or a subset of the gcc builtins on some or all architectures, we should make this explicitely through #define's.

The justification of commit 6edfba1b33c701108717f4e036320fc39abe1912 "it was only added for x86-64, so dropping it should be safe" has been proved as wrong since it broke at least mips and m68k.

This patch therefore reverts commit 6edfba1b33c701108717f4e036320fc39abe1912 and re-adds -ffreestanding to the CFLAGS.

---

This patch was (except for a rediff due to unrelated changes) already sent on:
- 21 Aug 2006

 Makefile                    |    2 +-
 arch/i386/Makefile          |    3 ---
 arch/mips/Makefile          |    2 --
 arch/um/Makefile-i386       |    4 ----
 include/asm-x86_64/string.h |   18 ++++++++++++++----
 5 files changed, 15 insertions(+), 14 deletions(-)

@@ -39,9 +39,6 @@
 
 cflags-$(CONFIG_REGPARM) += -mregparm=3
 
-# temporary until string.h is fixed
-cflags-y += -ffreestanding
-
 # Disable unit-at-a-time mode on pre-gcc-4.0 compilers, it makes gcc use
 # a lot more stack due to the lack of sharing of stacklots:
 CFLAGS				+= $(shell if [ $(call cc-version) -lt 0400 ] ; then echo $(call cc-option,-fno-unit-at-a-time); fi ;)

@@ -83,8 +83,6 @@
 LDFLAGS_vmlinux			+= -G 0 -static -n -nostdlib
 MODFLAGS			+= -mlong-calls
 
-cflags-y += -ffreestanding
-
 #
 # We explicitly add the endianness specifier if needed, this allows
 # to compile kernels with a toolchain for the other endianness. We

@@ -33,9 +33,5 @@
 # prevent gcc from keeping the stack 16 byte aligned. Taken from i386.
 cflags-y += $(call cc-option,-mpreferred-stack-boundary=2)
 
-# Prevent sprintf in nfsd from being converted to strcpy and resulting in
-# an unresolved reference.
-cflags-y += -ffreestanding
-
 CFLAGS += $(cflags-y)
 USER_CFLAGS += $(cflags-y)

@@ -41,15 +41,26 @@
 
 
 #define __HAVE_ARCH_MEMSET
-void *memset(void *s, int c, size_t n);
+#define memset __builtin_memset
 
 #define __HAVE_ARCH_MEMMOVE
 void * memmove(void * dest,const void *src,size_t count);
 
+/* Use C out of line version for memcmp */ 
+#define memcmp __builtin_memcmp
 int memcmp(const void * cs,const void * ct,size_t count);
+
+/* out of line string functions use always C versions */ 
+#define strlen __builtin_strlen
 size_t strlen(const char * s);
-char *strcpy(char * dest,const char *src);
-char *strcat(char * dest, const char * src);
+
+#define strcpy __builtin_strcpy
+char * strcpy(char * dest,const char *src);
+
+#define strcat __builtin_strcat
+char * strcat(char * dest, const char * src);
+
+#define strcmp __builtin_strcmp
 int strcmp(const char * cs,const char * ct);
 
 #endif /* __KERNEL__ */

@@ -308,7 +308,7 @@
 CPPFLAGS        := -D__KERNEL__ $(LINUXINCLUDE)
 
 CFLAGS          := -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
-                   -fno-strict-aliasing -fno-common
+                   -fno-strict-aliasing -fno-common -ffreestanding
 AFLAGS          := -D__ASSEMBLY__
 
 # Read KERNELRELEASE from include/config/kernel.release (if it exists)

If anything then -ffreestanding needs to be added to arch/m68k/Makefile (assuming it doesn't compile at all right now)

-Andi

Looking at the effect of -ffreestanding on ARM, it appears that on one hand, the overall image size is reduced by 0.016% but we end up with worse code - eg, strlen() of the same string in the same function evaluated multiple times vs once without -ffreestanding.

The difference probably comes down to the lack of __attribute__((pure)) on our string functions in linux/string.h.

If we are going to go for -ffreestanding, we need to fix linux/string.h in that respect _first_.

Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

We are talking about reverting the patch that removed -ffreestanding, and that broke at least two architectures although it wrongly claimed it would have been a safe patch.

I agree that there is room for improvement at our string functions, but small optimizations like the ones you are mentioning are not much when talking about reverting a patch that is both theoretically wrong and has proven practically to cause problems.

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Hi,

Your patch is nevertheless the wrong fix for one of these archs.

bye, Roman

Hi Roman,

it's correct, since with -ffreestanding gcc no longer has the right to assume it had a full libc available.

Yes, -ffreestanding disables gcc builtins, but the correct fix for this issue is not to remove -ffreestanding, but to check where we want to use gcc builtins and enable them explicitely.

This takes more time than the quick'n'dirty patch that removed -ffreestanding, but it does it right (and we might perhaps get rid of things like include/asm-i386/string.h).

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Hi,

BS, even without it gcc can't make such assumption.
There is not a single optimization, which would be invalid in a kernel environment and would be "fixed" by this option, so please stop this nonsense.

bye, Roman

You are wrong.

Section 5.1.2.2.2 of ISO/IEC 9899:1999 says:
In a hosted environment, a program may use all functions, macros, type definitions, and objects described in the library clause (clause 7).

Since a hosted environment means gcc+libc, it's therefore clear that gcc can assume the presence of a full libc if gcc isn't told that it's used as a freestanding environment.

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Hi,

Define "full libc".
Explain what exactly -ffreestanding fixes, which is not valid for the kernel.

byem Roman

Everything described in clause 7 of ISO/IEC 9899:1999.

It's simply correct since the kernel doesn't provide everything described in clause 7 of ISO/IEC 9899:1999.

And it fixes compile errors caused by the fact that gcc is otherwise allowed to replace calls to any standard C function with semantically equivalent calls to other standard C functions - in a hosted environment the latter are guaranteed to be present.

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Hi,

Its behaviour is also defined by the environment, so what gcc can assume is rather limited and you have not shown a single example, that any such assumption would be invalid for the kernel.

The kernel uses standard C, so your point is?
You already got two NACKs from arch maintainers, why the hell are you still pushing this patch? The builtin functions are useful and you want to force arch maintainers to have to enable every single one manually and to maintain a list of these functions over multiple versions of gcc?

bye, Roman

ISO/IEC 9899:1999 clause 7 defines the libc part of a hosted environment.

A standard C freestanding environment or a standard C hosted environment?

The same people who justified removing -ffreestanding with the "it was only added for x86-64, so dropping it should be safe" that has proven wrong now put their arch maintainers hats on for NACKing reverting this patch...

It could be done per architecture or globally for some functions.

And it doesn't sound like a bad idea to check the current code and think of what it does and what it should do - many architecture specific things (like much of include/asm-i386/string.h) seem to be more historically than architecture specific.

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Hi,

Which is a problem for the kernel exactly how?
BTW the standard specifies the minimum requirements for a libc, so talking about "full libc" is ambiguous at best.

As far as gcc is concerned it's a hosted environment, where we provide only what we actually use, but anything we do provide is compliant.

And you keep ignoring there might be better solutions...

We're happy to hear about it, once you've done this.

bye, Roman

Wrong. Your patch unconditionally adds it for _ALL_ architectures. Below is the extract which you posted which supports this fact.

For the elimination of any doubt, I do _NOT_ want this patch merged as is. Take that as the _third_ architecture maintainer who has NACK'd your patch (as you should've taken my first objection as that and apparantly didn't.)

... and maybe you should copy linux-arch with architecture-wide changes so that all architecture maintainers are aware of what you're trying to do?

@@ -308,7 +308,7 @@
 CPPFLAGS        := -D__KERNEL__ $(LINUXINCLUDE)
 
 CFLAGS          := -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
-                   -fno-strict-aliasing -fno-common
+                   -fno-strict-aliasing -fno-common -ffreestanding
 AFLAGS          := -D__ASSEMBLY__
 
 # Read KERNELRELEASE from include/config/kernel.release (if it exists)

Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Andis patch "x86_64: Don't define string functions to builtin" (sic) that removed -ffreestanding for all architectures except i386 and that broke at least two architectures was neither sent to linux-arch nor to linux-kernel.

And I'm getting bashed for sendind a patch to revert it "only" to linux-kernel...

cu
Adrian
(who has just deleted all his cross compilers for getting rid of all
these troubles)

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Hi,

No, you get "bashed" for pushing a patch that only hides problems instead of fixing them and that only creates new problems.

Your help is certainly appreciated, but you have to work a bit more with the people who actually use these platforms.

bye, Roman

You're not getting bashed - you're getting directed to the correct place to discuss this issue. There are some architecture maintainers who might have some input who aren't subscribed to linux-kernel.

Incidentally, the addition of -ffreestanding was only sent to linux-kernel in the first place back in 2004, so this actually gives those architecture maintainers a chance to actually comment on it. At the time, no one commented on it - maybe it got missed? I certainly did.

http://marc.theaimsgroup.com/?l=linux-kernel&m=110194462121275&w=2

As far as your argument that the kernel is not a hosted environment, that's debatable (as you're finding out).

If we decide that we want the compiler to treat our source as if it were a hosted environment, and we provide sufficient implementation of a conforming nature of a hosted environment then that is our perogative to do so. That is a decision that we are entirely free to make. By doing so, we take on the responsibility to provide whatever is required for a hosted environment as opposed to the more limited functionality of a freestanding environment.

We _have_ taken on that responsibility. We have even taken on the responsibility to provide the compiler's internal library functions in some cases.

The question really comes down to two points:

1. do the C standard defined library services we use in the kernel have the behaviour required by the C standard
2. do we provide everything that the compiler would want to use in the environment mode we have chosen.

The answer to both is generally yes, except when something is omitted for a reason. (eg, float support.)

And two final points:

- according to what I read in the gcc manual, free standing environments are required to provide float support. We don't, so it could be possible to argue that we provide neither a freestanding nor a hosted environment, and, therefore, we shouldn't be using gcc at all.

- architecture maintainers should be left to decide what implementation option they require the kernel to be built in, since it's the architecture support code which provides most of the runtime environment.

If that's how you feel and what you've done, you seem to have disqualified yourself from handling generic kernel changes.

I notice that you didn't respond to anything else in my message, so I can only assume that you've accepted my reasoning and are no longer going to push this patch. So treat the above as the finer detailed reasoning.

I'm not anti--ffreestanding per se, but if we _are_ going to switch (back) to that compiler mode, I want to ensure that we aren't walking backwards from the point I _thought_ we were at.

Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Ick, can anybody be persuaded to post actual effective code changes? IMHO the kernel should use "-ffreestanding", which is defined as follows in the GCC manual (which, I admit, has been wrong before, but it's a start at least):

"Assert that the compilation takes place in a freestanding environment. Implies "-fno-builtin". A freestanding environment is one in which the standard library may not exist, and program startup may not necessarily be at "main" (which does not necessarily have a return type of "int"). The most obvious example is an OS kernel."

According to the above, there are two effects for turning on "-ffreestanding":
1) "main" doesn't have a special argument list and return value.
2) "-fno-builtin"

The docs for "-fno-builtin" say:

"Don't recognize built-in functions that do not begin with __builtin_ as prefix.

GCC normally generates special code to handle certain built-in functions more efficiently; for instance, calls to "alloca" may become single instructions that adjust the stack directly, and calls to "memcpy" may become inline copy loops. The resulting code is often both smaller and faster, but since the function calls no longer appear as such, you cannot set a breakpoint on those calls, nor can you change the behavior of the functions by linking with a different library. In addition, when a function is recognized as a built-in function, GCC may use information about that function to warn about problems with calls to that function, or to generate more efficient code, even if the resulting code still contains calls to that function. For example, warnings are given with "-Wformat" for bad calls to "printf", when "printf" is a builtin, and "strlen" is known not to modify global memory.

[...]

If you wish to enable built-in functions selectively when using -fno-builtin or -ffreestanding, you may define macros such as:
#define abs(n) __builtin_abs((n))
#define strcpy(d, s) __builtin_strcpy((d),(s))"

SO, the total effects of "-ffreestanding" are:
1) "main" is not a special function
2) "foo" does not automatically mean "__builtin_foo"

Why are you arguing so much over this change? As far as I can tell, turning on "-ffreestanding" is a no-brainer; "main" isn't special in the kernel and allowing arch maintainers to selectively turn on or off "__builtin_*" effects for kernel functions in per-arch headers is a good thing.

If your arch absolutely *must* have "strlen" act like "__builtin_strlen" for some reason, just do this:
#define strlen(x) __builtin_strlen(x)

But please note that __builtin_ functions are defined by the GCC people to be able to decay to _any_ function call. For example, the GCC people replied to Adrian Bunk's email this way:
http://lists.debian.org/debian-gcc/2005/03/msg00101.html

"You explicitly requested the __builtin_sprintf implementation, which is allowed to decay to strcpy, and overrode the -ffreestanding in this case."

So the meaning of __builtin_sprintf is NOT 'have GCC emit sprintf code', and it is NOT 'emit assembly where easy and sprintf where otherwise', it is 'emit calls to any function(s) in the standard C library which implement the requested functionality.'

So it may be OK for sprintf(buf,"%s",str); to decay to strcpy(buf, str) in the kernel, but if it's not the ONLY ways to turn it off are -fno-builtin-sprintf, -fno-builtin, and -ffreestanding. Explicitly disabling these optimizations is virtually guaranteed that we'll miss one, we should turn them all off and selectively enable the ones that make sense.

Freestanding environments need to provide float support _if_they_need_it_. We don't need it, don't use it, and so don't have to provide it.

Cheers,
Kyle Moffett

I've already specified the changes on ARM, and suggested a fix for them - but that got poo-poo'd. I said:

So the effective code changes you ask for are: "multiple calls to standard library functions that would not otherwise be made without -ffreestanding".

Hence, for -ffreestanding to be acceptable to me, we need to fix linux/string.h _first_. That's really all I'm asking for but apparantly that's too much to ask for.

It's not realistic to post the actual code changes because virtually every line is different - due to differences in the register allocation caused by the variations in code generation. Hence, to compare it properly it's a painstaking line by line read of each to understand what's going on and manual compare.

Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Hi,

No, that would be a complete PITA and so far no arch maintainer wants to do this.
gcc cannot do arbitrary transformations, the result has to be the same according to the standard and since the standard functions we provide are standards compliant (modulo bugs, minus fp support) there is no problem. It's possible that gcc generates a call to a function we don't provide, but the resulting link failure is hard to miss.

bye, Roman