Skip to contentSkip to conversation

m32r: set CHECKFLAGS properly

PATCH In review Al Viro 5 participants 16 messages Sep 26, 2005 – Sep 29, 2005

lore
Share & tools
lore

Al Viro

author
#0 lore raw

PATCH 1/1 m32r: set CHECKFLAGS properly

We do _not_ need "sparse" in sparse arguments ;-)
What we do need is __BIG_ENDIAN__; right now unconditional, when m32r starts using CPU_LITTLE_ENDIAN, we'll need to adjust.

----

arch/m32r/Makefile +1 -1
diff -urN RC14-rc2-git5-rpc-iomem/arch/m32r/Makefile RC14-rc2-git5-sparse-m32r/arch/m32r/Makefile
@@ -24,7 +24,7 @@
 CFLAGS += $(cflags-y)
 AFLAGS += $(aflags-y)
 
-CHECKFLAGS	:= $(CHECK) -D__m32r__
+CHECKFLAGS	+= -D__m32r__ -D__BIG_ENDIAN__=1
 
 head-y	:= arch/m32r/kernel/head.o arch/m32r/kernel/init_task.o
 
#1 lore raw

Please don't specify __BIG_ENDIAN__. ;-)
We have supported both big- and little-endian for the m32r kernel. I hope it will be kept unconditional.

Now, the endianness is to be determined by a (cross)compiler:
- For the big-endian, a compiler (m32r-linux-gcc or m32r-linux-gnu-gcc) provides a predefined macro __BIG_ENDIAN__.
- For little-endian, a compiler (m32rle-linux-gcc or m32rle-linux-gnu-gcc) provides a predefined macro __LITTLE_ENDIAN__.

Here is a modified patch.

Thank you.

From: Al Viro <[email protected]>
Date: Mon, 26 Sep 2005 06:18:04 +0100

Quoted 2 lines

What we do need is __BIG_ENDIAN__; right now unconditional, when m32r starts using CPU_LITTLE_ENDIAN, we'll need to adjust.

----

 arch/m32r/Makefile |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/m32r/Makefile +1 -1
Index: linux-2.6.14-rc2/arch/m32r/Makefile
===================================================================
@@ -24,7 +24,7 @@ aflags-$(CONFIG_ISA_M32R)	+= -DNO_FPU -W
 CFLAGS += $(cflags-y)
 AFLAGS += $(aflags-y)
 
-CHECKFLAGS	:= $(CHECK) -D__m32r__
+CHECKFLAGS	+= -D__m32r__
 
 head-y	:= arch/m32r/kernel/head.o arch/m32r/kernel/init_task.o
 

--
Hirokazu Takata <[email protected]>
Linux/M32R Project: http://www.linux-m32r.org/

#11 lore raw
In reply to Al Viro

Please don't specify __BIG_ENDIAN__. ;-)
We have supported both big- and little-endian for the m32r kernel. I hope it will be kept unconditional.

Now, the endianness is to be determined by a (cross)compiler:
- For the big-endian, a compiler (m32r-linux-gcc or m32r-linux-gnu-gcc) provides a predefined macro __BIG_ENDIAN__.
- For little-endian, a compiler (m32rle-linux-gcc or m32rle-linux-gnu-gcc) provides a predefined macro __LITTLE_ENDIAN__.

Here is a modified patch.

Thank you.

From: Al Viro <[email protected]>
Date: Mon, 26 Sep 2005 06:18:04 +0100

Quoted 2 lines

What we do need is __BIG_ENDIAN__; right now unconditional, when m32r starts using CPU_LITTLE_ENDIAN, we'll need to adjust.

----

 arch/m32r/Makefile |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/m32r/Makefile +1 -1
Index: linux-2.6.14-rc2/arch/m32r/Makefile
===================================================================
@@ -24,7 +24,7 @@ aflags-$(CONFIG_ISA_M32R)	+= -DNO_FPU -W
 CFLAGS += $(cflags-y)
 AFLAGS += $(aflags-y)
 
-CHECKFLAGS	:= $(CHECK) -D__m32r__
+CHECKFLAGS	+= -D__m32r__
 
 head-y	:= arch/m32r/kernel/head.o arch/m32r/kernel/init_task.o
 

--
Hirokazu Takata <[email protected]>
Linux/M32R Project: http://www.linux-m32r.org/

author
#2 lore raw
In reply to Hirokazu Takata
Quoted 10 lines · Hirokazu Takata

Please don't specify __BIG_ENDIAN__. ;-)
We have supported both big- and little-endian for the m32r kernel. I hope it will be kept unconditional.

Now, the endianness is to be determined by a (cross)compiler:
- For the big-endian, a compiler (m32r-linux-gcc or m32r-linux-gnu-gcc) provides a predefined macro __BIG_ENDIAN__.
- For little-endian, a compiler (m32rle-linux-gcc or m32rle-linux-gnu-gcc) provides a predefined macro __LITTLE_ENDIAN__.

Quoted 3 lines

Here is a modified patch.

Thank you.

You do realize that this way sparse will get neither? It does not pick predefined symbols from gcc; thus the -D<your_arch>, etc.

One solution would be to really run cross-gcc on something like

#define HAVE(x) -D##x=x
#ifdef __LITTLE_ENDIAN__
HAVE(__LITTLE_ENDIAN__)
#endif
#ifdef __BIG_ENDIAN__
HAVE(__BIG_ENDIAN__)
#endif

with grep -- -D on the output and have it added to CHECKFLAGS. The only problem being, for a generic solution we want to make sure that this sucker is run _after_ we have decided on gcc arguments.

Note that it's potentially more than just m32r - e.g. s390 wants __s390x__ added only for case of 64bit target. Similar for powerpc if these get merged, etc.

Sam, any help in that area? Ideally we want to have something equivalent to
PREDEFINED_WE_MIGHT_WANT = __m32r__ __LITTLE_ENDIAN__ __BIG_ENDIAN__ and CHECKFLAGS done from that - basically, the subset of cross-gcc predefined symbols reproduced for sparse. Ideally with -m64 added if we have sizeof(long) == 8 on target, to take care of all that crap in one go.

Suggestions?

#12 lore raw
In reply to Hirokazu Takata
Quoted 6 lines · Hirokazu Takata

Now, the endianness is to be determined by a (cross)compiler:
- For the big-endian, a compiler (m32r-linux-gcc or m32r-linux-gnu-gcc) provides a predefined macro __BIG_ENDIAN__.
- For little-endian, a compiler (m32rle-linux-gcc or m32rle-linux-gnu-gcc) provides a predefined macro __LITTLE_ENDIAN__.

Hmm.. You need to tell sparse _some_ way which one you use, since sparse won't do it.

Picking one at random is fine, of course. It doesn't even have to match the one you'll compile with, although that means that sparse will obviously be testing a different configuration than the one you'd actually compile.

So I think having -D__BIG_ENDIAN__ in the sparse flags is better than not having anything at all (since otherwise it won't be able to check anything). And having something that matches the compiler would be better still.

Linus

#3 lore raw
In reply to Al Viro
Quoted 1 line · Al Viro

You do realize that this way sparse will get neither? It does not pick predefined symbols from gcc; thus the -D<your_arch>, etc.

How about sticking this in some global Makefile somewhere? This will give sparse the same list of defines that GCC uses:

CHECKFLAGS += $(shell echo | gcc -E - -dM | sed -re 's/^#define +([^ ]+) +(.*)$/-D\1=\2/g')

Or you could do this:

include/linux/checkerdefines.h:

        echo | gcc -E - -dM >$@

And in linux/stddef.h or linux/compiler.h or something do:
#if defined(__CHECKER__) and not defined(_LINUX_CHECKERDEFINES_H)

# define _LINUX_CHECKERDEFINES_H 1
# include <linux/checkerdefines.h>
#endif

Cheers,
Kyle Moffett

--
Q: Why do programmers confuse Halloween and Christmas?
A: Because OCT 31 == DEC 25.

author
#4 lore raw
Quoted 7 lines · Kyle Moffett
Quoted 2 lines · Al Viro

You do realize that this way sparse will get neither? It does not pick predefined symbols from gcc; thus the -D<your_arch>, etc.

How about sticking this in some global Makefile somewhere? This will give sparse the same list of defines that GCC uses:

Quoted 7 lines

CHECKFLAGS += $(shell echo | gcc -E - -dM | sed -re 's/^#define +([^ ] +) +(.*)$/-D\1=\2/g')

Or you could do this:

include/linux/checkerdefines.h:

        echo | gcc -E - -dM >$@

*Hell*, no.
1) any use of cross-gcc is immediately b0rken
2) anything like building i386 on amd64 (native toolchain, -m32
in arguments) is b0rken
3) way, way, *WAY* too much spew. gcc pre-defines shitloads of
stuff and some of that stuff very definitely should not be there for sparse.

IOW, it's not that simple. You need final CC, CFLAGS and CPPFLAGS for that. Moreover, your variant won't do at all - change .config and you might need to rebuild that file on biarch platforms (parisc and s390 at the very least; ppc/ppc64 are also moving in that direction).

#5 lore raw
Quoted 4 lines · Al Viro

*Hell*, no.
1) any use of cross-gcc is immediately b0rken
2) anything like building i386 on amd64 (native toolchain, -m32
in arguments) is b0rken

So s/gcc/$(SOME_MAKEFILE_VARIABLE)/g I was simplifying for the sake of suggestion/example; adding in support for all of those is nontrivial and best left to those who thoroughly understand the kernel Makefile system (IE: Not me :-D).

Quoted 2 lines

3) way, way, *WAY* too much spew. gcc pre-defines shitloads of
stuff and some of that stuff very definitely should not be there for sparse.

Why not? Some of that stuff may get used in kernel headers, which sparse should definitely have defined. Besides, sparse is designed to check C source code, which will be compiled with said GCC using those preprocessing defines. Why should it use a different set of defines?

Quoted 1 line

IOW, it's not that simple. You need final CC, CFLAGS and CPPFLAGS for that.

See responses to (1) and (2)

Quoted 1 line

Moreover, your variant won't do at all - change .config and you might need to rebuild that file on biarch platforms (parisc and s390 at the very least; ppc/ppc64 are also moving in that direction).

We do far worse messes in per-arch Makefiles, and this seems reasonably easy to put in an arch-generic makefile, unless there is some significant complication you didn't mention.

Cheers,
Kyle Moffett

--
I have yet to see any problem, however complicated, which, when you looked at it in the right way, did not become still more complicated.
-- Poul Anderson

author
#6 lore raw
Quoted 9 lines · Kyle Moffett
Quoted 3 lines

3) way, way, *WAY* too much spew. gcc pre-defines shitloads of
stuff and some of that stuff very definitely should not be there for sparse.

Why not? Some of that stuff may get used in kernel headers, which sparse should definitely have defined. Besides, sparse is designed to check C source code, which will be compiled with said GCC using those preprocessing defines. Why should it use a different set of defines?

First of all, some of that stuff should not be used in kernel headers and getting a warning about such uses is a Good Thing(tm). What's more, some are actively *wrong* for kernel - __STDC_HOSTED__, for one, is simply a lie. And no, sparse (or any other C compiler) is not required to have the same pile as gcc does.

There's another reason why limited subset is good - it documents the subset we are using. And having to reach gcc source just to figure out which architectures might have given symbol... No, thanks.

#7 lore raw
Quoted 4 lines · Al Viro
Quoted 1 line · Kyle Moffett

Why not? Some of that stuff may get used in kernel headers, which sparse should definitely have defined. Besides, sparse is designed to check C source code, which will be compiled with said GCC using those preprocessing defines. Why should it use a different set of defines?

First of all, some of that stuff should not be used in kernel headers and getting a warning about such uses is a Good Thing(tm). What's more, some are actively *wrong* for kernel - __STDC_HOSTED__, for one, is simply a lie.

So maybe put a "grep" in the middle to select the ones you want or "grep -v" to remove the ones you don't? I don't see why this is such a big deal. There are obviously missing defines that sparse needs to correctly operate with kernel headers, and I'm not sure why we should specify them in several places in the Makefiles, including one set for each arch.?

Quoted 1 line

And no, sparse (or any other C compiler) is not required to have the same pile as gcc does.

But when we're using sparse to check kernel sources, it should have a similar set to what the regular compiler (IE: gcc) has when building kernel sources.

Cheers,
Kyle Moffett

--
Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.
-- Brian Kernighan

#9 lore raw
In reply to Al Viro
Quoted 10 lines

Sam, any help in that area? Ideally we want to have something equivalent to
PREDEFINED_WE_MIGHT_WANT = __m32r__ __LITTLE_ENDIAN__ __BIG_ENDIAN__ and CHECKFLAGS done from that - basically, the subset of cross-gcc predefined symbols reproduced for sparse. Ideally with -m64 added if we have sizeof(long) == 8 on target, to take care of all that crap in one go.

Suggestions?

The most simple solution would be to provide a small script that create the defines as we need and run it for each invocation of sparse. The script should use same trick as scripts/gcc-version.sh does.

So we could have:
#!/bin/sh
compiler="$*"

BIG=$(echo __BIG_ENDIAN__ | $compiler -E -xc - | tail -n 1) LITTLE=$(echo __LITTLE_ENDIAN__ | $compiler -E -xc - | tail -n 1)

Then BIG would be set to "1" if this is big endian, and "__BIG_ENDIAN__" if little endian.
A little bit of shell script and we have the defines we want for m32r. Then we could add calling this script as part of sparse invocation.

The better solution would be to find the relevant flags before we start building the kernel. This is not so easy if we want access to final CFLAGS. But for the architecture the important ones are defined in arch/Makefile so placing this late in the file and use $(CC) $(CFLAGS) should be OK in almost all cases.

Too late for me to cook up a patch right now - but simple. Oh, and I agree. We do NOT want all the gcc defined flags.

We should restrict to a subset in the kernel - so it's good if sparse warns/errors out on usage of the non-trivial defines.
Sam

author
#8 lore raw
In reply to Kyle Moffett
Quoted 6 lines · Kyle Moffett
Quoted 2 lines

And no, sparse (or any other C compiler) is not required to have the same pile as gcc does.

But when we're using sparse to check kernel sources, it should have a similar set to what the regular compiler (IE: gcc) has when building kernel sources.

Have you ever looked at that set of defines? Note that even with gcc most of them are supposed to be only used in gcc headers. Most of those are never used in the kernel code, and for a good reason. Only two are actually used - stdarg.h and (for raid6 with CONFIG_ALTIVEC) altivec.h. Guess what? The former doesn't use any of these defines and in case of the latter the only define used would be a lie - sparse does *not* handle -maltivec.

It gets even better when you get to defines that describe compiler capacities. sparse, for example, does *not* support wchar_t et.al.; gcc does. Should we lie about that support? Should sparse binary that kinda-sorta imitates e.g. builtins of gcc 3.3 (and declares __GNUC... accordingly) pick bogus ones from gcc4 you are using for build?

There are very few defines we really want out of that pile. "Take everything" is nowhere near sanity, especially if you consider the differences between gcc versions (and gcc builds, while we are at it).

author
#13 lore raw
In reply to Linus Torvalds
Quoted 18 lines · Linus Torvalds
Quoted 6 lines · Hirokazu Takata

Now, the endianness is to be determined by a (cross)compiler:
- For the big-endian, a compiler (m32r-linux-gcc or m32r-linux-gnu-gcc) provides a predefined macro __BIG_ENDIAN__.
- For little-endian, a compiler (m32rle-linux-gcc or m32rle-linux-gnu-gcc) provides a predefined macro __LITTLE_ENDIAN__.

Hmm.. You need to tell sparse _some_ way which one you use, since sparse won't do it.

Picking one at random is fine, of course. It doesn't even have to match the one you'll compile with, although that means that sparse will obviously be testing a different configuration than the one you'd actually compile.

Quoted 4 lines

So I think having -D__BIG_ENDIAN__ in the sparse flags is better than not having anything at all (since otherwise it won't be able to check anything). And having something that matches the compiler would be better still.

Really interesting question is why do we need two toolchains at all. Note that little-endian m32r gcc at least appears to understand -mbe/-mbig-endian and binutils handles both endianness just fine.

Does that really work and is there any reason why big-endian one could not handle -mle the same way with minimal changes? IOW, do they have to differ in anything except the default target endianness?

Note that dependencies on "host endianness == target endianness" are practically guaranteed to cause bugs in cross-compiler, so any of those would very likely to be a bug in need of fixing anyway...

author
#10 lore raw
In reply to Sam Ravnborg
Quoted 18 lines · Sam Ravnborg
Quoted 10 lines

Sam, any help in that area? Ideally we want to have something equivalent to
PREDEFINED_WE_MIGHT_WANT = __m32r__ __LITTLE_ENDIAN__ __BIG_ENDIAN__ and CHECKFLAGS done from that - basically, the subset of cross-gcc predefined symbols reproduced for sparse. Ideally with -m64 added if we have sizeof(long) == 8 on target, to take care of all that crap in one go.

Suggestions?

The most simple solution would be to provide a small script that create the defines as we need and run it for each invocation of sparse. The script should use same trick as scripts/gcc-version.sh does.

So we could have:
#!/bin/sh
compiler="$*"

Quoted 7 lines

BIG=$(echo __BIG_ENDIAN__ | $compiler -E -xc - | tail -n 1) LITTLE=$(echo __LITTLE_ENDIAN__ | $compiler -E -xc - | tail -n 1)

Then BIG would be set to "1" if this is big endian, and "__BIG_ENDIAN__" if little endian.
A little bit of shell script and we have the defines we want for m32r. Then we could add calling this script as part of sparse invocation.

Oww... Somehow that doesn't look like a good idea. Two spawned gcc per sparse run? Not pretty...

Quoted 5 lines

The better solution would be to find the relevant flags before we start building the kernel. This is not so easy if we want access to final CFLAGS. But for the architecture the important ones are defined in arch/Makefile so placing this late in the file and use $(CC) $(CFLAGS) should be OK in almost all cases.

Alternatively, we could generate something like
#define HAVE(x) -D##x=x
#ifdef <symbol>
HAVE(<symbol>)
#endif
...

into arch/$ARCH/kernel/predefines.S, have arch/$ARCH/kernel/predefines.s generated by normal rules and have echo grep -- -D arch/.../predefines.s do the rest...

#14 lore raw
In reply to Al Viro

From: Al Viro <[email protected]>
Date: Wed, 28 Sep 2005 01:18:40 +0100

Quoted 12 lines · Linus Torvalds
Quoted 11 lines · Hirokazu Takata
Quoted 6 lines

Now, the endianness is to be determined by a (cross)compiler:
- For the big-endian, a compiler (m32r-linux-gcc or m32r-linux-gnu-gcc) provides a predefined macro __BIG_ENDIAN__.
- For little-endian, a compiler (m32rle-linux-gcc or m32rle-linux-gnu-gcc) provides a predefined macro __LITTLE_ENDIAN__.

Hmm.. You need to tell sparse _some_ way which one you use, since sparse won't do it.

In the top Makefile, CHECKFLAGS is defined as follows:
CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ $(CF)

Can I use CF for user specific CHECKFLAGS ?
What do you think about we specify it manyally for a biendian target?

ex. m32r
sparse check for biendian target

  $ make ARCH=m32r CF=-D__BIG_ENDIAN__=1 C=1   ... for big-endian
  $ make ARCH=m32r CF=-D__LITTLE__=1 C=1       ... for little-endian

arch/m32r/Makefile:
CHECKFLAGS += -D__m32r__

Quoted 9 lines
Quoted 9 lines

Picking one at random is fine, of course. It doesn't even have to match the one you'll compile with, although that means that sparse will obviously be testing a different configuration than the one you'd actually compile.

So I think having -D__BIG_ENDIAN__ in the sparse flags is better than not having anything at all (since otherwise it won't be able to check anything). And having something that matches the compiler would be better still.

I understand what you said.
I think having -D__BIG_ENDIAN__ is one of the realistic solutions, because it seems that there is no good solution about this problem as other people said in this ML-thread.

Quoted 3 lines

Really interesting question is why do we need two toolchains at all. Note that little-endian m32r gcc at least appears to understand -mbe/-mbig-endian and binutils handles both endianness just fine.

IMHO, I think it is for convinience of building userland programs. For example, in case of building programs (like Debian/GNU Linux packages), it is useful that we can simply use a compiler without not only modifying any source/Makefile but also specifing any target-specific options of the compiler.

Quoted 3 lines

Does that really work and is there any reason why big-endian one could not handle -mle the same way with minimal changes? IOW, do they have to differ in anything except the default target endianness?

I have no idea...

Quoted 3 lines

Note that dependencies on "host endianness == target endianness" are practically guaranteed to cause bugs in cross-compiler, so any of those would very likely to be a bug in need of fixing anyway...

I think so, too.
I hope we can do check both of target endianness intentionally.

-- Takata

author
#15 lore raw
Quoted 10 lines · Hirokazu Takata

In the top Makefile, CHECKFLAGS is defined as follows:
CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ $(CF)

Can I use CF for user specific CHECKFLAGS ?
What do you think about we specify it manyally for a biendian target?

ex. m32r
sparse check for biendian target

  $ make ARCH=m32r CF=-D__BIG_ENDIAN__=1 C=1   ... for big-endian
  $ make ARCH=m32r CF=-D__LITTLE__=1 C=1       ... for little-endian

That's doable, but... I'd rather not make it mandatory - CF is for "specific for this build" flags and when used for parallel cross-builds such requirements make it rather nasty to deal with.

Quoted 4 lines

I understand what you said.
I think having -D__BIG_ENDIAN__ is one of the realistic solutions, because it seems that there is no good solution about this problem as other people said in this ML-thread.

Note that it's not just sparse. There is a legitimate need to make endianness of target visible to Kconfig - not to mention anything else, there are drivers that are really broken on big-endian targets.