Skip to contentSkip to conversation

Announce: gcc bogus warning repository

DISCUSSION Jeff Garzik 10 participants 24 messages Oct 1, 2006 – Oct 4, 2006

lore
Share & tools
lore
#0 lore raw

The level of warnings in a kernel build has lately increased to the point where it is hiding bugs and otherwise making life difficult.

In particular, recent gcc versions throw warnings when it thinks a variable "MAY be used uninitialized", which is not terribly helpful due to the fact that most of these warnings are bogus.

For those that may find this valuable, I have started a git repo that silences these bogus warnings, after careful auditing of code paths to ensure that the warning truly is bogus.

The results may be found in the "gccbug" branch of
git://git.kernel.org/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git

This repository will NEVER EVER be pushed upstream. It exists solely for those who want to decrease their build noise, thereby exposing true bugs.

The audit has already uncovered several minor bugs, lending credence to my theory that too many warnings hides bugs.

#1 lore raw
Quoted 3 lines · Jeff Garzik

This repository will NEVER EVER be pushed upstream. It exists solely for those who want to decrease their build noise, thereby exposing true bugs.

Another way to deal with that is to use remap-log... See
git://git.kernel.org/pub/scm/linux/kernel/git/viro/remapper.git/
for current version; it's _very_ good at reducing noise in diffs from line renumbering, leaving the real changes. While the number of bogus warnings is very high, indeed, the rate at which they appear is not _that_ horrible...

#2 lore raw
In reply to Jeff Garzik
Quoted 7 lines · Jeff Garzik

The level of warnings in a kernel build has lately increased to the point where it is hiding bugs and otherwise making life difficult.

In particular, recent gcc versions throw warnings when it thinks a variable "MAY be used uninitialized", which is not terribly helpful due to the fact that most of these warnings are bogus.

Quoted 7 lines

For those that may find this valuable, I have started a git repo that silences these bogus warnings, after careful auditing of code paths to ensure that the warning truly is bogus.

The results may be found in the "gccbug" branch of
git://git.kernel.org/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git

Steven Rostedt an I worked on this problem in May. Steven came up with, a nice way to handle these warnings, which doesn't increase code size. Here's the post if your interested.

http://lkml.org/lkml/2006/5/11/50

Daniel

#18 lore raw
In reply to Jeff Garzik
Quoted 15 lines · On Sun, 01 Oct 2006 09:44:55 -0400 Jeff Garzik wrote:

The level of warnings in a kernel build has lately increased to the point where it is hiding bugs and otherwise making life difficult.

In particular, recent gcc versions throw warnings when it thinks a variable "MAY be used uninitialized", which is not terribly helpful due to the fact that most of these warnings are bogus.

For those that may find this valuable, I have started a git repo that silences these bogus warnings, after careful auditing of code paths to ensure that the warning truly is bogus.

The results may be found in the "gccbug" branch of
git://git.kernel.org/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git

Quoted 6 lines

This repository will NEVER EVER be pushed upstream. It exists solely for those who want to decrease their build noise, thereby exposing true bugs.

The audit has already uncovered several minor bugs, lending credence to my theory that too many warnings hides bugs.

I usually build with must_check etc. enabled then grep them away if I want to look for other messages. I think that the situation is not so disastrous.

---
~Randy

#19 lore raw
Quoted 11 lines · Randy Dunlap
Quoted 9 lines · On Sun, 01 Oct 2006 09:44:55 -0400 Jeff Garzik wrote:

The level of warnings in a kernel build has lately increased to the point where it is hiding bugs and otherwise making life difficult.

In particular, recent gcc versions throw warnings when it thinks a variable "MAY be used uninitialized", which is not terribly helpful due to the fact that most of these warnings are bogus.

For those that may find this valuable, I have started a git repo that silences these bogus warnings, after careful auditing of code paths to ensure that the warning truly is bogus.

The results may be found in the "gccbug" branch of
git://git.kernel.org/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git

Quoted 7 lines
Quoted 3 lines

This repository will NEVER EVER be pushed upstream. It exists solely for those who want to decrease their build noise, thereby exposing true bugs.

The audit has already uncovered several minor bugs, lending credence to my theory that too many warnings hides bugs.

I usually build with must_check etc. enabled then grep them away if I want to look for other messages. I think that the situation is not so disastrous.

I think it's both sad, and telling, that the high level of build noise has trained kernel hackers to tune out warnings, and/or build tools of ever-increasing sophistication just to pick out the useful messages from all the noise.

If you have to grep useful stuff out of the noise, you've already lost.

Jeff

#20 lore raw
Quoted 1 line · Jeff Garzik

Randy Dunlap wrote:

[snip]

Quoted 11 lines
Quoted 10 lines
Quoted 6 lines

This repository will NEVER EVER be pushed upstream. It exists solely for those who want to decrease their build noise, thereby exposing true bugs.

The audit has already uncovered several minor bugs, lending credence to my theory that too many warnings hides bugs.

I usually build with must_check etc. enabled then grep them away if I want to look for other messages. I think that the situation is not so disastrous.

Quoted 6 lines

I think it's both sad, and telling, that the high level of build noise has trained kernel hackers to tune out warnings, and/or build tools of ever-increasing sophistication just to pick out the useful messages from all the noise.

If you have to grep useful stuff out of the noise, you've already lost.

The question is whether the GCC guys are actually doing anything about the problem. If they are, we should do nothing. If they aren't, maybe it's time for "x = x" hacks like Steven's.

Is GCC 4.2 any better with this class of warnings?

Signature · 5 lines
Cheers,
Alistair.

Final year Computer Science undergraduate.
1F2 55 South Clerk Street, Edinburgh, UK.
#21 lore raw
Quoted 19 lines · Alistair John Strachan
Quoted 1 line · Jeff Garzik

Randy Dunlap wrote:

[snip]

Quoted 16 lines
Quoted 10 lines
Quoted 6 lines

This repository will NEVER EVER be pushed upstream. It exists solely for those who want to decrease their build noise, thereby exposing true bugs.

The audit has already uncovered several minor bugs, lending credence to my theory that too many warnings hides bugs.

I usually build with must_check etc. enabled then grep them away if I want to look for other messages. I think that the situation is not so disastrous.

I think it's both sad, and telling, that the high level of build noise has trained kernel hackers to tune out warnings, and/or build tools of ever-increasing sophistication just to pick out the useful messages from all the noise.

Quoted 6 lines
Quoted 1 line

If you have to grep useful stuff out of the noise, you've already lost.

The question is whether the GCC guys are actually doing anything about the problem. If they are, we should do nothing. If they aren't, maybe it's time for "x = x" hacks like Steven's.
...

Let's be fair to gcc:

gcc correctly tells it "may be used uninitialized" - that's different from cases where gcc tells "is used uninitialized".

Sometimes, it requires _much_ context seeing that a condition is actually not possible.

And there are even cases where it's technically impossible for a compiler to figure out itself that a condition is not possible.

Quoted 2 lines

Cheers,
Alistair.

cu
Adrian

Signature · 4 lines
       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed
#3 lore raw
In reply to Daniel Walker
Quoted 8 lines · On Sun, 01 Oct 2006 08:40:13 -0700 Daniel Walker <[email protected]> wrote:
Quoted 7 lines · Jeff Garzik

The level of warnings in a kernel build has lately increased to the point where it is hiding bugs and otherwise making life difficult.

In particular, recent gcc versions throw warnings when it thinks a variable "MAY be used uninitialized", which is not terribly helpful due to the fact that most of these warnings are bogus.

Quoted 7 lines
Quoted 7 lines

For those that may find this valuable, I have started a git repo that silences these bogus warnings, after careful auditing of code paths to ensure that the warning truly is bogus.

The results may be found in the "gccbug" branch of
git://git.kernel.org/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git

Funny - I started doing the exact same thing here just eight hours ago. I'll send you anything which isn't in that git tree, thanks.

Quoted 5 lines

Steven Rostedt an I worked on this problem in May. Steven came up with, a nice way to handle these warnings, which doesn't increase code size. Here's the post if your interested.

http://lkml.org/lkml/2006/5/11/50

I think we should merge that and use it. No overhead, self-documenting, easily greppable for.

The downsides are that it muckies up the source a little and introduces a very small risk that real use-uninitialised bugs will be hidden. But I believe the benefit outweighs those disadvantages.

#22 lore raw
In reply to Jeff Garzik
Quoted 28 lines · On Sun, 01 Oct 2006 13:20:45 -0400 Jeff Garzik wrote:
Quoted 20 lines · Randy Dunlap
Quoted 18 lines · On Sun, 01 Oct 2006 09:44:55 -0400 Jeff Garzik wrote:

The level of warnings in a kernel build has lately increased to the point where it is hiding bugs and otherwise making life difficult.

In particular, recent gcc versions throw warnings when it thinks a variable "MAY be used uninitialized", which is not terribly helpful due to the fact that most of these warnings are bogus.

For those that may find this valuable, I have started a git repo that silences these bogus warnings, after careful auditing of code paths to ensure that the warning truly is bogus.

The results may be found in the "gccbug" branch of
git://git.kernel.org/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git

This repository will NEVER EVER be pushed upstream. It exists solely for those who want to decrease their build noise, thereby exposing true bugs.

Quoted 6 lines
Quoted 2 lines

The audit has already uncovered several minor bugs, lending credence to my theory that too many warnings hides bugs.

I usually build with must_check etc. enabled then grep them away if I want to look for other messages. I think that the situation is not so disastrous.

Quoted 6 lines

I think it's both sad, and telling, that the high level of build noise has trained kernel hackers to tune out warnings, and/or build tools of ever-increasing sophistication just to pick out the useful messages from all the noise.

If you have to grep useful stuff out of the noise, you've already lost.

I often build with C=1 (sparse checking), so the amount of output has to be grepped IMO. It's certainly too much to read otherwise. We just have different perspectives, I guess.

---
~Randy

#4 lore raw
In reply to Andrew Morton
Quoted 3 lines · Andrew Morton

The downsides are that it muckies up the source a little and introduces a very small risk that real use-uninitialised bugs will be hidden. But I believe the benefit outweighs those disadvantages.

How about just marking the ones I've already done in #gccbug?

If I'm taking the time to audit the code, and separate out bogosities from real bugs, it would be nice not to see that effort wasted.

#gccbug includes _only_ the bogosities. I didn't just blindly paper over everything with a 'may be used uninitialized' warning. I'm well over halfway through the 'make allmodconfig' build, and as LKML emails can attest, have found several valid warnings.

Jeff

#5 lore raw
Quoted 9 lines · Jeff Garzik
Quoted 3 lines · Andrew Morton

The downsides are that it muckies up the source a little and introduces a very small risk that real use-uninitialised bugs will be hidden. But I believe the benefit outweighs those disadvantages.

How about just marking the ones I've already done in #gccbug?

If I'm taking the time to audit the code, and separate out bogosities from real bugs, it would be nice not to see that effort wasted.

There was a long thread on this, it's not about anyone not reviewing the code properly when the warning is first silenced. It's that future changes might create new problems that are also silenced. I don't think it's a huge concern, especially since there's was a config option to turn the warning backs on.

Daniel

#6 lore raw
Quoted 7 lines · Daniel Walker
Quoted 6 lines · Jeff Garzik
Quoted 3 lines · Andrew Morton

The downsides are that it muckies up the source a little and introduces a very small risk that real use-uninitialised bugs will be hidden. But I believe the benefit outweighs those disadvantages.

How about just marking the ones I've already done in #gccbug?

Quoted 7 lines
Quoted 1 line

If I'm taking the time to audit the code, and separate out bogosities from real bugs, it would be nice not to see that effort wasted.

There was a long thread on this, it's not about anyone not reviewing the code properly when the warning is first silenced. It's that future changes might create new problems that are also silenced. I don't think it's a huge concern, especially since there's was a config option to turn the warning backs on.

That doesn't address my question at all.

If there is no difference between real non-init bugs and bogus warnings, then a config option doesn't make any difference at all, does it? Real bugs are still hidden either way: if the warnings are turned on, the bugs are lost in the noise. if the warnings are turned off, the bugs are completely hidden.

Jeff

#7 lore raw
Quoted 1 line · Jeff Garzik

That doesn't address my question at all.

Did you have a question?

Quoted 5 lines

If there is no difference between real non-init bugs and bogus warnings, then a config option doesn't make any difference at all, does it? Real bugs are still hidden either way: if the warnings are turned on, the bugs are lost in the noise. if the warnings are turned off, the bugs are completely hidden.

If you turn the warnings on, at least you have a chance to see a warning even if it's mixed with others.

Daniel

#8 lore raw
Quoted 6 lines · Daniel Walker
Quoted 1 line · Jeff Garzik

That doesn't address my question at all.

Did you have a question?

Quoted 8 lines
Quoted 5 lines

If there is no difference between real non-init bugs and bogus warnings, then a config option doesn't make any difference at all, does it? Real bugs are still hidden either way: if the warnings are turned on, the bugs are lost in the noise. if the warnings are turned off, the bugs are completely hidden.

If you turn the warnings on, at least you have a chance to see a warning even if it's mixed with others.

And that's better than the current situation in which respects, exactly?

#9 lore raw
Quoted 13 lines · Al Viro
Quoted 12 lines · Daniel Walker
Quoted 1 line · Jeff Garzik

That doesn't address my question at all.

Did you have a question?

Quoted 5 lines

If there is no difference between real non-init bugs and bogus warnings, then a config option doesn't make any difference at all, does it? Real bugs are still hidden either way: if the warnings are turned on, the bugs are lost in the noise. if the warnings are turned off, the bugs are completely hidden.

Quoted 4 lines
Quoted 2 lines

If you turn the warnings on, at least you have a chance to see a warning even if it's mixed with others.

And that's better than the current situation in which respects, exactly?

Seeing the warnings is the current situation.

Daniel

#10 lore raw
Quoted 3 lines · Daniel Walker
Quoted 1 line

And that's better than the current situation in which respects, exactly?

Seeing the warnings is the current situation.

I bow to your incredible power of observation. Now that you've shared that revelation with the list, could you explain what does blanket silencing of these warnings buy you, oh wan^H^Hise one?

#11 lore raw
Quoted 8 lines · Al Viro
Quoted 3 lines · Daniel Walker
Quoted 1 line

And that's better than the current situation in which respects, exactly?

Seeing the warnings is the current situation.

I bow to your incredible power of observation. Now that you've shared that revelation with the list, could you explain what does blanket silencing of these warnings buy you, oh wan^H^Hise one?

Did you see me silencing anything (with your crystal ball?) ? Cause I'm not.

Daniel

#12 lore raw
Quoted 4 lines · Daniel Walker
Quoted 3 lines · Al Viro
Quoted 2 lines · Daniel Walker
Quoted 1 line

And that's better than the current situation in which respects, exactly?

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

Seeing the warnings is the current situation.

I bow to your incredible power of observation. Now that you've shared that revelation with the list, could you explain what does blanket silencing of these warnings buy you, oh wan^H^Hise one?

Did you see me silencing anything (with your crystal ball?) ? Cause I'm not.

And what, in your opinion, does the patch in question achieve if not that?

PS: sorry, I leave crystal balls to RMS et.al. - never had been into that kind of scam...

#13 lore raw
Quoted 11 lines · Al Viro
Quoted 10 lines · Daniel Walker
Quoted 8 lines · Al Viro
Quoted 3 lines · Daniel Walker
Quoted 1 line

And that's better than the current situation in which respects, exactly?

Seeing the warnings is the current situation.

I bow to your incredible power of observation. Now that you've shared that revelation with the list, could you explain what does blanket silencing of these warnings buy you, oh wan^H^Hise one?

Quoted 5 lines
Quoted 2 lines

Did you see me silencing anything (with your crystal ball?) ? Cause I'm not.

And what, in your opinion, does the patch in question achieve if not that?

You mean the patch from Steven posted in May? Well it does appear to silence the warning. You didn't like the approach it seems? Please tell me why .

Daniel

#14 lore raw
Quoted 7 lines · Daniel Walker
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines

I bow to your incredible power of observation. Now that you've shared that revelation with the list, could you explain what does blanket silencing of these warnings buy you, oh wan^H^Hise one?

Did you see me silencing anything (with your crystal ball?) ? Cause I'm not.

Quoted 6 lines
Quoted 2 lines

And what, in your opinion, does the patch in question achieve if not that?

You mean the patch from Steven posted in May? Well it does appear to silence the warning. You didn't like the approach it seems? Please tell me why .

Read the list archives...

#16 lore raw
In reply to Andrew Morton

Andrew> The downsides are that it muckies up the source a little Andrew> and introduces a very small risk that real Andrew> use-uninitialised bugs will be hidden. But I believe the Andrew> benefit outweighs those disadvantages.

Not sure I agree -- it adds one more thing that must be maintained when reorganizing code. I think there is a pretty high risk of this sort of warning silencing hiding a bug introduced later, which would have triggered an "is used uninitialized" warning.

Perhaps asking for a gcc flag that turns off "may be used" warnings but leaves "is used" warnings would be useful (or does it already exist?)

#15 lore raw
In reply to Al Viro
Quoted 8 lines · On Sun, 1 Oct 2006 20:33:30 +0100 Al Viro <[email protected]> wrote:
Quoted 7 lines · Daniel Walker
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines

I bow to your incredible power of observation. Now that you've shared that revelation with the list, could you explain what does blanket silencing of these warnings buy you, oh wan^H^Hise one?

Did you see me silencing anything (with your crystal ball?) ? Cause I'm not.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

And what, in your opinion, does the patch in question achieve if not that?

You mean the patch from Steven posted in May? Well it does appear to silence the warning. You didn't like the approach it seems? Please tell me why .

Read the list archives...

There isn't much point in doing that if they're as useless as this thread.

Fact is, those bogus warnings are harmful and cause real problems to not be noticed. There is value in finding some way of making real warnings apparent to all developers and testers.

An external post-processor isn't very useful because few people will run it. If we can integrate such a thing into the build system and make it available to all developers and testers then OK.

#17 lore raw
In reply to Daniel Walker
Quoted 16 lines · Daniel Walker
Quoted 7 lines · Jeff Garzik

The level of warnings in a kernel build has lately increased to the point where it is hiding bugs and otherwise making life difficult.

In particular, recent gcc versions throw warnings when it thinks a variable "MAY be used uninitialized", which is not terribly helpful due to the fact that most of these warnings are bogus.

Quoted 7 lines

For those that may find this valuable, I have started a git repo that silences these bogus warnings, after careful auditing of code paths to ensure that the warning truly is bogus.

The results may be found in the "gccbug" branch of
git://git.kernel.org/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git

Quoted 7 lines

Steven Rostedt an I worked on this problem in May. Steven came up with, a nice way to handle these warnings, which doesn't increase code size. Here's the post if your interested.

http://lkml.org/lkml/2006/5/11/50

Daniel

I wouldn't hide this in a macro, though. This could cause future maintainers a lot of problems after code is revised. If you are going to assign a variable's value to itself to eliminate compiler warnings, please do it openly, preferably with a comment.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.16.24 on an i686 machine (5592.72 BogoMips). New book: http://www.AbominableFirebug.com/
_


****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

#23 lore raw
In reply to Jeff Garzik
Quoted 2 lines · Jeff Garzik

If you have to grep useful stuff out of the noise, you've already lost.

My initial thought was to agree. And I still almost agree with this statement.

Almost, because every so often, it makes sense to see a shipload of warnings and manually weed through them all. 99% will be false positives, but there is the remaining 1% indicating real bugs.

Of course, once these have been dealt with, the warnings give 100% noise and 0% useful output. It is time to wait another year or four before enough bugs have accumulated to make it worth the effort again.

That said, if you need to grep every day, you've definitely lost.

Jörn

Signature · 4 lines
Write programs that do one thing and do it well. Write programs to work
together. Write programs to handle text streams, because that is a
universal interface.
-- Doug MacIlroy