Interesting race condition...

DISCUSSION Rob Landley 13 participants 26 messages Jul 23, 2004 – Aug 20, 2004 Proc filesystem

Share & tools

Rob Landley

author

2004-07-23 03:04 UTC #0 lore raw

I just saw a funky thing. Here's the cut and past from the xterm...

[root@(none) root]# ps ax | grep hack
9964 pts/1 R 0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm HISTSIZE=1000 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=
[root@(none) root]# ps ax | grep hack
9966 pts/1 S 0:00 grep hack

Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps or the 2.6.7 kernel or what...

Rob

Signature · 2 lines

www.linucon.org: Linux Expo and Science Fiction Convention
October 8-10, 2004 in Austin Texas.  (I'm the con chair.)

Barry K. Nathan

2004-07-23 07:33 UTC #1 lore raw

Quoted 2 lines · Rob Landley

Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps or the 2.6.7 kernel or what...

I've seen this too, on Fedora Core 2 and pretty recent (within the last 2 weeks) fc-devel. Hard enough to reproduce (and not enough of a bother IMO) that I haven't filed it in Bugzilla though.

(Now that you mention Fedora Core, I can't remember seeing this with any other distributions.)

-Barry K. Nathan <[email protected]>

Hugo Mills

2004-07-23 07:56 UTC #2 lore raw

Quoted 10 lines · Barry K. Nathan

Quoted 2 lines · Rob Landley
Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps or the 2.6.7 kernel or what...
I've seen this too, on Fedora Core 2 and pretty recent (within the last 2 weeks) fc-devel. Hard enough to reproduce (and not enough of a bother IMO) that I haven't filed it in Bugzilla though.
(Now that you mention Fedora Core, I can't remember seeing this with any other distributions.)

I've seen it on Debian unstable/sid, running a 32-bit userspace on
a 64-bit amd64 2.6.6 from kernel.org.

Hugo.

Signature · 4 lines

=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 1C335860 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
    --- There is no dark side to the Moon, really. As a matter of ---    
                          fact,  it's all dark.

Rob Landley

author

2004-07-24 08:12 UTC #3 lore raw

In reply to Barry K. Nathan

Quoted 4 lines · Barry K. Nathan

Quoted 2 lines · Rob Landley
Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps or the 2.6.7 kernel or what...

Quoted 8 lines

I've seen this too, on Fedora Core 2 and pretty recent (within the last 2 weeks) fc-devel. Hard enough to reproduce (and not enough of a bother IMO) that I haven't filed it in Bugzilla though.
(Now that you mention Fedora Core, I can't remember seeing this with any other distributions.)
-Barry K. Nathan <[email protected]>

Oh I can't reproduce it either. (Maybe if I set some kind of loop and left it running for a few days...)

I just thought it was really odd. There should be a null terminator before the environment variables, and it's getting stomped somehow. Maybe it was right as the process was starting...?

Rob

Signature · 2 lines

www.linucon.org: Linux Expo and Science Fiction Convention
October 8-10, 2004 in Austin Texas.  (I'm the con chair.)

Marc Ballarin

2004-07-24 13:50 UTC #4 lore raw

Quoted 2 lines · Rob Landley <rob <at> landley.net> writes:

Oh I can't reproduce it either. (Maybe if I set some kind of loop and left it running for a few days...)

I could reproduce it on an otherwise idle system (2 GHz Athlon, kernel 2.6.7). On a loaded system the bug did not occur, which certainly indicates a race condition.

Using the following Bash script, the bug appeared 23 times in 122,221 iterations:
while [ 1 ];do
ps ax | grep hack >> TEST
done

The bug *seems* to be in bash, since an equivalent script in tcsh had no problems:
while ( 1 )
ps ax | grep hack >> TEST2
end

This issue has the potential to break a lot of shell scripts in an almost undebugable way. Should someone file a bug report via 'bashbug'?

Marc

David Weinehall

2004-07-26 17:33 UTC #5 lore raw

Quoted 15 lines · Marc Ballarin

Quoted 2 lines · Rob Landley <rob <at> landley.net> writes:
Oh I can't reproduce it either. (Maybe if I set some kind of loop and left it running for a few days...)
I could reproduce it on an otherwise idle system (2 GHz Athlon, kernel 2.6.7). On a loaded system the bug did not occur, which certainly indicates a race condition.
Using the following Bash script, the bug appeared 23 times in 122,221 iterations:
while [ 1 ];do
ps ax | grep hack >> TEST
done

Quoted 8 lines

The bug *seems* to be in bash, since an equivalent script in tcsh had no problems:
while ( 1 )
ps ax | grep hack >> TEST2
end
This issue has the potential to break a lot of shell scripts in an almost undebugable way. Should someone file a bug report via 'bashbug'?

I get this using posh as well, so it doesn't seem to be bash-specific.

Regards: David Weinehall

Signature · 3 lines

 /) David Weinehall <[email protected]> /) Northern lights wander      (\
//  Maintainer of the v2.0 kernel   //  Dance across the winter sky //
\)  http://www.acc.umu.se/~tao/    (/   Full colour fire           (/

Marc Ballarin

2004-07-26 19:14 UTC #6 lore raw

In reply to Marc Ballarin

Marc Ballarin <Ballarin.Marc <at> gmx.de> writes:

Ok, I could repruduce this issue in bash and tcsh, using both, procps 3.1.5/3.2.2 from procps.sourceforge.net and procps 2.0.16 from tech9.net/rml/procps.

So, the bug is not in the shell (which is obvious, on second thought), but in the kernel, glibc or procps - or the combination thereof.

I'm using kernel 2.6.7, gcc-3.3.3, and glibc-2.3.3 with NPTL on Gentoo.

Regards

P. Benie

2004-07-23 10:04 UTC #7 lore raw

In reply to Rob Landley

Quoted 9 lines · Rob Landley

I just saw a funky thing. Here's the cut and past from the xterm...
[root@(none) root]# ps ax | grep hack
9964 pts/1 R 0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm HISTSIZE=1000 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=
[root@(none) root]# ps ax | grep hack
9966 pts/1 S 0:00 grep hack
Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps or the 2.6.7 kernel or what...

The race is in the shell's pipeline - the processes don't start at exactly the same time, and sometimes ps has completed before the shell has started grep. This is the expected behaviour.

Peter

Rob Landley

author

2004-07-24 08:16 UTC #8 lore raw

Quoted 12 lines · P. Benie

Quoted 11 lines · Rob Landley
I just saw a funky thing. Here's the cut and past from the xterm...
[root@(none) root]# ps ax | grep hack
9964 pts/1 R 0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm
HISTSIZE=1000 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd= 40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*. com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz = [root@(none) root]# ps ax | grep hack
9966 pts/1 S 0:00 grep hack

Quoted 6 lines

Quoted 2 lines
Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps or the 2.6.7 kernel or what...
The race is in the shell's pipeline - the processes don't start at exactly the same time, and sometimes ps has completed before the shell has started grep. This is the expected behaviour.

It's expected behavior for PS to show a process's environment variables as part of its command line?

Quoted 1 line

Peter

Rob

Signature · 2 lines

www.linucon.org: Linux Expo and Science Fiction Convention
October 8-10, 2004 in Austin Texas.  (I'm the con chair.)

P. Benie

2004-07-24 09:08 UTC #9 lore raw

Quoted 2 lines · Rob Landley

It's expected behavior for PS to show a process's environment variables as part of its command line?

Duh! Need more coffee...

Peter

Bill Davidsen

2004-07-27 20:38 UTC #10 lore raw

In reply to Rob Landley

Quoted 18 lines · Rob Landley

Quoted 16 lines · P. Benie
Quoted 13 lines · Rob Landley
I just saw a funky thing. Here's the cut and past from the xterm...
[root@(none) root]# ps ax | grep hack
9964 pts/1 R 0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm
HISTSIZE=1000 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd= 40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*. com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz = [root@(none) root]# ps ax | grep hack
9966 pts/1 S 0:00 grep hack
Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps or the 2.6.7 kernel or what...

Quoted 6 lines

Quoted 3 lines
The race is in the shell's pipeline - the processes don't start at exactly the same time, and sometimes ps has completed before the shell has started grep. This is the expected behaviour.
It's expected behavior for PS to show a process's environment variables as part of its command line?

When piped. For instance
ps eaxf
does not, while
ps eaxf | cat
does, at least on my systems. I tried on RHEL AS3.0 thru a four year old version of Slackware, all did the same thing.

Signature · 3 lines

   -bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
 last possible moment - but no longer"  -me

Paul Jackson

2004-07-28 08:07 UTC #11 lore raw

Quoted 5 lines · Bill

When piped. For instance
ps eaxf
does not [show environ], while
ps eaxf | cat
does

Neither the behaviour I'm seeing on my SuSE 8.2 box, and the code I see in some random procps-3.2.1 I happened to have laying around, agree with your description.

Rather, both behaviour and code have the 'e' option as a BSD option (applicable if no '-' option flag) requesting that the environment be shown, and I get the environment so shown, regardless of whether the output is a pipe or a tty.

... This subthread is of course off-topic to Rob Landley's original post - he wasn't using the 'e' option.

Signature · 3 lines

                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <[email protected]> 1.650.933.1373

Robert White

2004-08-04 20:04 UTC #12 lore raw

In reply to Bill Davidsen

Using procps version 2.0.7 the inclusion of "e" in the arguments is documented to return environment of the process.

So the below issues with "ps eaxf" are only surprising in that you claim the pipe is necessary to see the environment. Ps has "always" done slightly different things when used at the head of a pipe. In particular it will trim it's right side when not in a pipe to keep the display readable, while in a pipe the lines will be "as long as necessary."

The question of why the original poster was getting the environment when only using "ps ax" is interesting. I'd look for PS_PERSONALITY (etc) in the ps-executing environment. [consult your manual.]

And so on...

Rob.

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Bill Davidsen
Sent: Tuesday, July 27, 2004 1:40 PM
To: [email protected]
Subject: Re: Interesting race condition...

Quoted 18 lines · Rob Landley

Quoted 16 lines · P. Benie
Quoted 13 lines · Rob Landley
I just saw a funky thing. Here's the cut and past from the xterm...
[root@(none) root]# ps ax | grep hack
9964 pts/1 R 0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm
HISTSIZE=1000 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd= 40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*. com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz = [root@(none) root]# ps ax | grep hack
9966 pts/1 S 0:00 grep hack
Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps or the 2.6.7 kernel or what...

Quoted 7 lines

Quoted 3 lines
The race is in the shell's pipeline - the processes don't start at exactly the same time, and sometimes ps has completed before the shell has started grep. This is the expected behaviour.
It's expected behavior for PS to show a process's environment variables as part of its command line?

When piped. For instance
ps eaxf
does not, while
ps eaxf | cat
does, at least on my systems. I tried on RHEL AS3.0 thru a four year old version of Slackware, all did the same thing.

Signature · 8 lines

    -bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Roger Luethi

2004-08-04 20:46 UTC #13 lore raw

Quoted 2 lines · Robert White

Using procps version 2.0.7 the inclusion of "e" in the arguments is documented to return environment of the process.

Not the environment of somebody else's process, though. But that is currently possible in mainline, the problem is not with ps but with the kernel.

Quoted 2 lines

The question of why the original poster was getting the environment when only using "ps ax" is interesting. I'd look for PS_PERSONALITY (etc) in

Basically, if anyone reads /proc/pid/cmdline early enough, when mm->arg_end is still 0, the kernel will blast out the process environment through that interface. Thus, you get the data of /proc/pid/environ without the access restrictions of that file. Not good if you happen to pass sensitive information using environment variables.

Check out the patch I posted earlier in this thread.

Roger

Paul Jackson

2004-07-28 08:10 UTC #14 lore raw

In reply to Rob Landley

Quoted 1 line · Rob

I just saw a funky thing. Here's the cut and past from the xterm...

Can you reproduce this by cat'ing /proc/<pid>/cmdline? Can you get a dump of the proc cmdline file to leak the environment sometimes?

It is this file that 'ps' is dumping for these options. Adding the 'e' option would also dump the /proc/<pid>/environ file (if readable).

But you aren't adding 'e', so presumably the environment is "leaking" into the the cmdline file.

I suspect a kernel bug here - the ps code seems rather obvious and unimpeachable.

Signature · 3 lines

                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <[email protected]> 1.650.933.1373

Marc Ballarin

2004-07-28 11:53 UTC #15 lore raw

Quoted 6 lines · On Wed, 28 Jul 2004 01:05:46 -0700 Paul Jackson <[email protected]> wrote:

Quoted 1 line · Rob
I just saw a funky thing. Here's the cut and past from the xterm...
Can you reproduce this by cat'ing /proc/<pid>/cmdline? Can you get a dump of the proc cmdline file to leak the environment sometimes?

Quoted 9 lines

It is this file that 'ps' is dumping for these options. Adding the 'e' option would also dump the /proc/<pid>/environ file (if readable).
But you aren't adding 'e', so presumably the environment is "leaking" into the the cmdline file.
I suspect a kernel bug here - the ps code seems rather obvious and unimpeachable.

I ran the following loop for a while (> 9 million times) and could not reproduce the bug, but that might just be coincidence. Conditions were the same as in my other, succesful test.

while [ 1 ];do
cat /proc/self/cmdline >> TEST
done

Marc

Rob Landley

author

2004-07-28 16:46 UTC #16 lore raw

Quoted 22 lines · Marc Ballarin

On Wed, 28 Jul 2004 01:05:46 -0700
Quoted 6 lines · Paul Jackson
Quoted 1 line · Rob
I just saw a funky thing. Here's the cut and past from the xterm...
Can you reproduce this by cat'ing /proc/<pid>/cmdline? Can you get a dump of the proc cmdline file to leak the environment sometimes?
Quoted 8 lines
It is this file that 'ps' is dumping for these options. Adding the 'e' option would also dump the /proc/<pid>/environ file (if readable).
But you aren't adding 'e', so presumably the environment is "leaking" into the the cmdline file.
I suspect a kernel bug here - the ps code seems rather obvious and unimpeachable.
I ran the following loop for a while (> 9 million times) and could not reproduce the bug, but that might just be coincidence. Conditions were the same as in my other, succesful test.

Quoted 5 lines

while [ 1 ];do
cat /proc/self/cmdline >> TEST
done
Marc

I might have actually just killed the process I was grepping for (I was grepping to see if it had actually gone away yet). Dunno if that helps...

I haven't been able to reproduce it either. I think the pipe to grep (or something similar) is important because your test isn't doing what mine did:
one process was looking at another process at the instant of process creation (or perhaps exit). Maybe the appropriate null terminator is written out after the process goes live?

This is a UP system, but I have preemption on...

Rob

Signature · 2 lines

www.linucon.org: Linux Expo and Science Fiction Convention
October 8-10, 2004 in Austin Texas.  (I'm the con chair.)

Rob Landley

author

2004-07-28 16:44 UTC #17 lore raw

In reply to Paul Jackson

Quoted 5 lines · Paul Jackson

Quoted 1 line · Rob
I just saw a funky thing. Here's the cut and past from the xterm...
Can you reproduce this by cat'ing /proc/<pid>/cmdline? Can you get a dump of the proc cmdline file to leak the environment sometimes?

I saw it exactly once, I'm afraid. Somebody else said they also saw it... once. It smelled to me like a race condition, with the process starting/exiting right as ps looked at it, but I haven't seen it again. (I could run the command in a loop overnight...)

Rob

Signature · 2 lines

www.linucon.org: Linux Expo and Science Fiction Convention
October 8-10, 2004 in Austin Texas.  (I'm the con chair.)

Tristan Wibberley

2004-07-28 19:21 UTC #18 lore raw

Quoted 11 lines · Rob Landley

Quoted 5 lines · Paul Jackson
Quoted 1 line · Rob
I just saw a funky thing. Here's the cut and past from the xterm...
Can you reproduce this by cat'ing /proc/<pid>/cmdline? Can you get a dump of the proc cmdline file to leak the environment sometimes?
I saw it exactly once, I'm afraid. Somebody else said they also saw it... once. It smelled to me like a race condition, with the process starting/exiting right as ps looked at it, but I haven't seen it again. (I could run the command in a loop overnight...)

I think I have seen this exactly once. I ran helixplayer from a bash command line and it quit, immediately spouting something about how it couldn't find a file whose name looked remarkably similar to some environment variables. I had put it down to screwed terminal settings so I couldn't see I had pasted something in, and I immediately issued a reset, so I have no more information I'm afraid.

Signature · 1 line

I have a little sig...

Roger Luethi

2004-07-30 00:01 UTC #19 lore raw

In reply to Rob Landley

Quoted 9 lines · Rob Landley

I just saw a funky thing. Here's the cut and past from the xterm...
[root@(none) root]# ps ax | grep hack
9964 pts/1 R 0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm HISTSIZE=1000 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=
[root@(none) root]# ps ax | grep hack
9966 pts/1 S 0:00 grep hack
Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps or the 2.6.7 kernel or what...

If somebody posted a solution for this, I didn't see it. There's a race in the kernel, and considering the permissions on /proc/PID/{cmdline,environ} a security bug as well: If you win the race with a starting process, you can read its environment.

This should plug the hole. Can you give it a spin?

Roger

fs/proc/base.c renamed from fs/proc/base.c.orig +2 -0

@@ -329,6 +329,8 @@ static int proc_pid_cmdline(struct task_
 	struct mm_struct *mm = get_task_mm(task);
 	if (!mm)
 		goto out;
+	if (!mm->arg_end)
+		goto out;	/* Shh! No looking before we're done */
 
  	len = mm->arg_end - mm->arg_start;

Jesper Juhl

2004-07-30 00:20 UTC #20 lore raw

Quoted 23 lines · Roger Luethi

Quoted 9 lines · Rob Landley
I just saw a funky thing. Here's the cut and past from the xterm...
[root@(none) root]# ps ax | grep hack
9964 pts/1 R 0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm HISTSIZE=1000 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=
[root@(none) root]# ps ax | grep hack
9966 pts/1 S 0:00 grep hack
Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps or the 2.6.7 kernel or what...
If somebody posted a solution for this, I didn't see it. There's a race in the kernel, and considering the permissions on /proc/PID/{cmdline,environ} a security bug as well: If you win the race with a starting process, you can read its environment.
This should plug the hole. Can you give it a spin?
Roger

fs/proc/base.c static int proc_pid_cmdline(struct task_ · line 336

Quoted 8 lines

fs/proc/base.c renamed from fs/proc/base.c.orig +2 -0

@@ -329,6 +329,8 @@ static int proc_pid_cmdline(struct task_
 	struct mm_struct *mm = get_task_mm(task);
 	if (!mm)
 		goto out;
+	if (!mm->arg_end)
+		goto out;	/* Shh! No looking before we're done */
 
 	len = mm->arg_end - mm->arg_start;

I might be reading the code wrong or not be fully aware of the circumstances here, so please bear with me, but I took a look at get_task_mm() and it can return NULL. If it does you'd end up dereferencing a NULL pointer with "if (!mm->arg_end)" - or is that garanteed not to happen here?

Wouldn't something like

if (!mm || !mm->arg_end)

	goto out;

be safer?

Just my two pennies..

- Jesper Juhl <[email protected]>

Jesper Juhl

2004-07-30 00:24 UTC #21 lore raw

Quoted 39 lines · Jesper Juhl

Quoted 26 lines · Roger Luethi
Quoted 10 lines · Rob Landley
I just saw a funky thing. Here's the cut and past from the xterm...
[root@(none) root]# ps ax | grep hack
9964 pts/1 R 0:00 grep hack HOSTNAME= SHELL=/bin/bash
TERM=xterm HISTSIZE=1000 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz= [root@(none) root]# ps ax | grep hack
9966 pts/1 S 0:00 grep hack
Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps or the 2.6.7 kernel or what...
If somebody posted a solution for this, I didn't see it. There's a race in the kernel, and considering the permissions on /proc/PID/{cmdline,environ} a security bug as well: If you win the race with a starting process, you can read its environment.
This should plug the hole. Can you give it a spin?
Roger
--- linux-2.6.8-rc2-bk1/fs/proc/base.c.orig 2004-07-30 01:43:23.535967505
+0200
+++ linux-2.6.8-rc2-bk1/fs/proc/base.c 2004-07-30 01:43:27.428303752
+0200
fs/proc/base.c static int proc_pid_cmdline(struct task_ · line 336
Quoted 8 lines
fs/proc/base.c renamed from fs/proc/base.c.orig +2 -0
@@ -329,6 +329,8 @@ static int proc_pid_cmdline(struct task_
 	struct mm_struct *mm = get_task_mm(task);
 	if (!mm)
 		goto out;
+	if (!mm->arg_end)
+		goto out;	/* Shh! No looking before we're done */
 
 	len = mm->arg_end - mm->arg_start;
 
I might be reading the code wrong or not be fully aware of the circumstances here, so please bear with me, but I took a look at get_task_mm() and it can return NULL. If it does you'd end up dereferencing a NULL pointer with "if (!mm->arg_end)" - or is that garanteed not to happen here?

Quoted 7 lines

Wouldn't something like
if (!mm || !mm->arg_end)
	goto out;
be safer?

Arrgh, that's what happens when you try to read patches at 02:25 For some reason my eyes read the patch as

- if (!mm)
- goto out;
+ if (!mm->arg_end)
+ goto out; /* Shh! No looking before we're done */

but that's not what it looks like :( Sorry about that. There's no problem except for me being too sleepy.

/Jesper Juhl

Marc Ballarin

2004-07-30 08:26 UTC #22 lore raw

In reply to Roger Luethi

Quoted 12 lines · On Fri, 30 Jul 2004 01:56:54 +0200 Roger Luethi <[email protected]> wrote:

If somebody posted a solution for this, I didn't see it. There's a race in the kernel, and considering the permissions on /proc/PID/{cmdline,environ} a security bug as well: If you win the race with a starting process, you can read its environment.
This should plug the hole. Can you give it a spin?
Roger

fs/proc/base.c static int proc_pid_cmdline(struct task_ · line 336

Quoted 8 lines

fs/proc/base.c renamed from fs/proc/base.c.orig +2 -0

@@ -329,6 +329,8 @@ static int proc_pid_cmdline(struct task_
 	struct mm_struct *mm = get_task_mm(task);
 	if (!mm)
 		goto out;
+	if (!mm->arg_end)
+		goto out;	/* Shh! No looking before we're done */
 
  	len = mm->arg_end - mm->arg_start;

Yes, this seems to fix it. First I replaced "goto out" with a printk, and the printks matched the occurence of the bug.
However, I got multiple printks per bug (between 2 and 7). Is that supposed to happen?

Anyway, I've added back the "goto out" (after printk), and the bug no longer
occurs. The printk still happens, so this code path does get hit.

Regards, and thanks for the fix

Roger Luethi

2004-07-30 08:38 UTC #23 lore raw

Quoted 9 lines · Marc Ballarin

Quoted 9 lines

fs/proc/base.c renamed from fs/proc/base.c.orig +2 -0

@@ -329,6 +329,8 @@ static int proc_pid_cmdline(struct task_
 	struct mm_struct *mm = get_task_mm(task);
 	if (!mm)
 		goto out;
+	if (!mm->arg_end)
+		goto out;	/* Shh! No looking before we're done */

Quoted 7 lines

fs/proc/base.c static int proc_pid_cmdline(struct task_ · line 336
Quoted 2 lines
fs/proc/base.c renamed from fs/proc/base.c.orig +0 -0
  	len = mm->arg_end - mm->arg_start;
 
Yes, this seems to fix it. First I replaced "goto out" with a printk, and the printks matched the occurence of the bug.
However, I got multiple printks per bug (between 2 and 7). Is that supposed to happen?

Every time we proceed through proc_pid_cmdline with mm->arg_end == 0 is a bug. AFAICT anyway. Whether you see an "occurence of the bug" depends on how you measure it. Of course you are wecome to add more printks following the code path to see how and when it propagates to where you are catching it.

Roger

Lee Revell

2004-08-20 10:15 UTC #24 lore raw

In reply to Roger Luethi

Quoted 11 lines · Roger Luethi

Quoted 9 lines · Rob Landley
I just saw a funky thing. Here's the cut and past from the xterm...
[root@(none) root]# ps ax | grep hack
9964 pts/1 R 0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm HISTSIZE=1000 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=
[root@(none) root]# ps ax | grep hack
9966 pts/1 S 0:00 grep hack
Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps or the 2.6.7 kernel or what...

Quoted 7 lines

If somebody posted a solution for this, I didn't see it. There's a race in the kernel, and considering the permissions on /proc/PID/{cmdline,environ} a security bug as well: If you win the race with a starting process, you can read its environment.
This should plug the hole. Can you give it a spin?

Was this ever merged? I just hit this bug with 2.6.8.1:

rlrevell@mindpipe:~$ ps auxww | grep jack
rlrevell 10498  0.1  5.9 28656 28624 pts/0   SLl+ 06:07   0:00 jackd -n 1000000 --realtime -d alsa --rate 48000 -D -P hw:0,0 -C hw:0,2 -p 32 -S
rlrevell 10502  0.1  2.3 11432 11432 pts/2   SLl+ 06:07   0:00 jack_simple_client foo
rlrevell 10509  0.0  0.0     4    4 pts/1    R+   06:09   0:00 grep jack SSH_AGENT_PID=667 TERM=rxvt SHELL=/bin/bash CVSROOT=:pserver:[email protected]:/cvsroot/alsa GTK_RC_FILES=/etc/gtk/gtkrc:/home/rlrevell/.gtkrc-1.2-gnome2 WINDOWID=33554437 USER=rlrevell LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.avi=01;35:*.fli=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.ogg=01;35:*.mp3=01;35:*.wav=01;35: GNOME_KEYRING_SOCKET=/tmp/keyring-oFgJ2h/socket SSH_AUTH_SOCK=/tmp/ssh-JvFuVGV623/agent.623 SESSION_MANAGER=local/mindpipe:/tmp/.ICE-unix/623 USERNAME=rlrevell DESKTOP_SESSION=default PATH=/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games PWD=/home/rlrevell GDMSESSION=default HISTCONTROL=ignoredups HOME=/home/rlrevell SHLVL=1 GNOME_DESKTOP_SESSION_ID=Default LOGNAME=rlrevell DISPLAY=:0.0 COLORTERM=rxvt XAUTHORITY=/home/rlrevell/.Xauthority _=/bin/grep /bin/grep    
rlrevell@mindpipe:~$ ps auxww | grep jack
rlrevell 10498  0.1  5.9 28656 28624 pts/0   SLl+ 06:07   0:00 jackd -n 1000000 --realtime -d alsa --rate 48000 -D -P hw:0,0 -C hw:0,2 -p 32 -S
rlrevell 10502  0.1  2.3 11432 11432 pts/2   SLl+ 06:07   0:00 jack_simple_client foo
rlrevell 10511  0.0  0.0  1576  464 pts/1    S+   06:09   0:00 grep jack
rlrevell@mindpipe:~$ ps auxww | grep jack
rlrevell 10498  0.1  5.9 28656 28624 pts/0   SLl+ 06:07   0:00 jackd -n 1000000 --realtime -d alsa --rate 48000 -D -P hw:0,0 -C hw:0,2 -p 32 -S
rlrevell 10502  0.1  2.3 11432 11432 pts/2   SLl+ 06:07   0:00 jack_simple_client foo
rlrevell 10513  0.0  0.0   240  124 pts/1    R+   06:09   0:00 grep jack
rlrevell@mindpipe:~$ ps auxww | grep jack
rlrevell 10498  0.1  5.9 28656 28624 pts/0   SLl+ 06:07   0:00 jackd -n 1000000 --realtime -d alsa --rate 48000 -D -P hw:0,0 -C hw:0,2 -p 32 -S
rlrevell 10502  0.1  2.3 11432 11432 pts/2   SLl+ 06:07   0:00 jack_simple_client foo
rlrevell 10515  0.0  0.0  1576  464 pts/1    S+   06:09   0:00 grep jack
rlrevell@mindpipe:~$ ps auxww | grep jack

Lee

Marc Ballarin

2004-08-20 12:41 UTC #25 lore raw

Quoted 3 lines · On Fri, 20 Aug 2004 06:15:09 -0400 Lee Revell <[email protected]> wrote:

Was this ever merged? I just hit this bug with 2.6.8.1:

It's only in -mm series so far.

Regards