Skip to contentSkip to conversation

binfmt_script and ^M

PATCH In review Ivo Timmermans 27 participants 48 messages Feb 27, 2001 – Mar 9, 2001 exec and binfmt

lore
Share & tools
lore
#0 lore raw

When running a script (perl in this case) that has DOS-style newlines (\r\n), Linux 2.4.2 can't find an interpreter because it doesn't recognize the \r. The following patch should fix this (untested).

Please Cc me on replies, I'm not on this list. Thanks.

binfmt_script.c renamed from binfmt_script.c~ +2 -0
@@ -36,6 +36,8 @@
 	bprm->buf[BINPRM_BUF_SIZE - 1] = '\0';
 	if ((cp = strchr(bprm->buf, '\n')) == NULL)
 		cp = bprm->buf+BINPRM_BUF_SIZE-1;
+	if (cp - 1 == '\r')
+	  cp--;
 	*cp = '\0';
 	while (cp > bprm->buf) {
 		cp--;
Signature · 1 line
Ivo Timmermans
#1 lore raw
Quoted 3 lines

When running a script (perl in this case) that has DOS-style newlines (\r\n), Linux 2.4.2 can't find an interpreter because it doesn't recognize the \r. The following patch should fix this (untested).

Fix the script. The kernel expects a specific format

Alan

#2 lore raw
In reply to Ivo Timmermans
Quoted 5 lines · Alan Cox
Quoted 3 lines

When running a script (perl in this case) that has DOS-style newlines (\r\n), Linux 2.4.2 can't find an interpreter because it doesn't recognize the \r. The following patch should fix this (untested).

Fix the script. The kernel expects a specific format

For what reason? Is it a standard to not allow it, or does it break other things?

Signature · 1 line
Ivo Timmermans
#3 lore raw
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

(\r\n), Linux 2.4.2 can't find an interpreter because it doesn't recognize the \r. The following patch should fix this (untested).

Fix the script. The kernel expects a specific format

For what reason? Is it a standard to not allow it, or does it break other things?

The line terminator is \n so if you have

#!/usr/bin/perl\r\n

Then the command to run is "/usr/bin/perl\r" - and \r is a valid file name component

#4 lore raw
Quoted 8 lines · Alan Cox
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

(\r\n), Linux 2.4.2 can't find an interpreter because it doesn't recognize the \r. The following patch should fix this (untested).

Fix the script. The kernel expects a specific format

For what reason? Is it a standard to not allow it, or does it break other things?

Quoted 6 lines

The line terminator is \n so if you have

#!/usr/bin/perl\r\n

Then the command to run is "/usr/bin/perl\r" - and \r is a valid file name component

Agreed. If you insist "fix" it with.....

cd /usr/bin
ln -s perl perl\r

Roger.

Signature · 4 lines
** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* There are old pilots, and there are bold pilots. 
* There are also old, bald pilots. 
#5 lore raw
In reply to Ivo Timmermans
Quoted 6 lines · Alan Cox
Quoted 4 lines

When running a script (perl in this case) that has DOS-style newlines (\r\n), Linux 2.4.2 can't find an interpreter because it doesn't recognize the \r. The following patch should fix this (untested).

Fix the script. The kernel expects a specific format

How about letting the kernel return ENOEXEC instead of ENOENT? It would give the luser just the little extra hint about converting their files to Unix format.

$ ls
testscript
$ head -1 testscript
#!/bin/sh
$ ./testscript
bash: ./testscript: No such file or directory

versus

$ ./testscript
bash: ./testscript: Exec format error

I haven't got a clue what Posix requires though.

Signature · 1 line
Erik Hensema ([email protected])
#6 lore raw
In reply to Ivo Timmermans

Hi!

Quoted 12 lines
Quoted 6 lines
Quoted 4 lines

When running a script (perl in this case) that has DOS-style newlines (\r\n), Linux 2.4.2 can't find an interpreter because it doesn't recognize the \r. The following patch should fix this (untested).

Fix the script. The kernel expects a specific format

How about letting the kernel return ENOEXEC instead of ENOENT? It would give the luser just the little extra hint about converting their files to Unix format.

Quoted 6 lines
$ ls
testscript
$ head -1 testscript
#!/bin/sh
$ ./testscript
bash: ./testscript: No such file or directory

What kernel wants to say is "/usr/bin/perl\r: no such file". Saying ENOEXEC would be even more confusing.

Signature · 2 lines
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.
#7 lore raw
In reply to Ivo Timmermans
Quoted 7 lines · Pavel Machek
Quoted 4 lines
$ head -1 testscript
#!/bin/sh
$ ./testscript
bash: ./testscript: No such file or directory

What kernel wants to say is "/usr/bin/perl\r: no such file". Saying ENOEXEC would be even more confusing.

So, why don't we make bash say that, then? As I guess that we've all been bitten by this before.

What are the chances for something like this to be included?

Greetings,
Jan.

execute_cmd.c renamed from bash-2.04/execute_cmd.c +42 -1
@@ -3035,6 +3035,42 @@
     }
 }
 
+/* Look for #!INTERPRETER in file COMMAND, and return INTERPRETER . */
+static char *
+extract_hash_bang_interpreter (char *command, char buf[80])
+{
+  int fd;
+  char *interpreter;
+
+  interpreter = "";
+  fd = open (command, O_RDONLY);
+  if (fd >= 0)
+    {
+      int len;
+	      
+      len = read (fd, (char *)buf, 80);
+      close (fd);
+	      
+      if (len > 0
+	  && buf[0] == '#' && buf[1] == '!')
+	{
+	  int i;
+	  int start;
+		  
+	  for (i = 2; whitespace (buf[i]) && i < len; i++)
+	    ;
+	  
+	  for (start = i;
+	       !whitespace (buf[i]) && buf[i] != '\n' && i < len;
+	       i++)
+	    ;
+
+	  interpreter = substring ((char *)buf, start, i);
+	}
+    }
+  return interpreter;
+}
+
 /* Execute a simple command that is hopefully defined in a disk file
    somewhere.
 
@@ -3155,7 +3191,12 @@
 
       if (command == 0)
 	{
-	  internal_error ("%s: command not found", pathname);
+	  char buf[80];
+	  char *interpreter = extract_hash_bang_interpreter (pathname, buf);
+	      
+	  internal_error ("%s: command not found: `%s'", pathname,
+			  interpreter);
+	  
 	  exit (EX_NOTFOUND);	/* Posix.2 says the exit status is 127 */
 	}
 
Signature · 2 lines
Jan Nieuwenhuizen <[email protected]> | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org
#8 lore raw
Quoted 10 lines · Jan Nieuwenhuizen
Quoted 7 lines · Pavel Machek
Quoted 4 lines
$ head -1 testscript
#!/bin/sh
$ ./testscript
bash: ./testscript: No such file or directory

What kernel wants to say is "/usr/bin/perl\r: no such file". Saying ENOEXEC would be even more confusing.

Quoted 8 lines

So, why don't we make bash say that, then? As I guess that we've all been bitten by this before.

What are the chances for something like this to be included?

Greetings,
Jan.

[SNIPPED...]

So why would you even consider breaking bash as a work-around for a broken script?

Somebody must have missed the boat entirely. Unix does not, never has, and never will end a text line with '\r'. It's Microsoft junk that does that, a throwback to CP/M, a throwback to MDS/200.

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of course you get it all back when you reboot..."; Actual explanation obtained from the Micro$oft help desk.

#9 lore raw

PATCH 1/1 : print missing interpreter name [Was: Re: binfmt_script and ^M]

Quoted 2 lines · Richard B. Johnson

So why would you even consider breaking bash as a work-around for a broken script?

I don't.

Quoted 3 lines

Somebody must have missed the boat entirely. Unix does not, never has, and never will end a text line with '\r'. It's Microsoft junk that does that, a throwback to CP/M, a throwback to MDS/200.

Yes, we all know that, but you missed the point. As far as this patch goes, it's got nothing to do with the '\r'. It's meant to get a more informative error message from bash, if ``#!INTERPRETER'' does not exist. Look:

    $ cat /bin/foo.sh
    #!/foo/bar/baz
    echo bar
    $ /bin/bash -c /bin/foo.sh 
    /bin/bash: /bin/foo.sh: No such file or directory
    $ ./build/bash -c /bin/foo.sh 
    ./build/bash: /foo/bar: No such file or directory

Maybe the message could even be better, but having /foo/bar' printed, ie, the file that the kernel says does not exist, iso /bin/foo.sh', the name of the script, that certainly does exist, may help. Possibly both should be printed.

Greetings,
Jan.

I made a silly mistake in previous patch, sorry.

ChangeLog renamed from bash-2.04.orig/ChangeLog +5 -0
@@ -0,0 +1,5 @@
+2001-03-05  Jan Nieuwenhuizen  <[email protected]>
+
+	* execute_cmd.c (extract_hash_bang_interpreter): New function.
+	(shell_execve): More informative error message.
+
execute_cmd.c renamed from bash-2.04.orig/execute_cmd.c +41 -0

53 lines not on this page · raw message

Signature · 2 lines
Jan Nieuwenhuizen <[email protected]> | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org
#10 lore raw

Re: [PATCH]: print missing interpreter name [Was: Re: binfmt_script and ^M]

Quoted 24 lines · Jan Nieuwenhuizen
Quoted 2 lines · Richard B. Johnson

So why would you even consider breaking bash as a work-around for a broken script?

I don't.

Quoted 3 lines

Somebody must have missed the boat entirely. Unix does not, never has, and never will end a text line with '\r'. It's Microsoft junk that does that, a throwback to CP/M, a throwback to MDS/200.

Yes, we all know that, but you missed the point. As far as this patch goes, it's got nothing to do with the '\r'. It's meant to get a more informative error message from bash, if ``#!INTERPRETER'' does not exist. Look:

    $ cat /bin/foo.sh
    #!/foo/bar/baz
    echo bar
    $ /bin/bash -c /bin/foo.sh 
    /bin/bash: /bin/foo.sh: No such file or directory
    $ ./build/bash -c /bin/foo.sh 
    ./build/bash: /foo/bar: No such file or directory
Quoted 7 lines

Maybe the message could even be better, but having /foo/bar' printed, ie, the file that the kernel says does not exist, iso /bin/foo.sh', the name of the script, that certainly does exist, may help. Possibly both should be printed.

Greetings,
Jan.

No. I did not miss the point. The 'No such file or directory' error (when you can see the ^$^$)#@@*& filename with 'ls'), usually means that there is something wrong with the file. Usually, I have found that it was an executable linked against some other runtime library than what I have. strace finds this quickly.

A common problem after a so-called upgrade. So, the bash error output (without additional text) is consistent when there is something wrong with the file-name as well.

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of course you get it all back when you reboot..."; Actual explanation obtained from the Micro$oft help desk.

#11 lore raw

PATCH 1/1 : print missing interpreter name [Was: Re: binfmt_script and ^M]

Quoted 3 lines · Richard B. Johnson

No. I did not miss the point. The 'No such file or directory' error (when you can see the ^$^$)#@@*& filename with 'ls'), usually means that there is something wrong with the file.

Now, let's see. When this error happens, it can be one of these:

1a) the script itself is broken (eg, it has `#!/bin/bash\r') 1b) the interpreter is missing, (eg, the script has
`#!/usr/bin/perl', while perl got removed during the grand perl repackaging)
2) the interpreter is broken, (eg, missing shared library)
3) the script does not exist

In any case, the heart of the problems 1) and 2) is expressed by the fact that there's something fishy with #!INTERPRETER. In the case of 1a, the script should be fixed, in case 1b, the system should be fixed, but this doesn't really matter. Either way: INTERPRETER is the file the kernel is talking about when it says ENOENT. Only in case 3), the ENOENT is about the script itself.

It seems reasonable, and helpful, for bash to print the name of the INTERPRETER.

Quoted 2 lines

Usually, I have found that it was an executable linked against some other runtime library than what I have. strace finds this quickly.

Well, this doesn't mean very much, other than you happen to experience
2) most (and you're fast at identifying the strange error). But even in this case, the name of the broken interpreter is what you should be warned about. Maybe others usually experience 1b), who cares?

In this third incarnation of my patch (today really isn't my day), you see these messages for 1-3:

1)

    $ cat /bin/foo.sh 
    #!/foo/bar/baz
    echo bar
    $ /bin/bash -c /bin/foo.sh 
    /bin/bash: /bin/foo.sh: No such file or directory
    $ ./new/bash -c /bin/foo.sh 
    ./new/bash: /bin/foo.sh: /foo/bar/baz: No such file or directory

2)

    $ cat /bin/no-ld.sh 
    #!/usr/bin/urg
    $ ldd /bin/urg                
	    libkpathsea.so.3 => not found
            [..]
    $ /bin/bash -c /bin/no-ld.sh 
    /bin/bash: /bin/no-ld.sh: No such file or directory
    $ ./new/bash -c /bin/no-ld.sh 
    ./new/bash: /bin/no-ld.sh: /usr/bin/urg: No such file or directory

3)

    $ /bin/bash -c /bin/no-such-foo.sh
    /bin/bash: /bin/no-such-foo.sh: No such file or directory
    $ ./new/bash -c /bin/no-such-foo.sh
    ./new/bash: /bin/no-such-foo.sh: No such file or directory

Ok, here goes try #3:

ChangeLog renamed from bash-2.04.orig/ChangeLog +5 -0
@@ -0,0 +1,5 @@
+2001-03-05  Jan Nieuwenhuizen  <[email protected]>
+
+	* execute_cmd.c (extract_hash_bang_interpreter): New function.
+	(shell_execve): More informative error message.
+
execute_cmd.c renamed from bash-2.04.orig/execute_cmd.c +47 -1

60 lines not on this page · raw message

Signature · 2 lines
Jan Nieuwenhuizen <[email protected]> | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org
#12 lore raw
In reply to Jan Nieuwenhuizen
Quoted 2 lines · Richard B. Johnson

Somebody must have missed the boat entirely. Unix does not, never has, and never will end a text line with '\r'.

Unix does not, never has, and never will end a text line with ' ' (a space character) or with \t (a tab character). Yet if I begin a shell script with '#!/bin/sh ' or '#!/bin/sh\t', the training white space is striped and /bin/sh gets exec'd. Since \r has no special significance to Unix, I'd expect it to be treated the same as any other whitespace character -- it should be striped, and /bin/sh should get exec'd.

Signature · 2 lines
John Kodis <[email protected]>
Phone: 301-286-7376
#13 lore raw
Quoted 11 lines · John Kodis
Quoted 2 lines · Richard B. Johnson

Somebody must have missed the boat entirely. Unix does not, never has, and never will end a text line with '\r'.

Unix does not, never has, and never will end a text line with ' ' (a space character) or with \t (a tab character). Yet if I begin a shell script with '#!/bin/sh ' or '#!/bin/sh\t', the training white space is striped and /bin/sh gets exec'd. Since \r has no special significance to Unix, I'd expect it to be treated the same as any other whitespace character -- it should be striped, and /bin/sh should get exec'd.

Makes sense, IMHO...

Rik
--
Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose...

		http://www.surriel.com/

http://www.conectiva.com/ http://distro.conectiva.com.br/

#14 lore raw
In reply to John Kodis
Quoted 5 lines · John Kodis
Quoted 2 lines · Richard B. Johnson

Somebody must have missed the boat entirely. Unix does not, never has, and never will end a text line with '\r'.

Quoted 7 lines

Unix does not, never has, and never will end a text line with ' ' (a space character) or with \t (a tab character). Yet if I begin a shell script with '#!/bin/sh ' or '#!/bin/sh\t', the training white space is striped and /bin/sh gets exec'd. Since \r has no special significance to Unix, I'd expect it to be treated the same as any other whitespace character -- it should be striped, and /bin/sh should get exec'd.

No. the '\n' character is interpreted. '\t' is expanded on stdout if the terminal is "cooked". Other interpreted characters are:

		^C, ^\, ^U, ^D, ^@, ^Q, ^S, ^Z, ^R, ^O, ^W, ^V

These are all upper-case ASCII letters codes minus 64. The erase (VERASE) is special and is '?' + 64

The '\r' (^R) definitely has special significance to Unix. It's called "VREPRINT", in the termios structure member "c_cc". If it exists in a file instead of an output stream, these characters are not interpreted. All files in Unix are binary. It's the 'C' runtime library that may interpret file contents for programmer convenience. For instance, fgets() reads until the new-line character. If you happen to have a '\r' before that new-line, guess what? You will have a '\r' in your string. If you were attempting to do:

ls foo\r

You will get a file-not found error unless you have a file with '\r' as its last character.

There is really no such thing as "whitespace" in Unix compatible text. For instance, the text in a Makefile MUST use the tab character as a separator. Spaces won't do.

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of course you get it all back when you reboot..."; Actual explanation obtained from the Micro$oft help desk.

#15 lore raw
In reply to John Kodis

Also sprach Rik van Riel

Quoted 13 lines · John Kodis
Quoted 11 lines · Richard B. Johnson
Quoted 2 lines

Somebody must have missed the boat entirely. Unix does not, never has, and never will end a text line with '\r'.

Unix does not, never has, and never will end a text line with ' ' (a space character) or with \t (a tab character). Yet if I begin a shell script with '#!/bin/sh ' or '#!/bin/sh\t', the training white space is striped and /bin/sh gets exec'd. Since \r has no special significance to Unix, I'd expect it to be treated the same as any other whitespace character -- it should be striped, and /bin/sh should get exec'd.

Quoted 1 line

Makes sense, IMHO...

That only makes sense if:
#!/bin/shasdf\n
would also exec /bin/sh.

" " and \t are whitespace, \r is not whitespace.

Signature · 3 lines
Jeff McAdams                            Email: [email protected]
Head Network Administrator              Voice: (502) 966-3848
IgLou Internet Services                        (800) 436-4456
#16 lore raw
In reply to John Kodis
Quoted 14 lines · Jeff Mcadams

Also sprach Rik van Riel

Quoted 12 lines · John Kodis
Quoted 11 lines · Richard B. Johnson
Quoted 2 lines

Somebody must have missed the boat entirely. Unix does not, never has, and never will end a text line with '\r'.

Unix does not, never has, and never will end a text line with ' ' (a space character) or with \t (a tab character). Yet if I begin a shell script with '#!/bin/sh ' or '#!/bin/sh\t', the training white space is striped and /bin/sh gets exec'd. Since \r has no special significance to Unix, I'd expect it to be treated the same as any other whitespace character -- it should be striped, and /bin/sh should get exec'd.

Quoted 5 lines
Quoted 1 line

Makes sense, IMHO...

That only makes sense if:
#!/bin/shasdf\n
would also exec /bin/sh.

POSIX disagrees with you (accd to the manual page)

$ man isspace
       ....
       isspace()
              checks for white-space characters.  In the "C"  and
              "POSIX"   locales,   these  are:  space,  form-feed
              ('\f'), newline  ('\n'),  carriage  return  ('\r'),
              horizontal tab ('\t'), and vertical tab ('\v').
#17 lore raw
Quoted 24 lines · Paul Flinders
Quoted 14 lines · Jeff Mcadams

Also sprach Rik van Riel

Quoted 12 lines · John Kodis
Quoted 11 lines · Richard B. Johnson
Quoted 2 lines

Somebody must have missed the boat entirely. Unix does not, never has, and never will end a text line with '\r'.

Unix does not, never has, and never will end a text line with ' ' (a space character) or with \t (a tab character). Yet if I begin a shell script with '#!/bin/sh ' or '#!/bin/sh\t', the training white space is striped and /bin/sh gets exec'd. Since \r has no special significance to Unix, I'd expect it to be treated the same as any other whitespace character -- it should be striped, and /bin/sh should get exec'd.

Quoted 5 lines
Quoted 1 line

Makes sense, IMHO...

That only makes sense if:
#!/bin/shasdf\n
would also exec /bin/sh.

POSIX disagrees with you (accd to the manual page)

Quoted 7 lines
$ man isspace
       ....
       isspace()
              checks for white-space characters.  In the "C"  and
              "POSIX"   locales,   these  are:  space,  form-feed
              ('\f'), newline  ('\n'),  carriage  return  ('\r'),
              horizontal tab ('\t'), and vertical tab ('\v').

And what does POSIX say about "#!/bin/sh\r" ?
In other words: should the kernel look for the interpreter between the ! and the newline, or [the first space or newline] or the first whitespace?

IMHO, the first whitespace. Which means that "#!/bin/sh\r" should invoke /bin/sh. (though it is junk).

#18 lore raw
In reply to Paul Flinders
Quoted 8 lines · Pozsar Balazs

And what does POSIX say about "#!/bin/sh\r" ?
In other words: should the kernel look for the interpreter between the ! and the newline, or [the first space or newline] or the first whitespace?

IMHO, the first whitespace. Which means that "#!/bin/sh\r" should invoke /bin/sh. (though it is junk).

The line terminator, '\n', is what terminates the interpreter. White space (in this case, only ' ' and '\t') is used to seperate the arguments to the interpreter. This allows scripts to pass args to intepreters, as in #!/usr/bin/per -w or #!/usr/bin/env perl -w

So is '\r' a line terminator? For Linux, no. Should '\r' seperate arguments? No, that would be very strange.

robert

#19 lore raw
Quoted 11 lines · Robert Read
Quoted 7 lines · Pozsar Balazs

And what does POSIX say about "#!/bin/sh\r" ?
In other words: should the kernel look for the interpreter between the ! and the newline, or [the first space or newline] or the first whitespace?

IMHO, the first whitespace. Which means that "#!/bin/sh\r" should invoke /bin/sh. (though it is junk).

The line terminator, '\n', is what terminates the interpreter. White space (in this case, only ' ' and '\t') is used to seperate the

        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 1 line

arguments to the interpreter.

The last little tiny thing that bothers me: why? Why only ' ' and '\t' _in this case_? As someone mentioned, even isspace() returns whitespace.

A possible answer (that i can think of), is that those ar the whitespaces, which are in IFS (as said previously), taking out us from kernel-space into userspace. But imho we shouldn't define another set whitespace for this case, can't we just use what isspace() says?

(okay, I'm not for this '\r' thingy, I just want to see the reasons.)

Signature · 1 line
Balazs Pozsar.
#20 lore raw
In reply to Robert Read
Quoted 16 lines · Pozsar Balazs
Quoted 11 lines · Robert Read
Quoted 7 lines · Pozsar Balazs

And what does POSIX say about "#!/bin/sh\r" ?
In other words: should the kernel look for the interpreter between the ! and the newline, or [the first space or newline] or the first whitespace?

IMHO, the first whitespace. Which means that "#!/bin/sh\r" should invoke /bin/sh. (though it is junk).

The line terminator, '\n', is what terminates the interpreter. White space (in this case, only ' ' and '\t') is used to seperate the

        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 1 line

arguments to the interpreter.

Quoted 7 lines

The last little tiny thing that bothers me: why? Why only ' ' and '\t' _in this case_? As someone mentioned, even isspace() returns whitespace.

A possible answer (that i can think of), is that those ar the whitespaces, which are in IFS (as said previously), taking out us from kernel-space into userspace. But imho we shouldn't define another set whitespace for this case, can't we just use what isspace() says?

And isspace('\n') is also true. At question here is not the definition of whitespace. The question is, what is the definition of a command line? What characters are valid command line seperators?

robert

#21 lore raw
Quoted 4 lines · Robert Read

And isspace('\n') is also true. At question here is not the definition of whitespace. The question is, what is the definition of a command line? What characters are valid command line seperators?

It doesn't seem likely that '\r' is going to be accepted into the general kernel. I personally don't think the issue affects enough *nix systems for this to be a big issue.

I used to work at an ISP where I maintained a Slowaris box with about 600 websites on it; this issue came up frequently with customers uploading scripts in binary mode trying to run #!/usr/bin/perl^M. The solution for me was to just do the following:

cd /usr/bin
sudo ln -s perl^V^M perl

and it effectively solved the issue. I havn't looked at slowaris 8, but slowaris 7 still refuses to run #!/usr/bin/perl^M. Why not just use a simple one-time solution that will solve the problem & is portable to other OS's?

Jeff
--
Resisting temptation is easier when you think you'll probably get another chance later on.

#22 lore raw
In reply to Robert Read

[Jeff Coy]

Quoted 6 lines

this issue came up frequently with customers uploading scripts in binary mode trying to run #!/usr/bin/perl^M. The solution for me was to just do the following:

cd /usr/bin
sudo ln -s perl^V^M perl

So none of your customers tried '#!/usr/bin/perl -w^M'? (Come on, doesn't everyone use -w?)

I'm not for treating \r as IFS in the kernel, but the "simple one-time" solution is not perfect..

Peter

#23 lore raw
Quoted 6 lines · Peter Samuelson

[Jeff Coy]

Quoted 4 lines

this issue came up frequently with customers uploading scripts in binary mode trying to run #!/usr/bin/perl^M. The solution for me was to just do the following:

Quoted 9 lines
Quoted 2 lines

cd /usr/bin
sudo ln -s perl^V^M perl

So none of your customers tried '#!/usr/bin/perl -w^M'? (Come on, doesn't everyone use -w?)

I'm not for treating \r as IFS in the kernel, but the "simple one-time" solution is not perfect..

'#!/usr/bin/perl -w^M' works without any special handling; the link is not needed:

	11:15:52 jcoy@d-hopper::~
	$ cat -vet foo.pl
	#!/usr/bin/perl -w^M$
	^M$
	print "Hello, World!\n";^M$
	11:16:52 jcoy@d-hopper::~
	$ ./foo.pl
	Hello, World!

Jeff
--
The Harvard Law states: Under controlled conditions of light, temperature, humidity, and nutrition, the organism will do as it damn well pleases.
-- Larry Wall

#24 lore raw
In reply to Peter Samuelson
Quoted 2 lines · Jeff Coy

'#!/usr/bin/perl -w^M' works without any special handling; the link is not needed:

This is the main reason that I think that the kernel should treat \r as just another whitespace character: it's what most shells do, it's what most users expect, and it's the least surprising behavior.

Signature · 2 lines
John Kodis <[email protected]>
Phone: 301-286-7376
#25 lore raw
In reply to Peter Samuelson

John Kodis <[email protected]> writes:

|> On Tue, Mar 06, 2001 at 11:36:29AM -0700, Jeff Coy wrote:
|> 
|> > '#!/usr/bin/perl -w^M' works without any special handling; the link is
|> > not needed:
|> 
|> This is the main reason that I think that the kernel should treat \r
|> as just another whitespace character: it's what most shells do

Do they? Bourne shells don't, tcsh doesn't, zsh doesn't.

Andreas.

Signature · 5 lines
Andreas Schwab                                  "And now for something
SuSE Labs                                        completely different."
[email protected]
SuSE GmbH, Schanzäckerstr. 10, D-90443 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
#26 lore raw
In reply to Robert Read
Quoted 10 lines · Robert Read
Quoted 8 lines · Pozsar Balazs

And what does POSIX say about "#!/bin/sh\r" ?
In other words: should the kernel look for the interpreter between the ! and the newline, or [the first space or newline] or the first whitespace?

IMHO, the first whitespace. Which means that "#!/bin/sh\r" should invoke /bin/sh. (though it is junk).

Quoted 8 lines

The line terminator, '\n', is what terminates the interpreter. White space (in this case, only ' ' and '\t') is used to seperate the arguments to the interpreter. This allows scripts to pass args to intepreters, as in #!/usr/bin/per -w or #!/usr/bin/env perl -w

So is '\r' a line terminator? For Linux, no. Should '\r' seperate arguments? No, that would be very strange.

For research, I suggest a look at getopt(3) or whatever it's called. The command line args are seperated into chunks based upon what got seperated into argv[1]....[n], delimited by (hold my breath) white-space.

Of course, it's not getopt(), but "makeopt()" that we are looking for. ...Whatever chopped up those command-line arguments in the first place.

If __that__ corresponds so POSIX rules, then whatever follows must also comply.

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of course you get it all back when you reboot..."; Actual explanation obtained from the Micro$oft help desk.

#27 lore raw
In reply to John Kodis
Followup to:  <[email protected]>
By author:    "Richard B. Johnson" <[email protected]>
In newsgroup: linux.dev.kernel
Quoted 4 lines

The '\r' (^R) definitely has special significance to Unix. It's called "VREPRINT", in the termios structure member "c_cc".

'\r' is ^M, not ^R.

Quoted 3 lines

There is really no such thing as "whitespace" in Unix compatible text. For instance, the text in a Makefile MUST use the tab character as a separator. Spaces won't do.

Whitespace is defined by POSIX as '\n', '\r', '\t', '\v', '\f' or ' '. Occationally, the specific *kind* of whitespace matters -- for example, '\n' frequently have different behaviour; as does '\t' in make.

-hpa

Signature · 3 lines
<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
#28 lore raw
In reply to John Kodis

Paul Flinders <[email protected]> writes:

|> Jeff Mcadams wrote:
|> 
|> > Also sprach Rik van Riel
|> > >On Mon, 5 Mar 2001, John Kodis wrote:
|> > >> On Mon, Mar 05, 2001 at 08:40:22AM -0500, Richard B. Johnson wrote:
|> > >> > Somebody must have missed the boat entirely. Unix does not, never
|> > >> > has, and never will end a text line with '\r'.
|> >
|> > >> Unix does not, never has, and never will end a text line with ' ' (a
|> > >> space character) or with \t (a tab character).  Yet if I begin a
|> > >> shell script with '#!/bin/sh ' or '#!/bin/sh\t', the training white
|> > >> space is striped and /bin/sh gets exec'd.  Since \r has no special
|> > >> significance to Unix, I'd expect it to be treated the same as any
|> > >> other whitespace character -- it should be striped, and /bin/sh
|> > >> should get exec'd.
|> >
|> > >Makes sense, IMHO...
|> >
|> > That only makes sense if:
|> > #!/bin/shasdf\n
|> > would also exec /bin/sh.
|> 
|> POSIX disagrees with you (accd to the manual page)
|> 
|> $ man isspace

This has no significance here. The right thing to look at is $IFS, which does not contain \r by default. The shell only splits words by "IFS whitespace", and the kernel should be consistent with it:

$ echo -e 'ls foo\r' | sh
ls: foo: No such file or directory

Andreas.

Signature · 5 lines
Andreas Schwab                                  "And now for something
SuSE Labs                                        completely different."
[email protected]
SuSE GmbH, Schanzäckerstr. 10, D-90443 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
#29 lore raw
In reply to John Kodis
Quoted 8 lines · Andreas Schwab

This [isspace('\r') == 1] has no significance here. The right thing to

look at is $IFS, which does not contain \r by default. The shell only splits

words by "IFS whitespace", and the kernel should be consistent with it:

$ echo -e 'ls foo\r' | sh
ls: foo: No such file or directory

The problem with that argument is that #!<interpreter> can be applied to more than just shells which understand $IFS, so which environment variable does the kernel pick?

It's a difficult one - logically white space should terminate the interpreter name but the definition of what is, or isn't, white space is quite definately a user space issue. Unfortunately if you do use the user's locale to decide you then open the possibility that whether a scipt works or not depends on the locale - and that, surely, is equally unacceptable and deferring to a user-space "script launcher" app is going to open yet more problems.

In the end I suspect that the only practical way out _is_ to say that the kernel uses space (0x20) and tab (0x8) as white space and no other character.

This does miss the point though - whatever the rules are used to parse the interpreter name it is still confusing when the error reported by the shell is "No such file or directory". Especially when the file is sitting in front of you. Would it be so bad to add an ENOINTERP error?

#30 lore raw
In reply to John Kodis
Quoted 2 lines · Paul Flinders

uses space (0x20) and tab (0x8) as white space and no other character.

<sigh> I mean, of course, tab (_0x9_)

I just checked - the kernel isspace() macro says that \r is whitespace.

#31 lore raw
In reply to John Kodis
Quoted 11 lines · John Kodis
Quoted 2 lines · Richard B. Johnson

Somebody must have missed the boat entirely. Unix does not, never has, and never will end a text line with '\r'.

Unix does not, never has, and never will end a text line with ' ' (a space character) or with \t (a tab character). Yet if I begin a shell script with '#!/bin/sh ' or '#!/bin/sh\t', the training white space is striped and /bin/sh gets exec'd. Since \r has no special significance to Unix, I'd expect it to be treated the same as any other whitespace character -- it should be striped, and /bin/sh should get exec'd.

umm, last i checked a carriage return wasn't whitespace... space, horizontal tab, vertical tab, form feed constitute whitespace IIRC...

Kelsey Hudson [email protected]
Software Engineer
Compendium Technologies, Inc (619) 725-0771
---------------------------------------------------------------------------

#32 lore raw
In reply to John Kodis

Paul Flinders <[email protected]> writes:

|> Andreas Schwab wrote:
|> 
|> > This [isspace('\r') == 1] has no significance here.  The right thing to
|> 
|> > look at is $IFS, which does not contain \r by default.  The shell only splits
|> 
|> > words by "IFS whitespace", and the kernel should be consistent with it:
|> >
|> > $ echo -e 'ls foo\r' | sh
|> > ls: foo: No such file or directory
|> 
|> The problem with that argument is that #!<interpreter> can be applied
|> to more than just shells which understand $IFS, so which environment
|> variable does the kernel pick?

The kernel should use the same default value of IFS as the Bourne shell, ie. the same value you'll get with /bin/sh -c 'echo "$IFS"'. This is independent of any settings in the environment.

|> It's a difficult one - logically white space should terminate the interpreter

No, IFS-whitespace delimits arguments in the Bourne shell.

Andreas.

Signature · 5 lines
Andreas Schwab                                  "And now for something
SuSE Labs                                        completely different."
[email protected]
SuSE GmbH, Schanzäckerstr. 10, D-90443 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
#33 lore raw
In reply to John Kodis
Quoted 17 lines · Andreas Schwab

Paul Flinders <[email protected]> writes:

|> Andreas Schwab wrote:
|>
|> > This [isspace('\r') == 1] has no significance here.  The right thing to
|>
|> > look at is $IFS, which does not contain \r by default.  The shell only splits
|>
|> > words by "IFS whitespace", and the kernel should be consistent with it:
|> >
|> > $ echo -e 'ls foo\r' | sh
|> > ls: foo: No such file or directory
|>
|> The problem with that argument is that #!<interpreter> can be applied
|> to more than just shells which understand $IFS, so which environment
|> variable does the kernel pick?
Quoted 7 lines

The kernel should use the same default value of IFS as the Bourne shell, ie. the same value you'll get with /bin/sh -c 'echo "$IFS"'. This is independent of any settings in the environment.

|> It's a difficult one - logically white space should terminate the interpreter

No, IFS-whitespace delimits arguments in the Bourne shell.

Way back whenever processing #! was moved from the shell to the kernel** this argument would have made sense - today I'm not so sure.

But I'm quite happy for the kernel to use just space and tab if it wishes, or anything else for that matter but it _is_ confusing that the error code doesn't distinguish problems with the script from problems with the interpreter.

**Did linux ever rely on the shell for this?

#34 lore raw
Quoted 33 lines
Quoted 4 lines · Andreas Schwab

Paul Flinders <[email protected]> writes:

|> Andreas Schwab wrote:
|>
|> > This [isspace('\r') == 1] has no significance here.  The

right thing to

Quoted 2 lines

|>
|> > look at is $IFS, which does not contain \r by default.

The shell only splits

Quoted 2 lines

|>
|> > words by "IFS whitespace", and the kernel should be

consistent with it:

Quoted 8 lines
|> >
|> > $ echo -e 'ls foo\r' | sh
|> > ls: foo: No such file or directory
|>
|> The problem with that argument is that #!<interpreter> can be applied
|> to more than just shells which understand $IFS, so which environment
|> variable does the kernel pick?
Quoted 5 lines

The kernel should use the same default value of IFS as the Bourne shell, ie. the same value you'll get with /bin/sh -c 'echo "$IFS"'. This is independent of any settings in the environment.

|> It's a difficult one - logically white space should

terminate the interpreter

Quoted 2 lines

No, IFS-whitespace delimits arguments in the Bourne shell.

Way back whenever processing #! was moved from the shell to the kernel** this argument would have made sense - today I'm not so sure.

Quoted 6 lines

But I'm quite happy for the kernel to use just space and tab if it wishes, or anything else for that matter but it _is_ confusing that the error code doesn't distinguish problems with the script from problems with the interpreter.

**Did linux ever rely on the shell for this?

Maybe the correct answer would be to create a proc entry for this. That allow the user to decide what is whitespace on his machine, since nobody here appears to agree.

User: hmm... Wonder what happes if i do the following

       %cat '$#! \n\t\r' > /proc/whitespace

later, % config.sh : Error file not found.
Oops, bug report... ;-)

Laramie

#35 lore raw
In reply to Paul Flinders

I propose /proc/sys/kernel/im_too_lame_to_learn_how_to_use_the_most_basic_of_unix_tools_so_i_want_the_kernel_to_be_filled_with_crap_to_disguise_my_ineptitude

Any support?

Sean

Quoted 56 lines · Laramie Leavitt
Quoted 33 lines
Quoted 4 lines · Andreas Schwab

Paul Flinders <[email protected]> writes:

|> Andreas Schwab wrote:
|>
|> > This [isspace('\r') == 1] has no significance here.  The

right thing to

Quoted 2 lines

|>
|> > look at is $IFS, which does not contain \r by default.

The shell only splits

Quoted 2 lines

|>
|> > words by "IFS whitespace", and the kernel should be

consistent with it:

Quoted 8 lines
|> >
|> > $ echo -e 'ls foo\r' | sh
|> > ls: foo: No such file or directory
|>
|> The problem with that argument is that #!<interpreter> can be applied
|> to more than just shells which understand $IFS, so which environment
|> variable does the kernel pick?
Quoted 5 lines

The kernel should use the same default value of IFS as the Bourne shell, ie. the same value you'll get with /bin/sh -c 'echo "$IFS"'. This is independent of any settings in the environment.

|> It's a difficult one - logically white space should

terminate the interpreter

Quoted 2 lines

No, IFS-whitespace delimits arguments in the Bourne shell.

Way back whenever processing #! was moved from the shell to the kernel** this argument would have made sense - today I'm not so sure.

Quoted 6 lines

But I'm quite happy for the kernel to use just space and tab if it wishes, or anything else for that matter but it _is_ confusing that the error code doesn't distinguish problems with the script from problems with the interpreter.

**Did linux ever rely on the shell for this?

Maybe the correct answer would be to create a proc entry for this. That allow the user to decide what is whitespace on his machine, since nobody here appears to agree.

User: hmm... Wonder what happes if i do the following

       %cat '$#! \n\t\r' > /proc/whitespace

later, % config.sh : Error file not found.
Oops, bug report... ;-)

Laramie

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#36 lore raw
Quoted 5 lines · Sean Hunter

I propose /proc/sys/kernel/im_too_lame_to_learn_how_to_use_the_most_basic_of_unix_tools_so_i_want_the_kernel_to_be_filled_with_crap_to_disguise_my_ineptitude

Any support?

Hrm - make it part of the "fscking_moron" subsystem.

/proc/sys/kernel/fscking_moron/stupidity_workarounds/cant_handle_ascii/broken_shellscript_hack

It would be nice if the shell's error message gave a bit more information on the problem, but that's a userspace issue IMHO: make *sh check for this kind of mistake and say "interpreter for shell script not found", something like that. Whatever, it's NOT a kernel issue.

Maybe the kernel could return a different error value, though - just saying "file not found" isn't very helpful...

James.

#37 lore raw
In reply to Sean Hunter

Wouldn't it be easier to run the script interpreter through WINE ? This way we could workaround several Win32 peculiarities, and users wouldn't bother taking special steps when coding on their home PC.

Xav

Quoted 71 lines · Sean Hunter

I propose /proc/sys/kernel/im_too_lame_to_learn_how_to_use_the_most_basic_of_unix_tools_so_i_want_the_kernel_to_be_filled_with_crap_to_disguise_my_ineptitude

Any support?

Sean

Quoted 49 lines · Laramie Leavitt
Quoted 33 lines
Quoted 4 lines · Andreas Schwab

Paul Flinders <[email protected]> writes:

|> Andreas Schwab wrote:
|>
|> > This [isspace('\r') == 1] has no significance here.  The

right thing to

Quoted 2 lines

|>
|> > look at is $IFS, which does not contain \r by default.

The shell only splits

Quoted 2 lines

|>
|> > words by "IFS whitespace", and the kernel should be

consistent with it:

Quoted 8 lines
|> >
|> > $ echo -e 'ls foo\r' | sh
|> > ls: foo: No such file or directory
|>
|> The problem with that argument is that #!<interpreter> can be applied
|> to more than just shells which understand $IFS, so which environment
|> variable does the kernel pick?
Quoted 5 lines

The kernel should use the same default value of IFS as the Bourne shell, ie. the same value you'll get with /bin/sh -c 'echo "$IFS"'. This is independent of any settings in the environment.

|> It's a difficult one - logically white space should

terminate the interpreter

Quoted 2 lines

No, IFS-whitespace delimits arguments in the Bourne shell.

Way back whenever processing #! was moved from the shell to the kernel** this argument would have made sense - today I'm not so sure.

Quoted 6 lines

But I'm quite happy for the kernel to use just space and tab if it wishes, or anything else for that matter but it _is_ confusing that the error code doesn't distinguish problems with the script from problems with the interpreter.

**Did linux ever rely on the shell for this?

Maybe the correct answer would be to create a proc entry for this. That allow the user to decide what is whitespace on his machine, since nobody here appears to agree.

User: hmm... Wonder what happes if i do the following

       %cat '$#! \n\t\r' > /proc/whitespace

later, % config.sh : Error file not found.
Oops, bug report... ;-)

Quoted 7 lines

Laramie

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#38 lore raw
In reply to Paul Flinders
Quoted 5 lines · Sean Hunter

I propose /proc/sys/kernel/im_too_lame_to_learn_how_to_use_the_most_basic_of_unix_tools_so_i_want_the_kernel_to_be_filled_with_crap_to_disguise_my_ineptitude

Any support?

<sarcasm>
Hey, let's go even further! Let's add support in all programs for \r\n. And why not make all program use filenames that have an 8+3 char garbled equivalent where the last 3 are the indicators of the filetype. Oh, and let's do everything to make sure the user doesn't leave Gnome/KDE. And of course, let's add new features to all existing protocols and other standards to make them "superior" to other implementations. Oh, and of course, we must require an extra 64 MB of memory and 500 MB of diskspace for each release, and a 200MHz faster processor. And let us do all system settings through a registry.

OH! Let's change the name of the operating-system to something more catchy. Hmmm. Let's see. Windows maybe...
</sarcasm>

/David
_ _
// David Weinehall <[email protected]> /> Northern lights wander \\

//  Project MCA Linux hacker        //  Dance across the winter sky //
\>  http://www.acc.umu.se/~tao/    </   Full colour fire           </
#39 lore raw
In reply to Paul Flinders

----- Original Message -----
From: "David Weinehall" <[email protected]>
To: "Sean Hunter" <[email protected]>; "Laramie Leavitt" <[email protected]>; <[email protected]>
Sent: Tuesday, 6. March 2001 15:37
Subject: Re: binfmt_script and ^M

Quoted 9 lines · Sean Hunter
Quoted 5 lines

I propose /proc/sys/kernel/im_too_lame_to_learn_how_to_use_the_most_basic_of_unix_tools_so_i_want_the_kernel_to_be_filled_with_crap_to_disguise_my_ineptitude

Any support?

<sarcasm>
Hey, let's go even further! Let's add support in all programs for \r\n.

That is no sarcasm, it is ridiculous. CRLF line endings are ISO-IR-6 and US-ASCII standard, and even UN*X systems used them when they had printers (typewriters?) as output device, and no screens. With the Virtual Terminal, Virtual Console stuff times may have changed but we have so many old stuff in it... I won't remove them or didn't think of, but I remember you of:
- lost+found
- using ESC (or Alt???) as META for _shell commands_ which easily could be Ctrl-Left, Ctrl-Right, Ctrl-Del etc.
- EMACS :-((
- ED/EX/VI :-(

The following does _not_ have to do with any US-ASCII or ISO_646.irv:1991 standards which IIRC are inherited by POSIX.

Quoted 9 lines

And why not make all program use filenames that have an 8+3 char garbled equivalent where the last 3 are the indicators of the filetype. Oh, and let's do everything to make sure the user doesn't leave Gnome/KDE. And of course, let's add new features to all existing protocols and other standards to make them "superior" to other implementations. Oh, and of course, we must require an extra 64 MB of memory and 500 MB of diskspace for each release, and a 200MHz faster processor. And let us do all system settings through a registry.

Quoted 6 lines

OH! Let's change the name of the operating-system to something more catchy. Hmmm. Let's see. Windows maybe...
</sarcasm>

/David

I _do_ _not_ like Windoze either, but we live in a world where we have to cope with it. I am even to code windoze apps in order to support linux (no comment on this)...

-mirabilos

#40 lore raw
In reply to Paul Flinders
Quoted 2 lines · Sean Hunter

I propose /proc/sys/kernel/im_too_lame_to_learn_how_to_use_the_most_basic_of_unix_tools_so_i_want_the_kernel_to_be_filled_with_crap_to_disguise_my_ineptitude

Well, too me it seems that you are intolerant.

I think that it should not be added to kernel because:

#!/bin/sh
/usr/bin/perl^M

will write
--
: No such file or directory
--
(in real it writes '/usr/bin/perl<CR>: No such file or directory')

But what I do think is that more meaningful message should be printed to output, because it is not only ^M issue. You could mistype name of interpreter and don't notice (/usr/bin/eprl or similar typos), and printing message saying 'script.pl: file not found' is confusing.

I see problem somewhere else. There are editors and viewers (for example midnight commander) which will hide ^M from you, and left you totally confused (that's why I am using emacs ;-), because you have no idea why it doesn't work, because everything seems ok with this _broken_ behaviour. This is really BAD thing.

And even more BAD thing is the intolerance shown in this thread. People are not morons just because they don't understand confusing message which shell gives them. Behaviour of kernel is good, error message is wrong. What should be fixed is error message.

Signature · 4 lines
Ondřej Surý <[email protected]>         Globe Internet s.r.o. http://globe.cz/
Tel: +420235365000   Fax: +420235365009         Pláničkova 1, 162 00 Praha 6
Mob: +420605204544   ICQ: 24944126             Mapa: http://globe.namape.cz/
GPG fingerprint:          CC91 8F02 8CDE 911A 933F  AE52 F4E6 6A7C C20D F273
#41 lore raw
In reply to Paul Flinders

----- Original Message -----
From: "Sean Hunter" <[email protected]>
To: "Thorsten Glaser Geuer" <[email protected]>
Sent: Thursday, 8. March 2001 13:01
Subject: Re: binfmt_script and ^M

Quoted 30 lines · Thorsten Glaser Geuer
Quoted 17 lines

----- Original Message -----
From: "David Weinehall" <[email protected]>
To: "Sean Hunter" <[email protected]>; "Laramie Leavitt" <[email protected]>; <[email protected]>
Sent: Tuesday, 6. March 2001 15:37
Subject: Re: binfmt_script and ^M

Quoted 9 lines · Sean Hunter
Quoted 5 lines

I propose /proc/sys/kernel/im_too_lame_to_learn_how_to_use_the_most_basic_of_unix_tools_so_i_want_the_kernel_to_be_filled_with_crap_to_disguise_my_ineptitude

Any support?

<sarcasm>
Hey, let's go even further! Let's add support in all programs for \r\n.

Quoted 11 lines

That is no sarcasm, it is ridiculous. CRLF line endings are ISO-IR-6 and US-ASCII standard, and even UN*X systems used them when they had printers (typewriters?) as output device, and no screens. With the Virtual Terminal, Virtual Console stuff times may have changed but we have so many old stuff in it... I won't remove them or didn't think of, but I remember you of:
- lost+found
- using ESC (or Alt???) as META for _shell commands_ which easily could be Ctrl-Left, Ctrl-Right, Ctrl-Del etc.
- EMACS :-((
- ED/EX/VI :-(

Quoted 2 lines

This is pure bullshit, so I'm not copying my response to the list. None of that depends on the kernel.

I did not ever say that it depends on the kernel. The kernel _has to_ depend on _standards_ coz otherwise it wouldn't be POSIX compliant, it wouldn't obey one of the most basic ISO standards nowadays, etc. etc.
The kernel is faulty, and it has to use &h0D as CARRIAGE RETURN, whenever this behaviour seems expactable (stress on -able).

Quoted 12 lines

If he's to lame to do:

for i in find . -type f | grep -l "#! .*perl\r"; do perl -pie 's|\r||g' < $i
done

...or...

ln -snf which perl{,\r}

...or...

Quoted 8 lines

change his scripts to use "perl -w", which works, and he should use anyway

...or...

Run his scripts directly by doing "perl /the/path/to/broken/script" (which also works)

...or any of the other solutions that various people suggested

_I_ am not talking about perl coz in my eyes this is waste. But there are numerous other programmes with problems on it, too.

Quoted 4 lines

...then why should we accept a kernel patch to fix this non-problem? This is just plain ludicrous.

Quoted 25 lines

Sean

Quoted 23 lines

The following does _not_ have to do with any US-ASCII or ISO_646.irv:1991 standards which IIRC are inherited by POSIX.

Quoted 9 lines

And why not make all program use filenames that have an 8+3 char garbled equivalent where the last 3 are the indicators of the filetype. Oh, and let's do everything to make sure the user doesn't leave Gnome/KDE. And of course, let's add new features to all existing protocols and other standards to make them "superior" to other implementations. Oh, and of course, we must require an extra 64 MB of memory and 500 MB of diskspace for each release, and a 200MHz faster processor. And let us do all system settings through a registry.

Quoted 6 lines

OH! Let's change the name of the operating-system to something more catchy. Hmmm. Let's see. Windows maybe...
</sarcasm>

/David

I _do_ _not_ like Windoze either, but we live in a world where we have to cope with it. I am even to code windoze apps in order to support linux (no comment on this)...

-mirabilos

#42 lore raw

Re: binfmt_script and ^M (historical note)

In reply to John Kodis

Paul-

Minor historical note. The #!' processing was never done by the shell, this was always done in the kernel. Think about about it, the #' character denotes a comment line, the shell ignores that line. #!' was used to create a way for the kernel to execute a shell script directly. Since the kernel recognized the type of executable based on a 16-bit magic number #!' became a new magic number that meant "break the remainder of the line into program' and args' and then execute program' with args'".

A nice side effect of this is that it became a way to create shell scripts that worked no matter what shell a user was running. For efficiency, most shells just read and execute a shell script so releasing a Bourne shell script to a group of CSH users created problems. At the expense of a fork' and exec' the `#!' magic number solved this problem.

Quoted 43 lines · Paul Flinders
Quoted 17 lines · Andreas Schwab

Paul Flinders <[email protected]> writes:

|> Andreas Schwab wrote:
|>
|> > This [isspace('\r') == 1] has no significance here.  The right thing to
|>
|> > look at is $IFS, which does not contain \r by default.  The shell only splits
|>
|> > words by "IFS whitespace", and the kernel should be consistent with it:
|> >
|> > $ echo -e 'ls foo\r' | sh
|> > ls: foo: No such file or directory
|>
|> The problem with that argument is that #!<interpreter> can be applied
|> to more than just shells which understand $IFS, so which environment
|> variable does the kernel pick?
Quoted 7 lines

The kernel should use the same default value of IFS as the Bourne shell, ie. the same value you'll get with /bin/sh -c 'echo "$IFS"'. This is independent of any settings in the environment.

|> It's a difficult one - logically white space should terminate the interpreter

No, IFS-whitespace delimits arguments in the Bourne shell.

Way back whenever processing #! was moved from the shell to the kernel** this argument would have made sense - today I'm not so sure.

But I'm quite happy for the kernel to use just space and tab if it wishes, or anything else for that matter but it _is_ confusing that the error code doesn't distinguish problems with the script from problems with the interpreter.

**Did linux ever rely on the shell for this?

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Signature · 4 lines
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
[email protected]
Ph: 303/938-9838
#43 lore raw
In reply to John Kodis

[Dr. Kelsey Hudson]

Quoted 2 lines

umm, last i checked a carriage return wasn't whitespace... space, horizontal tab, vertical tab, form feed constitute whitespace IIRC...

Where and when did you check? Several sources disagree with you.

Peter

#44 lore raw
Quoted 5 lines · Peter Samuelson

[Dr. Kelsey Hudson]

Quoted 2 lines

umm, last i checked a carriage return wasn't whitespace... space, horizontal tab, vertical tab, form feed constitute whitespace IIRC...

Where and when did you check? Several sources disagree with you.

a long while ago... i should have checked before i opened my mouth :p

Kelsey Hudson [email protected]
Software Engineer
Compendium Technologies, Inc (619) 725-0771
---------------------------------------------------------------------------

#45 lore raw
In reply to Jan Nieuwenhuizen
Quoted 14 lines · Richard B. Johnson
Quoted 11 lines · Jan Nieuwenhuizen
Quoted 6 lines · Pavel Machek
Quoted 3 lines
$ head -1 testscript
#!/bin/sh
$ ./testscript bash: ./testscript: No such file or directory

What kernel wants to say is "/usr/bin/perl\r: no such file". Saying ENOEXEC would be even more confusing.

So, why don't we make bash say that, then? As I guess that we've all been bitten by this before.

Quoted 8 lines
Quoted 3 lines

What are the chances for something like this to be included?

Greetings, Jan.

[SNIPPED...]

So why would you even consider breaking bash as a work-around for a broken script?

Userfriendlyness.

Quoted 3 lines

Somebody must have missed the boat entirely. Unix does not, never has, and never will end a text line with '\r'. It's Microsoft junk that does that, a throwback to CP/M, a throwback to MDS/200.

Yes, _we_ all know that. However, it's not really intuitive to the user getting a 'No such file or directory' on a script he just created. Bash doesn't say:
bash: testscript: Script interpreter not found
but bash says:
bash: testscript: No such file or directory

Maybe we should create a new errno: EINTERPRETER or something like that and let the kernel return that instead of ENOENT.

Note that this has little to do with the \r\n problem but only with the _real_ underlying reason: the script interpreter is not found and ENOENT is returned confusing the user: the user thinks the _script_ is not found, while its there, for sure.

Signature · 1 line
Erik Hensema ([email protected])
#46 lore raw
In reply to Jan Nieuwenhuizen

Hi!

Quoted 11 lines
Quoted 3 lines

Somebody must have missed the boat entirely. Unix does not, never has, and never will end a text line with '\r'. It's Microsoft junk that does that, a throwback to CP/M, a throwback to MDS/200.

Yes, _we_ all know that. However, it's not really intuitive to the user getting a 'No such file or directory' on a script he just created. Bash doesn't say:
bash: testscript: Script interpreter not found
but bash says:
bash: testscript: No such file or directory

Quoted 2 lines

Maybe we should create a new errno: EINTERPRETER or something like that and let the kernel return that instead of ENOENT.

Agreen, EINTEPRETTER would be very nice, plus maybe EDYNLINKER. We already have 'level 3 stopped', so this should not hurt :-)).
Pavel

Signature · 2 lines
I'm [email protected]. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at [email protected]
#47 lore raw
In reply to Ivo Timmermans

Jan Nieuwenhuizen <[email protected]> writes:

|> Pavel Machek <[email protected]> writes:
|> 
|> > > $ head -1 testscript
|> > > #!/bin/sh
|> > > $ ./testscript
|> > > bash: ./testscript: No such file or directory
                           ^^^^^^^^^^^^^^^^^^^^^^^^^
|> > 
|> > What kernel wants to say is "/usr/bin/perl\r: no such file". Saying ENOEXEC
|> > would be even more confusing.
|> 
|> So, why don't we make bash say that, then?  As I guess that we've all
|> been bitten by this before.
|> 
|> What are the chances for something like this to be included?

Very low, because it would not change anything.

|> @@ -3155,7 +3191,12 @@
|>  
|>        if (command == 0)
|>  	{
|> -	  internal_error ("%s: command not found", pathname);
                               ^^^^^^^^^^^^^^^^^
|> +	  char buf[80];
|> +	  char *interpreter = extract_hash_bang_interpreter (pathname, buf);
|> +	      
|> +	  internal_error ("%s: command not found: `%s'", pathname,
|> +			  interpreter);
|> +	  
|>  	  exit (EX_NOTFOUND);	/* Posix.2 says the exit status is 127 */
|>  	}

Andreas.

Signature · 5 lines
Andreas Schwab                                  "And now for something
SuSE Labs                                        completely different."
[email protected]
SuSE GmbH, Schanzäckerstr. 10, D-90443 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5