Skip to contentSkip to conversation

tools: python portability fixes

PATCH In review Christoph Egger 4 participants 11 messages Jul 23, 2009 – Aug 4, 2009

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 tools: python portability fixes

Hi!

Attached patch applies portability fixes for NetBSD:

- remove useless get-path. It is a bash specific script which tries to gain information the build system already has.
- make install-wrap work with bourne shell on NetBSD
- pass which $(PYTHON) to install-wrap. It is safe to assume this always works because in case it doesn't the build system errors out very early in tools/check/check_python

Signature · 6 lines
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy, Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
tools/misc/Makefile +1 -2
diff -r bc3aca17cb88 tools/misc/Makefile
@@ -25,8 +25,7 @@ INSTALL_BIN := $(INSTALL_BIN-y)
 INSTALL_SBIN-y := xm xen-bugtool xen-python-path xend xenperf xsview xenpm xen-tmem-list-parse gtraceview gtracestat
 INSTALL_SBIN := $(INSTALL_SBIN-y)
 
-DEFAULT_PYTHON_PATH := $(shell $(XEN_ROOT)/tools/python/get-path)
-PYTHON_PATH ?= $(DEFAULT_PYTHON_PATH)
+PYTHON_PATH := $(shell which $(PYTHON))
 INSTALL_PYTHON_PROG = $(XEN_ROOT)/tools/python/install-wrap \
 "$(PYTHON_PATH)" $(INSTALL_PROG)
 
tools/python/get-path deleted +0 -22
diff -r bc3aca17cb88 tools/python/get-path
@@ -1,22 +0,0 @@
-#! /usr/bin/env bash
-set -e
-
-check () {
-	set +e
-	p=`type -p python$v`
-	r=$?
-	set -e
-	if [ $r = 0 ]; then
-		echo >&2 "${0##*/}: will use #!$p for python programs"
-		printf "%s\n" "$p"
-		exit 0
-	fi
-}
-
-v="$(python -V 2>&1)"
-v="${v#* }"
-check
-v="${v%.*}"
-check
-echo >&2 'python version not determined, will use env to find python at runtime'
-printf "/usr/bin/env python\n"
tools/python/install-wrap +34 -18

78 lines not on this page · raw message

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#1 lore raw
Quoted 10 lines · Christoph Egger

Attached patch applies portability fixes for NetBSD:

- remove useless get-path. It is a bash specific script which tries to gain information the build system already has.
- make install-wrap work with bourne shell on NetBSD
- pass which $(PYTHON) to install-wrap. It is safe to assume this always works because in case it doesn't the build system errors out very early in tools/check/check_python

Ian's on holiday until the end of next week so don't expect a swift ack/nack on this one.

-- Keir

#2 lore raw
In reply to Christoph Egger

Christoph Egger writes ("[PATCH] tools: python portability fixes"):

Quoted 1 line

Attached patch applies portability fixes for NetBSD:

Hi. As Keir says, I'm away, but:

Quoted 6 lines

- remove useless get-path. It is a bash specific script which tries to gain information the build system already has.
- make install-wrap work with bourne shell on NetBSD
- pass which $(PYTHON) to install-wrap. It is safe to assume this always works because in case it doesn't the build system errors out very early in tools/check/check_python

Isn't the effect of this to stop substituting #!/usr/bin/python2.5 (or whatever) for #!/usr/bin/python ? In which case it's wrong as I've already explained.

Ian.

#3 lore raw
Quoted 11 lines · On Monday 27 July 2009 20:57:28 Ian Jackson wrote:

Christoph Egger writes ("[PATCH] tools: python portability fixes"):

Quoted 1 line

Attached patch applies portability fixes for NetBSD:

Hi. As Keir says, I'm away, but:

Quoted 6 lines

- remove useless get-path. It is a bash specific script which tries to gain information the build system already has.
- make install-wrap work with bourne shell on NetBSD
- pass which $(PYTHON) to install-wrap. It is safe to assume this always works because in case it doesn't the build system errors out very early in tools/check/check_python

Quoted 3 lines

Isn't the effect of this to stop substituting #!/usr/bin/python2.5 (or whatever) for #!/usr/bin/python ? In which case it's wrong as I've already explained.

No, the effect is this:

When $(PYTHON) is python you get #!/usr/bin/python, when $(PYTHON) is python2.5 you get #!/usr/bin/python2.5, when $(PYTHON) is python2.6 you get #!/usr/bin/python2.6, when $(PYTHON) is /usr/bin/python2.4 you get #!/usr/bin/python2.4, etc.

When you install python on NetBSD, you don't have a "python" binary. You always have pythonN.M where N.M is the version number so you can have multiple python versions installed.
On NetBSD, you actually get #!/usr/pkg/bin/python2.5.
The get-path is a bash specific script which tries to get the information which $(PYTHON) already provides but doesn't work for the case where "python" doesn't exist.

The current approach is doomed to always fail on NetBSD.

On Linux you have a python symlink to pythonN.M. Remove the symlink manually, build and install xen with gmake PYTHON=pythonN.M and you can reproduce the issue on Linux.

The patch fixes it.

Christoph

Signature · 6 lines
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy, Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
#4 lore raw

Christoph Egger writes ("Re: [PATCH] tools: python portability fixes"):

Quoted 1 line

[stuff]

Ah, I see. I don't think this is right.

Quoted 1 line

When $(PYTHON) is python you get #!/usr/bin/python,

This, in particularly, is wrong. But I don't have time right now to investigate this properly. Can it wait until I'm back in the office (this coming Monda) ?

Quoted 1 line

The get-path is a bash specific script which tries to get the information

                    ^^^^^^^^^^^^^^^^^^^^

While I'm here, though: you keep mentioning this as if there was something wrong with this. I think having the Xen build system depend on bash being installed is absolutely fine.

Ian.

#5 lore raw
Quoted 10 lines · On Tuesday 28 July 2009 11:55:59 Ian Jackson wrote:

Christoph Egger writes ("Re: [PATCH] tools: python portability fixes"):

Quoted 1 line

[stuff]

Ah, I see. I don't think this is right.

Quoted 1 line

When $(PYTHON) is python you get #!/usr/bin/python,

This, in particularly, is wrong. But I don't have time right now to investigate this properly. Can it wait until I'm back in the office (this coming Monda) ?

yes.

Quoted 7 lines
Quoted 2 lines

The get-path is a bash specific script which tries to get the information

                    ^^^^^^^^^^^^^^^^^^^^

While I'm here, though: you keep mentioning this as if there was something wrong with this. I think having the Xen build system depend on bash being installed is absolutely fine.

On NetBSD, bash is a third-party package.
scripts having #!/bin/bash fail with "/bin/bash: No such file or directory" The bourne shell (#!/bin/sh) is mighty enough to not require bashism.

Christoph

Signature · 6 lines
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy, Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
#6 lore raw

Christoph Egger writes ("Re: [PATCH] tools: python portability fixes"):

Quoted 8 lines · On Tuesday 28 July 2009 11:55:59 Ian Jackson wrote:
Quoted 3 lines

While I'm here, though: you keep mentioning this as if there was something wrong with this. I think having the Xen build system depend on bash being installed is absolutely fine.

On NetBSD, bash is a third-party package.
scripts having #!/bin/bash fail with "/bin/bash: No such file or directory" The bourne shell (#!/bin/sh) is mighty enough to not require bashism.

I agree that it is sensible not to make scripts bash-specific for no reason. However, there are quite a few things that are much easier to do in bash and require enormous amounts of pratting about to do in POSIX sh.

Also, if we write #!/bin/sh at the top of the script, we get complaints from Solaris people if our script is not portable to the astonishingly ancient and crusty Solaris /bin/sh.

I don't want to get into NetBSD's decision to make bash be a third party package' (whatever that means), but surely there are plenty of things in the Xen build system that are third party packages'. You can't build Xen without hg and git and texinfo and GNU make and so on. It seems to me that bash is much less of an imposition.

Ian.

#7 lore raw
Quoted 11 lines · On Tuesday 04 August 2009 12:32:41 Ian Jackson wrote:

Christoph Egger writes ("Re: [PATCH] tools: python portability fixes"):

Quoted 9 lines · On Tuesday 28 July 2009 11:55:59 Ian Jackson wrote:
Quoted 3 lines

While I'm here, though: you keep mentioning this as if there was something wrong with this. I think having the Xen build system depend on bash being installed is absolutely fine.

On NetBSD, bash is a third-party package.
scripts having #!/bin/bash fail with "/bin/bash: No such file or directory" The bourne shell (#!/bin/sh) is mighty enough to not require bashism.

Quoted 8 lines

I agree that it is sensible not to make scripts bash-specific for no reason. However, there are quite a few things that are much easier to do in bash and require enormous amounts of pratting about to do in POSIX sh.

Also, if we write #!/bin/sh at the top of the script, we get complaints from Solaris people if our script is not portable to the astonishingly ancient and crusty Solaris /bin/sh.

The build system should call shell scripts via $(SHELL). On Solaris, $(SHELL) points to /bin/bash .

Quoted 2 lines

I don't want to get into NetBSD's decision to make bash be `a third party package' (whatever that means),

that means, it's not part of the base system.

Quoted 4 lines

but surely there are plenty of things in the Xen build system that are `third party packages'. You can't build Xen without hg and git and texinfo and GNU make and so on.
It seems to me that bash is much less of an imposition.

Actually the opposite is the case.
bash as a shell is special in how it is used.

You don't type /bin/hg or /bin/git but you type #!/bin/bash in scripts and that is where it fails.

The portable way is to make it work with POSIX sh and call it with $(SHELL) (see above).

Christoph

Signature · 6 lines
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy, Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
#8 lore raw

Christoph Egger writes ("Re: [PATCH] tools: python portability fixes"):

Quoted 2 lines

You don't type /bin/hg or /bin/git but you type #!/bin/bash in scripts and that is where it fails.

If you actually look at the script you're complaining about, it starts with
#! /usr/bin/env bash
which should work just fine. Right ?

Ian.

#9 lore raw
Quoted 8 lines · On Tuesday 04 August 2009 15:03:30 Ian Jackson wrote:

Christoph Egger writes ("Re: [PATCH] tools: python portability fixes"):

Quoted 2 lines

You don't type /bin/hg or /bin/git but you type #!/bin/bash in scripts and that is where it fails.

If you actually look at the script you're complaining about, it starts with
#! /usr/bin/env bash
which should work just fine. Right ?

Yes, given that bash is installed. The non-portable part in this line is the path to env. In Solaris, env is in /bin.

Christoph

Signature · 6 lines
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy, Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
#10 lore raw
Quoted 7 lines · Christoph Egger
Quoted 4 lines

If you actually look at the script you're complaining about, it starts with
#! /usr/bin/env bash
which should work just fine. Right ?

Yes, given that bash is installed. The non-portable part in this line is the path to env. In Solaris, env is in /bin.

/usr/bin/env works fine, since /bin is a symlink to /usr/bin

regards
john