tools/python: Pass linker to Python build process
PATCH RESEND 1/1 tools/python: Pass linker to Python build process
Unexpectedly the environment variable which needs to be passed is $LDSHARED and not $LD. Otherwise Python may find the build ld instead of the host ld.
Replace $(LDFLAGS) with $(SHLIB_LDFLAGS) as Python needs shared objects it can load at runtime, not executables.
This uses $(CC) instead of $(LD) since Python distutils appends $CFLAGS to $LDFLAGS which breaks many linkers.
---
This is now the *third* time this has been sent to the list. Mark Pryor has tested and confirms Python cross-building is working. There is one wart left which I'm unsure of the best approach for.
Having looked around a bit, I believe this is a Python 2/3 compatibility issue. "distutils" for Python 2 likely lacked a separate $LDSHARED or $LD variable, whereas Python 3 does have this. Alas this is pointless due to the above (unless you can cause distutils.py to do the final link step separately).
--- tools/pygrub/Makefile | 9 +++++---- tools/python/Makefile | 9 +++++---- 2 files changed, 10 insertions(+), 8 deletions(-)
tools/pygrub/Makefile +5 -4
@@ -3,20 +3,21 @@ XEN_ROOT = $(CURDIR)/../.. include $(XEN_ROOT)/tools/Rules.mk PY_CFLAGS = $(CFLAGS) $(PY_NOOPT_CFLAGS) -PY_LDFLAGS = $(LDFLAGS) $(APPEND_LDFLAGS) +PY_LDFLAGS = $(SHLIB_LDFLAGS) $(APPEND_LDFLAGS) INSTALL_LOG = build/installed_files.txt .PHONY: all all: build .PHONY: build build: - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py build + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py build .PHONY: install install: all $(INSTALL_DIR) $(DESTDIR)/$(bindir) - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) \ - setup.py install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" \ + LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py install \ + --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ --root="$(DESTDIR)" --install-scripts=$(LIBEXEC_BIN) --force set -e; if [ $(bindir) != $(LIBEXEC_BIN) -a \ "`readlink -f $(DESTDIR)/$(bindir)`" != \
tools/python/Makefile +5 -4
@@ -5,19 +5,20 @@ include $(XEN_ROOT)/tools/Rules.mk all: build PY_CFLAGS = $(CFLAGS) $(PY_NOOPT_CFLAGS) -PY_LDFLAGS = $(LDFLAGS) $(APPEND_LDFLAGS) +PY_LDFLAGS = $(SHLIB_LDFLAGS) $(APPEND_LDFLAGS) INSTALL_LOG = build/installed_files.txt .PHONY: build build: - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" $(PYTHON) setup.py build + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py build .PHONY: install install: $(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN) - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) \ - setup.py install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" \ + LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py install \ + --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ --root="$(DESTDIR)" --force $(INSTALL_PYTHON_PROG) scripts/convert-legacy-stream $(DESTDIR)$(LIBEXEC_BIN)
Signature · 8 lines
2.20.1 -- (\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/) \BS ( | [email protected] PGP 87145445 | ) / \_CS\ | _____ -O #include <stddisclaimer.h> O- _____ | / _/ 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
Quoted 16 lines · Elliott Mitchell
Unexpectedly the environment variable which needs to be passed is $LDSHARED and not $LD. Otherwise Python may find the build
ldinstead of the hostld.Replace $(LDFLAGS) with $(SHLIB_LDFLAGS) as Python needs shared objects it can load at runtime, not executables.
This uses $(CC) instead of $(LD) since Python distutils appends $CFLAGS to $LDFLAGS which breaks many linkers.
Signed-off-by Elliott Mitchell <[email protected]>---
This is now the *third* time this has been sent to the list. Mark Pryor has tested and confirms Python cross-building is working. There is one wart left which I'm unsure of the best approach for.
Quoted 5 lines
Having looked around a bit, I believe this is a Python 2/3 compatibility issue. "distutils" for Python 2 likely lacked a separate $LDSHARED or $LD variable, whereas Python 3 does have this. Alas this is pointless due to the above (unless you can cause
distutils.pyto do the final link step separately).
I think this is well-reasoned but I don't have time to figure out and verify the details.
Marek, do you have any comment on this?
Quoted 75 lines
--- tools/pygrub/Makefile | 9 +++++---- tools/python/Makefile | 9 +++++---- 2 files changed, 10 insertions(+), 8 deletions(-)tools/pygrub/Makefile +5
-4@@ -3,20 +3,21 @@ XEN_ROOT = $(CURDIR)/../.. include $(XEN_ROOT)/tools/Rules.mk PY_CFLAGS = $(CFLAGS) $(PY_NOOPT_CFLAGS) -PY_LDFLAGS = $(LDFLAGS) $(APPEND_LDFLAGS) +PY_LDFLAGS = $(SHLIB_LDFLAGS) $(APPEND_LDFLAGS) INSTALL_LOG = build/installed_files.txt .PHONY: all all: build .PHONY: build build: - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py build + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py build .PHONY: install install: all $(INSTALL_DIR) $(DESTDIR)/$(bindir) - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) \ - setup.py install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" \ + LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py install \ + --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ --root="$(DESTDIR)" --install-scripts=$(LIBEXEC_BIN) --force set -e; if [ $(bindir) != $(LIBEXEC_BIN) -a \ "`readlink -f $(DESTDIR)/$(bindir)`" != \tools/python/Makefile +5
-4@@ -5,19 +5,20 @@ include $(XEN_ROOT)/tools/Rules.mk all: build PY_CFLAGS = $(CFLAGS) $(PY_NOOPT_CFLAGS) -PY_LDFLAGS = $(LDFLAGS) $(APPEND_LDFLAGS) +PY_LDFLAGS = $(SHLIB_LDFLAGS) $(APPEND_LDFLAGS) INSTALL_LOG = build/installed_files.txt .PHONY: build build: - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" $(PYTHON) setup.py build + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py build .PHONY: install install: $(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN) - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) \ - setup.py install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" \ + LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py install \ + --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ --root="$(DESTDIR)" --force $(INSTALL_PYTHON_PROG) scripts/convert-legacy-stream $(DESTDIR)$(LIBEXEC_BIN)Signature · 8 lines
2.20.1 -- (\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/) \BS ( | [email protected] PGP 87145445 | ) / \_CS\ | _____ -O #include <stddisclaimer.h> O- _____ | / _/ 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
Quoted 17 lines · Wei Liu
Quoted 16 lines · Elliott Mitchell
Unexpectedly the environment variable which needs to be passed is $LDSHARED and not $LD. Otherwise Python may find the build
ldinstead of the hostld.Replace $(LDFLAGS) with $(SHLIB_LDFLAGS) as Python needs shared objects it can load at runtime, not executables.
This uses $(CC) instead of $(LD) since Python distutils appends $CFLAGS to $LDFLAGS which breaks many linkers.
Signed-off-by Elliott Mitchell <[email protected]>---
This is now the *third* time this has been sent to the list. Mark Pryor has tested and confirms Python cross-building is working. There is one wart left which I'm unsure of the best approach for.
Quoted 8 lines
Quoted 5 lines
Having looked around a bit, I believe this is a Python 2/3 compatibility issue. "distutils" for Python 2 likely lacked a separate $LDSHARED or $LD variable, whereas Python 3 does have this. Alas this is pointless due to the above (unless you can cause
distutils.pyto do the final link step separately).I think this is well-reasoned but I don't have time to figure out and verify the details.
Yes, it looks like distutils in Python 2 was even more limited than the one in Python 3.
Quoted 77 lines
Marek, do you have any comment on this?
Quoted 75 lines
--- tools/pygrub/Makefile | 9 +++++---- tools/python/Makefile | 9 +++++---- 2 files changed, 10 insertions(+), 8 deletions(-)tools/pygrub/Makefile +5
-4@@ -3,20 +3,21 @@ XEN_ROOT = $(CURDIR)/../.. include $(XEN_ROOT)/tools/Rules.mk PY_CFLAGS = $(CFLAGS) $(PY_NOOPT_CFLAGS) -PY_LDFLAGS = $(LDFLAGS) $(APPEND_LDFLAGS) +PY_LDFLAGS = $(SHLIB_LDFLAGS) $(APPEND_LDFLAGS) INSTALL_LOG = build/installed_files.txt .PHONY: all all: build .PHONY: build build: - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py build + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py build .PHONY: install install: all $(INSTALL_DIR) $(DESTDIR)/$(bindir) - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) \ - setup.py install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" \ + LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py install \ + --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ --root="$(DESTDIR)" --install-scripts=$(LIBEXEC_BIN) --force set -e; if [ $(bindir) != $(LIBEXEC_BIN) -a \ "`readlink -f $(DESTDIR)/$(bindir)`" != \tools/python/Makefile +5
-4@@ -5,19 +5,20 @@ include $(XEN_ROOT)/tools/Rules.mk all: build PY_CFLAGS = $(CFLAGS) $(PY_NOOPT_CFLAGS) -PY_LDFLAGS = $(LDFLAGS) $(APPEND_LDFLAGS) +PY_LDFLAGS = $(SHLIB_LDFLAGS) $(APPEND_LDFLAGS) INSTALL_LOG = build/installed_files.txt .PHONY: build build: - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" $(PYTHON) setup.py build + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py build .PHONY: install install: $(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN) - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) \ - setup.py install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" \ + LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py install \ + --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ --root="$(DESTDIR)" --force $(INSTALL_PYTHON_PROG) scripts/convert-legacy-stream $(DESTDIR)$(LIBEXEC_BIN)Signature · 8 lines
2.20.1 -- (\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/) \BS ( | [email protected] PGP 87145445 | ) / \_CS\ | _____ -O #include <stddisclaimer.h> O- _____ | / _/ 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
Signature · 5 lines
Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
Quoted 8 lines · Marek Marczykowski-G??recki
Quoted 7 lines · Wei Liu
Quoted 5 lines · Elliott Mitchell
Having looked around a bit, I believe this is a Python 2/3 compatibility issue. "distutils" for Python 2 likely lacked a separate $LDSHARED or $LD variable, whereas Python 3 does have this. Alas this is pointless due to the above (unless you can cause
distutils.pyto do the final link step separately).
Quoted 5 lines
Quoted 2 lines
I think this is well-reasoned but I don't have time to figure out and verify the details.
Yes, it looks like distutils in Python 2 was even more limited than the one in Python 3.
Actually feels like two steps forward, one step back. Separate $LDSHARED, yet $CFLAGS is appended to $LDFLAGS during invocation.
Architecture name is included the output Python extension filename, yet the only way to override requires writing Python code. I've got ideas to workaround this, but they're rather gross.
Signature · 4 lines
(\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/) \BS ( | [email protected] PGP 87145445 | ) / \_CS\ | _____ -O #include <stddisclaimer.h> O- _____ | / _/ 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
Quoted 11 lines · Elliott Mitchell
Unexpectedly the environment variable which needs to be passed is $LDSHARED and not $LD. Otherwise Python may find the build
ldinstead of the hostld.Replace $(LDFLAGS) with $(SHLIB_LDFLAGS) as Python needs shared objects it can load at runtime, not executables.
This uses $(CC) instead of $(LD) since Python distutils appends $CFLAGS to $LDFLAGS which breaks many linkers.
Signed-off-by Elliott Mitchell <[email protected]>
Quoted 74 lines
---
This is now the *third* time this has been sent to the list. Mark Pryor has tested and confirms Python cross-building is working. There is one wart left which I'm unsure of the best approach for.Having looked around a bit, I believe this is a Python 2/3 compatibility issue. "distutils" for Python 2 likely lacked a separate $LDSHARED or $LD variable, whereas Python 3 does have this. Alas this is pointless due to the above (unless you can cause
distutils.pyto do the final link step separately).--- tools/pygrub/Makefile | 9 +++++---- tools/python/Makefile | 9 +++++---- 2 files changed, 10 insertions(+), 8 deletions(-)tools/pygrub/Makefile +5
-4@@ -3,20 +3,21 @@ XEN_ROOT = $(CURDIR)/../.. include $(XEN_ROOT)/tools/Rules.mk PY_CFLAGS = $(CFLAGS) $(PY_NOOPT_CFLAGS) -PY_LDFLAGS = $(LDFLAGS) $(APPEND_LDFLAGS) +PY_LDFLAGS = $(SHLIB_LDFLAGS) $(APPEND_LDFLAGS) INSTALL_LOG = build/installed_files.txt .PHONY: all all: build .PHONY: build build: - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py build + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py build .PHONY: install install: all $(INSTALL_DIR) $(DESTDIR)/$(bindir) - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) \ - setup.py install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" \ + LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py install \ + --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ --root="$(DESTDIR)" --install-scripts=$(LIBEXEC_BIN) --force set -e; if [ $(bindir) != $(LIBEXEC_BIN) -a \ "`readlink -f $(DESTDIR)/$(bindir)`" != \tools/python/Makefile +5
-4@@ -5,19 +5,20 @@ include $(XEN_ROOT)/tools/Rules.mk all: build PY_CFLAGS = $(CFLAGS) $(PY_NOOPT_CFLAGS) -PY_LDFLAGS = $(LDFLAGS) $(APPEND_LDFLAGS) +PY_LDFLAGS = $(SHLIB_LDFLAGS) $(APPEND_LDFLAGS) INSTALL_LOG = build/installed_files.txt .PHONY: build build: - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" $(PYTHON) setup.py build + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py build .PHONY: install install: $(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN) - CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) \ - setup.py install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ + CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" \ + LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py install \ + --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \ --root="$(DESTDIR)" --force $(INSTALL_PYTHON_PROG) scripts/convert-legacy-stream $(DESTDIR)$(LIBEXEC_BIN)
Signature · 5 lines
Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
Quoted 8 lines · Marek Marczykowski-Górecki
Quoted 7 lines · Elliott Mitchell
Unexpectedly the environment variable which needs to be passed is $LDSHARED and not $LD. Otherwise Python may find the build
ldinstead of the hostld.Replace $(LDFLAGS) with $(SHLIB_LDFLAGS) as Python needs shared objects it can load at runtime, not executables.
Quoted 6 lines
Quoted 4 lines
This uses $(CC) instead of $(LD) since Python distutils appends $CFLAGS to $LDFLAGS which breaks many linkers.
Signed-off-by Elliott Mitchell <[email protected]>
Thanks. Applied.
Wei.