Skip to contentSkip to conversation

Add some intel-specific tune files, and related fixes

PATCH Superseded [email protected] 2 participants 10 messages Nov 7, 2011 – Nov 8, 2011

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Add some intel-specific tune files, and related fixes

From: Tom Zanussi <[email protected]>

This patchset adds some new tune files for corei7, ivb, and atom.

The corei7 and atom were build- and boot-tested on sugarbay (with corei7-avx), jasperforest (corei7), crownbay, fri2, fishriver (atom). The ivb is one I'll be using for new Ivy Bridge development and hasn't been tested yet, but will be needed for that.

This patchset also contains a few patches for packages that had build problems with the corei7-avx tuning. The python and gmp packages had problems because they tried to execute target code on the host and caused illegal instruction errors. The libzypp failed compilation because of the hyphens in the tune name (corei7-avx).

Tom Zanussi (6):
tune-corei7: Add tune file for Intel Core i7 machines
tune-ivb: Add tune file for Intel Ivy Bridge machines machine/include/tune-atom.inc: add atom-specific settings gmp_5.0.2: Set CC_FOR_BUILD to BUILD_CC
libzypp: fix mishandling of hyphenated arches
python: skip setup.py

 meta/conf/machine/include/tune-atom.inc            |   26 +++++++++++++++++-
 meta/conf/machine/include/tune-corei7.inc          |   17 ++++++++++++
 meta/conf/machine/include/tune-ivb.inc             |   11 ++++++++
 .../python/setup_py_skip_cross_import_check.patch  |   27 ++++++++++++++++++++
 meta/recipes-devtools/python/python_2.7.2.bb       |    7 ++++-
 meta/recipes-extended/libzypp/libzypp_git.bb       |    8 +++---
 meta/recipes-support/gmp/gmp_5.0.2.bb              |    3 +-
 7 files changed, 91 insertions(+), 8 deletions(-)
 create mode 100644 meta/conf/machine/include/tune-corei7.inc
 create mode 100644 meta/conf/machine/include/tune-ivb.inc
 create mode 100644 meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch
#1 lore raw

PATCH 1/6 tune-corei7: Add tune file for Intel Core i7 machines

From: Tom Zanussi <[email protected]>

Add a tune file for Intel Core i7 machines both with and without AVX support.

---
 meta/conf/machine/include/tune-corei7.inc |   17 +++++++++++++++++
 1 files changed, 17 insertions(+), 0 deletions(-)
 create mode 100644 meta/conf/machine/include/tune-corei7.inc
meta/conf/machine/include/tune-corei7.inc new file +17 -0
new file mode 100644
index 0000000..2e0c67f
@@ -0,0 +1,17 @@
+DEFAULTTUNE ?= "corei7"
+TUNE_PKGARCH ?= "${@bb.utils.contains("TUNE_FEATURES", "avx", "corei7-avx", "corei7", d)}"
+
+require conf/machine/include/tune-x86_64.inc
+
+TUNEVALID[corei7] = "Enable corei7-specific processor optimizations"
+TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "corei7", "-march=corei7${COREI7_AVX}", "", d)}"
+
+COREI7_AVX = "${@bb.utils.contains("TUNE_FEATURES", [ "corei7", "avx" ], "-avx", "", d)}"
+
+AVAILTUNES += "corei7"
+TUNE_FEATURES_tune-corei7 = "${TUNE_FEATURES_tune-x86-64} corei7"
+PACKAGE_EXTRA_ARCHS_tune-corei7 = "${PACKAGE_EXTRA_ARCHS_tune-x86-64} corei7"
+
+AVAILTUNES += "corei7-avx"
+TUNE_FEATURES_tune-corei7-avx = "${TUNE_FEATURES_tune-x86-64} corei7 avx"
+PACKAGE_EXTRA_ARCHS_tune-corei7-avx = "${PACKAGE_EXTRA_ARCHS_tune-x86-64} corei7-avx"
Signature · 1 line
1.7.0.4
#2 lore raw

PATCH 2/6 tune-ivb: Add tune file for Intel Ivy Bridge machines

From: Tom Zanussi <[email protected]>

Add a tune file for Intel Ivy Bridge machines.

---
 meta/conf/machine/include/tune-ivb.inc |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)
 create mode 100644 meta/conf/machine/include/tune-ivb.inc
meta/conf/machine/include/tune-ivb.inc new file +11 -0
new file mode 100644
index 0000000..3c0b670
@@ -0,0 +1,11 @@
+DEFAULTTUNE ?= "ivb"
+TUNE_PKGARCH ?= "${@bb.utils.contains("TUNE_FEATURES", "ivb", "ivb", "", d)}"
+
+require conf/machine/include/tune-x86_64.inc
+
+TUNEVALID[ivb] = "Enable ivb-specific processor optimizations"
+TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "ivb", "-march=core-avx-i", "", d)}"
+
+AVAILTUNES += "ivb"
+TUNE_FEATURES_tune-ivb = "${TUNE_FEATURES_tune-x86-64} ivb"
+PACKAGE_EXTRA_ARCHS_tune-ivb = "${PACKAGE_EXTRA_ARCHS_tune-x86-64} ivb"
Signature · 1 line
1.7.0.4
#3 lore raw

PATCH 3/6 machine/include/tune-atom.inc: add atom-specific settings

In reply to [email protected]

From: Tom Zanussi <[email protected]>

Atom tunings are currently the same as core2; this patch changes them to make use of atom-specific settings such as the atom arch and instruction support in gcc.

---
 meta/conf/machine/include/tune-atom.inc |   26 ++++++++++++++++++++++++--
 1 files changed, 24 insertions(+), 2 deletions(-)
meta/conf/machine/include/tune-atom.inc +24 -2
index 5e1bb74..ab4aa14 100644
@@ -1,2 +1,24 @@
-# Atom tunings are the same as core2 for now...
-require conf/machine/include/tune-core2.inc
+DEFAULTTUNE ?= "atom"
+TUNE_PKGARCH ?= "${@bb.utils.contains("TUNE_FEATURES", "m32", "atom", "atom-64", d)}"
+
+require conf/machine/include/tune-i586.inc
+
+# Extra tune features
+TUNEVALID[atom] = "Enable atom-specific processor optimizations"
+TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "atom", "-march=atom -mssse3 -mfpmath=sse", "", d)}"
+
+# Extra tune selections
+AVAILTUNES += "atom"
+TUNE_FEATURES_tune-atom ?= "${TUNE_FEATURES_tune-x86} atom"
+BASE_LIB_tune-atom ?= "lib"
+PACKAGE_EXTRA_ARCHS_tune-atom = "${PACKAGE_EXTRA_ARCHS_tune-x86} i386 i486 i586 i686 atom"
+
+AVAILTUNES += "atom-64"
+TUNE_FEATURES_tune-atom-64 ?= "${TUNE_FEATURES_tune-x86-64} atom"
+BASE_LIB_tune-atom-64 ?= "lib64"
+PACKAGE_EXTRA_ARCHS_tune-atom-64 = "${PACKAGE_EXTRA_ARCHS_tune-x86-64} atom-64"
+
+AVAILTUNES += "atom-64-x32"
+TUNE_FEATURES_tune-atom-64-x32 ?= "${TUNE_FEATURES_tune-x86-64-x32} atom"
+BASE_LIB_tune-atom-64-x32 ?= "libx32"
+PACKAGE_EXTRA_ARCHS_tune-atom-64-x32 = "${PACKAGE_EXTRA_ARCHS_tune-x86-64-x32} atom-64-x32"
Signature · 1 line
1.7.0.4
#4 lore raw

PATCH 4/6 gmp_5.0.2: Set CC_FOR_BUILD to BUILD_CC

In reply to [email protected]

From: Tom Zanussi <[email protected]>

CC_FOR_BUILD was compiling the test programs using the target's compile options and executing those on the host, causing errors such as:

/bin/sh: line 1: 15032 Illegal instruction ./gen-bases table 64 0 > mpn/mp_bases.c
/bin/sh: line 1: 15033 Illegal instruction ./gen-bases header 64 0 > mp_bases.h

Export CC_FOR_BUILD using BUILD_CC to fix the problem.

---
 meta/recipes-support/gmp/gmp_5.0.2.bb |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
meta/recipes-support/gmp/gmp_5.0.2.bb +2 -1
index 03fef45..16bdcbc 100644
@@ -2,11 +2,12 @@ require gmp.inc
 LICENSE="LGPLv3&GPLv3"
 LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \
 		    file://version.c;endline=18;md5=d8c56b52b9092346b9f93b4da65ef790"
-PR = "r0"
+PR = "r1"
 
 SRC_URI_append += "file://sh4-asmfix.patch \
                    file://use-includedir.patch "
 
+export CC_FOR_BUILD = "${BUILD_CC}"
 
 SRC_URI[md5sum] = "0bbaedc82fb30315b06b1588b9077cd3"
 SRC_URI[sha256sum] = "dbc2db76fdd4e99f85d5e35aa378ed62c283e0d586b91bd8703aff75a7804c28"
Signature · 1 line
1.7.0.4
#5 lore raw

PATCH 5/6 libzypp: fix mishandling of hyphenated arches

In reply to [email protected]

From: Tom Zanussi <[email protected]>

Several hyphen-to-underscore translations were missing, causing compiler errors trying to build arches with hyphens in their names. This adds the missing translations.

---
 meta/recipes-extended/libzypp/libzypp_git.bb |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)
meta/recipes-extended/libzypp/libzypp_git.bb +4 -4
index 20e8f7c..a6b270c 100644
@@ -11,7 +11,7 @@ DEPENDS  = "rpm boost curl libxml2 zlib sat-solver expat openssl udev"
 S = "${WORKDIR}/git"
 SRCREV = "15b6c52260bbc52b3d8e585e271b67e10cc7c433"
 PV = "0.0-git${SRCPV}"
-PR = "r15"
+PR = "r16"
 
 SRC_URI = "git://github.com/openSUSE/libzypp.git;protocol=git \
            file://no-doc.patch \
@@ -98,10 +98,10 @@ do_archgen () {
 		esac
 		if [ "${AVOID_CONSTRUCTOR}" != "true" ]; then
 		  echo -n "  const Arch Arch_${each_arch} " | tr - _		>> zypp/oe-arch.h
-		  echo "(_${each_arch});" 					>> zypp/oe-arch.h
+		  echo "(_${each_arch});" | tr - _				>> zypp/oe-arch.h
 		else
 		  echo -n "  const Arch Arch_${each_arch} " | tr - _		>> zypp/oe-arch.h
-		  echo "( IdString ( \"${each_arch}\" ) );" 			>> zypp/oe-arch.h
+		  echo "( IdString ( \"${each_arch}\" ) );" | tr - _		>> zypp/oe-arch.h
 		fi
 	done
 	echo "#endif /* OE_PROTO */"						>> zypp/oe-arch.h
@@ -142,7 +142,7 @@ do_archgen () {
 		COMPAT_WITH="${CARCH},${COMPAT} $COMPAT_WITH"
 	done
 	for each_compat in ${COMPAT_WITH} ; do
-		echo "        defCompatibleWith( ${each_compat} );"		>> zypp/oe-arch.h
+		echo "        defCompatibleWith( ${each_compat} );" | tr - _	>> zypp/oe-arch.h
 	done
 	echo "#endif /* DEF_COMPAT */"						>> zypp/oe-arch.h
 	echo ""									>> zypp/oe-arch.h
Signature · 1 line
1.7.0.4
#6 lore raw

PATCH 6/6 python: skip setup.py

In reply to [email protected]

From: Tom Zanussi <[email protected]>

build_extension() in setup.py, as part of the build process, does an 'import check' on the built extension. The import check in turn dlopen()'s the shared library associated with the extension, which isn't something that makes sense if that library was cross-compiled for a different architecture.

This was noticed with an x86_64 target that was compiled with avx support, because it caused 'illegal instruction' exceptions:

| /bin/sh: line 1: 14575 Illegal instruction ... -E ./setup.py -q build

For other target architectures, it doesn't necessarily cause illegal instruction exceptions, but still fails. For example, on arm, the failure pathway causes this warning:

*** WARNING: renaming "cmath" since importing it failed: .../cmath.so:
wrong ELF class: ELFCLASS32

This patch to setup.py and the associated recipe changes allow the whole 'import check' logic to be skipped when cross-compiling.

---
 .../python/setup_py_skip_cross_import_check.patch  |   27 ++++++++++++++++++++
 meta/recipes-devtools/python/python_2.7.2.bb       |    7 ++++-
 2 files changed, 33 insertions(+), 1 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch
meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch new file +27 -0
new file mode 100644
index 0000000..6ccdb94
@@ -0,0 +1,27 @@
+This patch skips over the 'import check' setup.py does when building
+extensions.  This generally won't work when cross-compiling.
+
+Upstream-Status: Inappropriate [embedded-specific]
+
+Signed-off-by: Tom Zanussi <[email protected]>
+
+Index: Python-2.7.2/setup.py
+===================================================================
+--- Python-2.7.2.orig/setup.py	2011-11-04 16:46:34.553796410 -0500
++++ Python-2.7.2/setup.py	2011-11-04 16:59:49.692802313 -0500
+@@ -287,6 +287,15 @@
+                           (ext.name, sys.exc_info()[1]))
+             self.failed.append(ext.name)
+             return
++
++        # If we're cross-compiling, we want to skip the import check
++        # i.e. we shouldn't be dynamically loading target shared libs
++        if os.environ.get('CROSS_COMPILE') is not None:
++            self.announce(
++                'WARNING: skipping import check for cross-compiled "%s"' %
++                ext.name)
++            return
++
+         # Workaround for Mac OS X: The Carbon-based modules cannot be
+         # reliably imported into a command-line Python
+         if 'Carbon' in ext.extra_link_args:
meta/recipes-devtools/python/python_2.7.2.bb +6 -1
index 45d8350..0981d5b 100644
@@ -1,7 +1,7 @@
 require python.inc
 DEPENDS = "python-native db gdbm openssl readline sqlite3 zlib"
 DEPENDS_sharprom = "python-native db readline zlib gdbm openssl"
-PR = "${INC_PR}.0"
+PR = "${INC_PR}.1"
 
 DISTRO_SRC_URI ?= "file://sitecustomize.py"
 DISTRO_SRC_URI_linuxstdbase = ""
@@ -18,6 +18,7 @@ SRC_URI += "\
   file://multilib.patch \
   file://cgi_py.patch \
   file://remove_sqlite_rpath.patch \
+  file://setup_py_skip_cross_import_check.patch \
 "
 
 S = "${WORKDIR}/Python-${PV}"
@@ -57,6 +58,8 @@ do_compile() {
 	# then call do_install twice we get Makefile.orig == Makefile.sysroot
 	install -m 0644 Makefile Makefile.sysroot
 
+	export CROSS_COMPILE="${TARGET_PREFIX}"
+
 	oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/pgen \
 		HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python \
 		STAGING_LIBDIR=${STAGING_LIBDIR} \
@@ -78,6 +81,8 @@ do_install() {
 	# make install needs the original Makefile, or otherwise the inclues would
 	# go to ${D}${STAGING...}/...
 	install -m 0644 Makefile.orig Makefile
+
+	export CROSS_COMPILE="${TARGET_PREFIX}"
 	
 	oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/pgen \
 		HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python \
Signature · 1 line
1.7.0.4
#7 lore raw
In reply to [email protected]
Quoted 4 lines · On 11/07/2011 07:53 AM, [email protected] wrote:

From: Tom Zanussi<[email protected]>

This patchset adds some new tune files for corei7, ivb, and atom.

Tom,

Where is the repo containing this patch set? Are you using the create_pull_request script, which generates the formatted message which includes the repo from which to pull this patchset?

Thanks
Sau!

Quoted 37 lines

The corei7 and atom were build- and boot-tested on sugarbay (with corei7-avx), jasperforest (corei7), crownbay, fri2, fishriver (atom). The ivb is one I'll be using for new Ivy Bridge development and hasn't been tested yet, but will be needed for that.

This patchset also contains a few patches for packages that had build problems with the corei7-avx tuning. The python and gmp packages had problems because they tried to execute target code on the host and caused illegal instruction errors. The libzypp failed compilation because of the hyphens in the tune name (corei7-avx).

Tom Zanussi (6):
tune-corei7: Add tune file for Intel Core i7 machines
tune-ivb: Add tune file for Intel Ivy Bridge machines machine/include/tune-atom.inc: add atom-specific settings gmp_5.0.2: Set CC_FOR_BUILD to BUILD_CC
libzypp: fix mishandling of hyphenated arches
python: skip setup.py

  meta/conf/machine/include/tune-atom.inc            |   26 +++++++++++++++++-
  meta/conf/machine/include/tune-corei7.inc          |   17 ++++++++++++
  meta/conf/machine/include/tune-ivb.inc             |   11 ++++++++
  .../python/setup_py_skip_cross_import_check.patch  |   27 ++++++++++++++++++++
  meta/recipes-devtools/python/python_2.7.2.bb       |    7 ++++-
  meta/recipes-extended/libzypp/libzypp_git.bb       |    8 +++---
  meta/recipes-support/gmp/gmp_5.0.2.bb              |    3 +-
  7 files changed, 91 insertions(+), 8 deletions(-)

create mode 100644 meta/conf/machine/include/tune-corei7.inc
create mode 100644 meta/conf/machine/include/tune-ivb.inc
create mode 100644 meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

#8 lore raw
Quoted 11 lines · Saul Wold
Quoted 4 lines · On 11/07/2011 07:53 AM, [email protected] wrote:

From: Tom Zanussi<[email protected]>

This patchset adds some new tune files for corei7, ivb, and atom.

Tom,

Where is the repo containing this patch set? Are you using the create_pull_request script, which generates the formatted message which includes the repo from which to pull this patchset?

I didn't push this patchset to a contrib repo, just posted them as patches. Not ok?

Tom

Quoted 40 lines

Thanks
Sau!

Quoted 37 lines

The corei7 and atom were build- and boot-tested on sugarbay (with corei7-avx), jasperforest (corei7), crownbay, fri2, fishriver (atom). The ivb is one I'll be using for new Ivy Bridge development and hasn't been tested yet, but will be needed for that.

This patchset also contains a few patches for packages that had build problems with the corei7-avx tuning. The python and gmp packages had problems because they tried to execute target code on the host and caused illegal instruction errors. The libzypp failed compilation because of the hyphens in the tune name (corei7-avx).

Tom Zanussi (6):
tune-corei7: Add tune file for Intel Core i7 machines
tune-ivb: Add tune file for Intel Ivy Bridge machines machine/include/tune-atom.inc: add atom-specific settings gmp_5.0.2: Set CC_FOR_BUILD to BUILD_CC
libzypp: fix mishandling of hyphenated arches
python: skip setup.py

  meta/conf/machine/include/tune-atom.inc            |   26 +++++++++++++++++-
  meta/conf/machine/include/tune-corei7.inc          |   17 ++++++++++++
  meta/conf/machine/include/tune-ivb.inc             |   11 ++++++++
  .../python/setup_py_skip_cross_import_check.patch  |   27 ++++++++++++++++++++
  meta/recipes-devtools/python/python_2.7.2.bb       |    7 ++++-
  meta/recipes-extended/libzypp/libzypp_git.bb       |    8 +++---
  meta/recipes-support/gmp/gmp_5.0.2.bb              |    3 +-
  7 files changed, 91 insertions(+), 8 deletions(-)

create mode 100644 meta/conf/machine/include/tune-corei7.inc
create mode 100644 meta/conf/machine/include/tune-ivb.inc
create mode 100644 meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

#9 lore raw
In reply to [email protected]
Quoted 15 lines · On 11/07/2011 07:53 AM, [email protected] wrote:

From: Tom Zanussi<[email protected]>

This patchset adds some new tune files for corei7, ivb, and atom.

The corei7 and atom were build- and boot-tested on sugarbay (with corei7-avx), jasperforest (corei7), crownbay, fri2, fishriver (atom). The ivb is one I'll be using for new Ivy Bridge development and hasn't been tested yet, but will be needed for that.

This patchset also contains a few patches for packages that had build problems with the corei7-avx tuning. The python and gmp packages had problems because they tried to execute target code on the host and caused illegal instruction errors. The libzypp failed compilation because of the hyphens in the tune name (corei7-avx).

Quoted 4 lines

Tom Zanussi (6):
tune-corei7: Add tune file for Intel Core i7 machines
tune-ivb: Add tune file for Intel Ivy Bridge machines machine/include/tune-atom.inc: add atom-specific settings

Richard needs talk with you these tune changes.

Quoted 4 lines

gmp_5.0.2: Set CC_FOR_BUILD to BUILD_CC
libzypp: fix mishandling of hyphenated arches
python: skip setup.py

Merged these 3 to OE-Core

Thanks
Sau!

Quoted 18 lines
  meta/conf/machine/include/tune-atom.inc            |   26 +++++++++++++++++-
  meta/conf/machine/include/tune-corei7.inc          |   17 ++++++++++++
  meta/conf/machine/include/tune-ivb.inc             |   11 ++++++++
  .../python/setup_py_skip_cross_import_check.patch  |   27 ++++++++++++++++++++
  meta/recipes-devtools/python/python_2.7.2.bb       |    7 ++++-
  meta/recipes-extended/libzypp/libzypp_git.bb       |    8 +++---
  meta/recipes-support/gmp/gmp_5.0.2.bb              |    3 +-
  7 files changed, 91 insertions(+), 8 deletions(-)

create mode 100644 meta/conf/machine/include/tune-corei7.inc
create mode 100644 meta/conf/machine/include/tune-ivb.inc
create mode 100644 meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core