Skip to contentSkip to conversation

Some ADT related refactoring

PATCH In review Laurentiu Palcu 3 participants 8 messages Sep 10, 2013

lore
Share & tools
lore
#0 lore raw

cover Some ADT related refactoring

Hi,

Currently, if one wants to create an ADT repo for a certain architecture, but for different machines (for example for arm: qemuarm and beagleboard), it will fail. That's because the meta-environment file created is the same:
meta-environment-arm

The problem here is that the environment file will contain compiler options specific to the MACHINE used for the last compilation. For example, if one compiles the toolchain for qemuarm followed by a beagleboard compilation, the meta-environment package will contain the latter settings.

This patchset will add the possibility to specify, in adt-installer.conf, the machine for which the toolchain is to be installed which will select the correct meta-environment file and target sysroot.

Thanks,
Laurentiu

The following changes since commit 0ca5d1fb38157564a2b9452ade32391d18a41b09:

yocto-bsp: remove apm as a default MACHINE_FEATURE (2013-09-09 16:28:47 +0100)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib lpalcu/b4783_target_arch_change
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b4783_target_arch_change

Laurentiu Palcu (5):
meta-environment: change package name to include MACHINE packagegroup-cross-canadian: change package name to include MACHINE populate_sdk_base.bbclass: select the proper cross-canadian package
meta-toolchain-qt: include the proper cross-canadian package
adt-installer: allow installation of other machines

 meta/classes/populate_sdk_base.bbclass             |    2 +-
 meta/recipes-core/meta/meta-environment.bb         |    2 +-
 .../packagegroups/packagegroup-cross-canadian.bb   |   13 +++-----
 .../installer/adt-installer/adt_installer          |   33 ++++++++++----------
 .../installer/adt-installer/adt_installer.conf     |   20 +++++++++---
 .../adt-installer/opkg/conf/opkg-sdk-i686.conf     |    1 +
 .../adt-installer/opkg/conf/opkg-sdk-x86_64.conf   |    1 +
 .../adt-installer/scripts/adt_installer_internal   |   27 +++++-----------
 meta/recipes-qt/meta/meta-toolchain-qt.inc         |    2 +-
 9 files changed, 50 insertions(+), 51 deletions(-)
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH 1/5 meta-environment: change package name to include MACHINE

Currently, the package name contains just the TRANSLATED_TARGET_ARCH. When compiling the toolchain for two machines, belonging to the same architecture (for example: qemuarm and beagleboard), this package gets overwritten and adt-installer repo will contain just one meta-environment package... This leads to situations like installing the toolchain for qemuarm and end up with meta-environment package with beagleboard cpu options.

[YOCTO #4783]

---
 meta/recipes-core/meta/meta-environment.bb |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-core/meta/meta-environment.bb +1 -1
index 9731300..3a71a9b 100644
@@ -37,7 +37,7 @@ do_install() {
     install -m 0644 -t ${D}/${SDKPATH} ${SDK_OUTPUT}/${SDKPATH}/*
 }
 
-PN = "meta-environment-${TRANSLATED_TARGET_ARCH}"
+PN = "meta-environment-${MACHINE}"
 PACKAGES = "${PN}"
 FILES_${PN}= " \
     ${SDKPATH}/* \
Signature · 1 line
1.7.9.5
#4 lore raw

PATCH 2/5 packagegroup-cross-canadian: change package name to include MACHINE

In reply to Laurentiu Palcu

Since meta-environment package name was changed to contain MACHINE, change packagegroup-cross-canadian package name too, in order to be able to select the proper environment files for a certain machine.

Also, remove the RPROVIDES since adt-installer doesn't really need it.

[YOCTO #4783]

---
 .../packagegroups/packagegroup-cross-canadian.bb   |   13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)
meta/recipes-core/packagegroups/packagegroup-cross-canadian.bb +5 -8
index a6c6c36..064edfb 100644
@@ -1,5 +1,5 @@
 SUMMARY = "Host SDK package for cross canadian toolchain"
-PN = "packagegroup-cross-canadian-${TRANSLATED_TARGET_ARCH}"
+PN = "packagegroup-cross-canadian-${MACHINE}"
 PR = "r0"
 LICENSE = "MIT"
 
@@ -10,13 +10,10 @@ inherit cross-canadian packagegroup
 
 PACKAGEGROUP_DISABLE_COMPLEMENTARY = "1"
 
-# For backwards compatibility after rename
-RPROVIDES_${PN} = "task-cross-canadian-${TRANSLATED_TARGET_ARCH}"
-
 RDEPENDS_${PN} = "\
-    binutils-cross-canadian-${TRANSLATED_TARGET_ARCH} \
-    gdb-cross-canadian-${TRANSLATED_TARGET_ARCH} \
-    gcc-cross-canadian-${TRANSLATED_TARGET_ARCH} \
-    meta-environment-${TRANSLATED_TARGET_ARCH} \
+    binutils-cross-canadian-${@' binutils-cross-canadian-'.join(all_multilib_tune_values(d,'TRANSLATED_TARGET_ARCH').split())} \
+    gdb-cross-canadian-${@' gdb-cross-canadian-'.join(all_multilib_tune_values(d, 'TRANSLATED_TARGET_ARCH').split())} \
+    gcc-cross-canadian-${@' gcc-cross-canadian-'.join(all_multilib_tune_values(d, 'TRANSLATED_TARGET_ARCH').split())} \
+    meta-environment-${MACHINE} \
     "
 
Signature · 1 line
1.7.9.5
#5 lore raw

PATCH 3/5 populate_sdk_base.bbclass: select the proper cross-canadian package

In reply to Laurentiu Palcu

Since packagegroup-cross-canadian package name has MACHINE in it, make the necessary changes here.

[YOCTO #4783]

---
 meta/classes/populate_sdk_base.bbclass |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/populate_sdk_base.bbclass +1 -1
index 998280e..bddcf62 100644
@@ -10,7 +10,7 @@ SDK_DEPLOY = "${TMPDIR}/deploy/sdk"
 
 SDKTARGETSYSROOT = "${SDKPATH}/sysroots/${REAL_MULTIMACH_TARGET_SYS}"
 
-TOOLCHAIN_HOST_TASK ?= "nativesdk-packagegroup-sdk-host packagegroup-cross-canadian-${@' packagegroup-cross-canadian-'.join(all_multilib_tune_values(d, 'TRANSLATED_TARGET_ARCH').split())}"
+TOOLCHAIN_HOST_TASK ?= "nativesdk-packagegroup-sdk-host packagegroup-cross-canadian-${MACHINE}"
 TOOLCHAIN_HOST_TASK_ATTEMPTONLY ?= ""
 TOOLCHAIN_TARGET_TASK ?= "packagegroup-core-standalone-sdk-target packagegroup-core-standalone-sdk-target-dbg"
 TOOLCHAIN_TARGET_TASK_ATTEMPTONLY ?= ""
Signature · 1 line
1.7.9.5
#6 lore raw

PATCH 4/5 meta-toolchain-qt: include the proper cross-canadian package

In reply to Laurentiu Palcu

packagegroup-cross-canadian package name was changed to contain MACHINE. Make the necessary changes here too.

[YOCTO #4783]

---
 meta/recipes-qt/meta/meta-toolchain-qt.inc |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-qt/meta/meta-toolchain-qt.inc +1 -1
index abf12fb..14591a8 100644
@@ -1,4 +1,4 @@
-TOOLCHAIN_HOST_TASK = "nativesdk-packagegroup-${QTNAME}-toolchain-host packagegroup-cross-canadian-${TRANSLATED_TARGET_ARCH}"
+TOOLCHAIN_HOST_TASK = "nativesdk-packagegroup-${QTNAME}-toolchain-host packagegroup-cross-canadian-${MACHINE}"
 TOOLCHAIN_TARGET_TASK = "packagegroup-${QTNAME}-toolchain-target"
 TOOLCHAIN_OUTPUTNAME = "${SDK_NAME}-toolchain-${QTNAME}-${DISTRO_VERSION}"
 
Signature · 1 line
1.7.9.5
#7 lore raw

PATCH 5/5 adt-installer: allow installation of other machines

In reply to Laurentiu Palcu

Currently, adt-installer allows only the installation of qemu target sysroots.

The changes in this patch do the following:
* add a new setting in adt-installer.conf (YOCTOADT_TARGET_MACHINE) for each target architecture. For example, for arm we can choose to use a qemuarm sysroot or a beagleboard sysroot. By default, only the qemu target sysroots are selected (current behavior);
* change adt_installer scripts to allow installing the correct meta-environment package for the selected machine;
* remove some left-over commented lines;
* use packagegroup-cross-canadian-${MACHINE} when installing the cross canadian packages instead of doing it separately for each package;
* change the opkg config files in order to be able to find the packagegroup package, which is allarch;

[YOCTO #4783]

---
 .../installer/adt-installer/adt_installer          |   33 ++++++++++----------
 .../installer/adt-installer/adt_installer.conf     |   20 +++++++++---
 .../adt-installer/opkg/conf/opkg-sdk-i686.conf     |    1 +
 .../adt-installer/opkg/conf/opkg-sdk-x86_64.conf   |    1 +
 .../adt-installer/scripts/adt_installer_internal   |   27 +++++-----------
 5 files changed, 42 insertions(+), 40 deletions(-)
meta/recipes-devtools/installer/adt-installer/adt_installer +17 -16
index a6042b9..76afcf3 100755
@@ -233,44 +233,44 @@ wget "$YOCTOADT_REPO/rootfs/$1" -P $LOCAL_DOWNLOAD --progress=bar:force 2>&1 | t
 
 
 
-#Need two input params, $1 -- arch_type(arm powerpc x86 mips) $2 rootfs_image_type (a list of sdk sato minimal lsb)
-get_qemu_image()
+#Need three input params:
+# $1 arch_type(arm powerpc x86 mips)
+# $2 machine(qemuarm beagleboard)
+# $3 rootfs_image_type (a list of sdk sato minimal lsb)
+get_image()
 {
+  local machine=$2
  
-  local target=`echo "$1" | sed -e 's/x86_64/x86-64/'`
-
   if [ "$1" == "x86" ] || [ "$1" == "x86_64" ]; then
-    qemu_kernel="bzImage-qemu$target.bin"
+    kernel="bzImage-$machine.bin"
   elif [ "$1" == "ppc" ] || [ "$1" == "mips" ]; then
-    qemu_kernel="vmlinux-qemu$target.bin"
+    kernel="vmlinux-$machine.bin"
   else
-    qemu_kernel="zImage-qemu$target.bin"
+    kernel="zImage-$machine.bin"
   fi
 
   #echo_info "[ADT_INST] Downloading qemu kernel binary: $qemu_kernel"
-  download_file qemu$1/$qemu_kernel
+  download_file $machine/$kernel
   check_result
   
   for image_type in $select_rootfs; do
-    #echo_info "[ADT_INST] Downloading rootfs file: core-image-$image_type-qemu$1.tar.bz2"
-    filename="core-image-$image_type-qemu$target.tar.bz2"
-    download_file qemu$1/$filename
+    #echo_info "[ADT_INST] Downloading rootfs file: core-image-$image_type-$machine.tar.bz2"
+    filename="core-image-$image_type-$machine.tar.bz2"
+    download_file $machine/$filename
     check_result
   done
 }
 
 download_images()
 {
-  #select_target_var="\$YOCTOADT_TARGET_$1"
-  #select_target=`eval echo $select_target_var`
-
-  #if [ "$select_target" == "Y" ]; then
   select_rootfs_var="\$YOCTOADT_ROOTFS_$1"
   select_sysroot_image_var="\$YOCTOADT_TARGET_SYSROOT_IMAGE_$1"
   select_sysroot_var="\$YOCTOADT_TARGET_SYSROOT_LOC_$1"
+  select_machine_var="\$YOCTOADT_TARGET_MACHINE_$1"
   select_rootfs=`eval echo $select_rootfs_var`
   select_sysroot_image=`eval echo $select_sysroot_image_var`
   select_sysroot=`eval echo $select_sysroot_var`
+  select_machine=`eval echo $select_machine_var`
 
   if [ -n "$select_sysroot" ]; then
     select_sysroot=`readlink -m $select_sysroot`
@@ -282,13 +282,14 @@ download_images()
     #echo_info "# To be downloaded rootfs image details defined in adt_installer.conf"
     #echo_info "############################################################################"
     echo_info "# Target architecture:\t\t$1"
+    echo_info "# Target machine:\t\t$select_machine"
     echo_info "# Root_fs images:\t\t$select_rootfs"
     echo_info "# Target sysroot image:\t\t$select_sysroot_image"
     echo_info "# Target sysroot loc:\t\t$select_sysroot"
     echo_info "\n"
     #echo_info "############################################################################\n"
     else
-    get_qemu_image $1 $select_rootfs
+    get_image $1 $select_machine $select_rootfs
     fi
   fi
 }
meta/recipes-devtools/installer/adt-installer/adt_installer.conf +15 -5
index 171d2ac..cc54acd 100644
@@ -43,23 +43,33 @@ YOCTOADT_NFS_UTIL="Y"
 YOCTOADT_ROOTFS_arm="minimal sato-sdk"
 #Specify which root filesystem file to use to extract as target sysroot.  Please ensure the entry is in the list of downloaded root filesystem files that specified above in YOCTOADT_ROOTFS_$arch
 YOCTOADT_TARGET_SYSROOT_IMAGE_arm="sato-sdk"
+#The machine for which you want to download the image. Each machine might have
+#different compilation options in order to select certain hardware models or
+#configurations
+YOCTOADT_TARGET_MACHINE_arm="qemuarm"
 #The location where the target sysroot will be setup
-YOCTOADT_TARGET_SYSROOT_LOC_arm="$HOME/test-yocto/arm"
+YOCTOADT_TARGET_SYSROOT_LOC_arm="$HOME/test-yocto/$YOCTOADT_TARGET_MACHINE_arm"
+
 
 #Here's another example for setting up target arch of x86, by uncommenting it will trigger the installer to download and setup 2 sysroot environment for 2 target arches: arm and x86.  If you want to add more target arch support, you can append more entries by following these samples
 #YOCTOADT_ROOTFS_x86="sato-sdk"
 #YOCTOADT_TARGET_SYSROOT_IMAGE_x86="sato-sdk"
-#YOCTOADT_TARGET_SYSROOT_LOC_x86="$HOME/test-yocto/x86"
+#YOCTOADT_TARGET_MACHINE_x86="qemux86"
+#YOCTOADT_TARGET_SYSROOT_LOC_x86="$HOME/test-yocto/$YOCTOADT_TARGET_MACHINE_x86"
 
 #Here's some template of other arches, which you need to change the value in ""
 #YOCTOADT_ROOTFS_x86_64="sato-sdk"
 #YOCTOADT_TARGET_SYSROOT_IMAGE_x86_64="sato-sdk"
-#YOCTOADT_TARGET_SYSROOT_LOC_x86_64="$HOME/test-yocto/x86_64"
+#YOCTOADT_TARGET_MACHINE_x86_64="qemux86-64"
+#YOCTOADT_TARGET_SYSROOT_LOC_x86_64="$HOME/test-yocto/$YOCTOADT_TARGET_MACHINE_x86_64"
 
 #YOCTOADT_ROOTFS_ppc="sato-sdk"
 #YOCTOADT_TARGET_SYSROOT_IMAGE_ppc="sato-sdk"
-#YOCTOADT_TARGET_SYSROOT_LOC_ppc="$HOME/test-yocto/ppc"
+#YOCTOADT_TARGET_MACHINE_ppc="qemuppc"
+#YOCTOADT_TARGET_SYSROOT_LOC_ppc="$HOME/test-yocto/$YOCTOADT_TARGET_MACHINE_ppc"
 
 #YOCTOADT_ROOTFS_mips="sato-sdk"
 #YOCTOADT_TARGET_SYSROOT_IMAGE_mips="sato-sdk"
-#YOCTOADT_TARGET_SYSROOT_LOC_mips="$HOME/test-yocto/mips"
+#YOCTOADT_TARGET_MACHINE_mips="qemumips"
+#YOCTOADT_TARGET_SYSROOT_LOC_mips="$HOME/test-yocto/$YOCTOADT_TARGET_MACHINE_mips"
+
meta/recipes-devtools/installer/adt-installer/opkg/conf/opkg-sdk-i686.conf +1 -0
index 71c62e8..ee169f6 100644
@@ -3,3 +3,4 @@ arch any 6
 arch noarch 11
 arch i686-nativesdk 16
 src yp-i686-nativesdk ADTREPO_URL/adt-ipk/i686-nativesdk
+src yp-i686-nativesdk ADTREPO_URL/adt-ipk/all
meta/recipes-devtools/installer/adt-installer/opkg/conf/opkg-sdk-x86_64.conf +1 -0
index f0d0634..199ea50 100644
@@ -3,3 +3,4 @@ arch any 6
 arch noarch 11
 arch x86_64-nativesdk 16
 src yp-x86_64-nativesdk ADTREPO_URL/adt-ipk/x86_64-nativesdk
+src yp-x86_64-nativesdk ADTREPO_URL/adt-ipk/all
meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal +8 -19
index 462199c..abed90d 100755
@@ -121,23 +121,12 @@ for pkg in $BASE_HOSTSDK_PKGNAMES; do
   check_result
 done
 
-for native_target_type in $YOCTOADT_TARGETS; do
-  native_target_type=`echo "$native_target_type" | sed -e 's/x86_64/x86-64/' -e 's/ppc/powerpc/'  -e 's/x86$/i586/'`
-  echo_info "Installing cross toolchain for $native_target_type ..."
-  echo_info "Installing binutils for $native_target_type ..."
-  $OPKG_INSTALL_NATIVE_CMD binutils-cross-canadian-$native_target_type &>> $YOCTOADT_INSTALL_LOG_FILE
+for target_type in $YOCTOADT_TARGETS; do
+  machine_var="\$YOCTOADT_TARGET_MACHINE_$target_type"
+  machine=`eval echo $machine_var`
+  echo_info "Installing cross canadian packages for $machine ..."
+  $OPKG_INSTALL_NATIVE_CMD packagegroup-cross-canadian-$machine &>> $YOCTOADT_INSTALL_LOG_FILE
   check_result
-  echo_info "Installing gcc for $native_target_type ..."
-  $OPKG_INSTALL_NATIVE_CMD gcc-cross-canadian-$native_target_type &>> $YOCTOADT_INSTALL_LOG_FILE
-  check_result
-  echo_info "Installing gdb for $native_target_type ..."
-  $OPKG_INSTALL_NATIVE_CMD gdb-cross-canadian-$native_target_type &>> $YOCTOADT_INSTALL_LOG_FILE
-  check_result
-
-  echo_info "Installing environment file for $native_target_type ..."
-  $OPKG_INSTALL_NATIVE_CMD meta-environment-$native_target_type &>> $YOCTOADT_INSTALL_LOG_FILE
-  check_result
-
 done
 
 if [ "$YOCTOADT_QEMU" == "Y" ] || [ "$YOCTOADT_QEMU" = "y" ]; then
@@ -255,9 +244,9 @@ fi
 
 echo_info "Installing target sysroot for arch: $1, rootfs type: $target_sysroot_image, location: $target_sysroot"
 
-
-qemu_type=`echo "$1" | sed -e 's/x86_64/x86-64/'`
-sysroot_image_name="core-image-$target_sysroot_image-qemu$qemu_type.tar.bz2"
+target_machine_var="\$YOCTOADT_TARGET_MACHINE_$1"
+target_machine=`eval echo $target_machine_var`
+sysroot_image_name="core-image-$target_sysroot_image-$target_machine.tar.bz2"
     #echo_info "Extracting rootfs: $sysroot_image_name, using pseudo..."
 
 $SUDO scripts/extract_rootfs $sysroot_image_name $target_sysroot $OECORE_NATIVE_SYSROOT $user_inst_type
Signature · 1 line
1.7.9.5
#2 lore raw

Re: [PATCH 1/5] meta-environment: change package name to include MACHINE

In reply to Laurentiu Palcu
Quoted 20 lines · Laurentiu Palcu

Currently, the package name contains just the TRANSLATED_TARGET_ARCH. When compiling the toolchain for two machines, belonging to the same architecture (for example: qemuarm and beagleboard), this package gets overwritten and adt-installer repo will contain just one meta-environment package... This leads to situations like installing the toolchain for qemuarm and end up with meta-environment package with beagleboard cpu options.

[YOCTO #4783]

---
 meta/recipes-core/meta/meta-environment.bb |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-core/meta/meta-environment.bb +0 -0
index 9731300..3a71a9b 100644

meta/recipes-core/meta/meta-environment.bb do_install() { · line 40

Quoted 5 lines
meta/recipes-core/meta/meta-environment.bb +1 -1
@@ -37,7 +37,7 @@ do_install() {
     install -m 0644 -t ${D}/${SDKPATH} ${SDK_OUTPUT}/${SDKPATH}/*
 }
 
-PN = "meta-environment-${TRANSLATED_TARGET_ARCH}"
+PN = "meta-environment-${MACHINE}"

You should probably change PACKAGE_ARCH as well.

Quoted 10 lines
meta/recipes-core/meta/meta-environment.bb +0 -0
 PACKAGES = "${PN}"
 FILES_${PN}= " \
     ${SDKPATH}/* \
Signature · 6 lines
1.7.9.5

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#3 lore raw

Re: [PATCH 1/5] meta-environment: change package name to include MACHINE

Quoted 24 lines · Martin Jansa
Quoted 23 lines · Laurentiu Palcu

Currently, the package name contains just the TRANSLATED_TARGET_ARCH. When compiling the toolchain for two machines, belonging to the same architecture (for example: qemuarm and beagleboard), this package gets overwritten and adt-installer repo will contain just one meta-environment package... This leads to situations like installing the toolchain for qemuarm and end up with meta-environment package with beagleboard cpu options.

[YOCTO #4783]

---
 meta/recipes-core/meta/meta-environment.bb |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-core/meta/meta-environment.bb +0 -0
index 9731300..3a71a9b 100644
@@ -37,7 +37,7 @@ do_install() {
     install -m 0644 -t ${D}/${SDKPATH} ${SDK_OUTPUT}/${SDKPATH}/*
 }
 
Quoted 4 lines

meta/recipes-core/meta/meta-environment.bb do_install() { · line 40

Quoted 2 lines
meta/recipes-core/meta/meta-environment.bb +1 -1
-PN = "meta-environment-${TRANSLATED_TARGET_ARCH}"
+PN = "meta-environment-${MACHINE}"

You should probably change PACKAGE_ARCH as well.

No, this isn't the correct thing to do in this case. The PN itself is machine specific so the multiple packages can coexist in one nativesdk arch.

Cheers,

Richard