Skip to contentSkip to conversation

Problems with ptest and package splitting

PATCH Superseded Björn Stenberg 2 participants 8 messages Mar 20, 2013 – Apr 10, 2013

lore
Share & tools
lore
#0 lore raw

Hi. I need a little help.

The ${PN}-ptest group is not added to PACKAGES by default in bitbake.conf like all the other groups. Instead it is added to PACKAGES in ptest.bbclass. This way it is only added for those recipes that implement ptest.

However it turns out this doesn't work when package splitting. Instead of putting ptest files into packages-split/${PN}-ptest, those files are put into the core ${PN} package which is obviously not what we want.

I'm scratching my head a bit about why this happens, and would like help from people with deeper bitbake knowledge. I guess the splitting is somehow done outside the package scope, but why and how do we fix it?

Putting ${PN}-ptest in PACKAGES in bitbake.conf "solves" this issue, but Richard Purdie was against that earlier since we yet have only a few "ptest enabled" packages.

Signature · 1 line
Björn
#1 lore raw

PATCH 1/1 ptest bugfix: Make all ptest files go into -ptest package

Move ${PN}-ptest to start of PACKAGES to ensure all ptest files are packaged in the -ptest package.

Add QA exclusions to insane.bbclass to ensure -ptest packages can contain any files they need.

This patch fixes bug #4069.

---
 meta/classes/insane.bbclass |    6 +++---
 meta/classes/ptest.bbclass  |    9 +--------
 2 files changed, 4 insertions(+), 11 deletions(-)
meta/classes/insane.bbclass +3 -3
index 2f10688..5170580 100644
@@ -216,7 +216,7 @@ def package_qa_check_dev(path, name, d, elf, messages):
     Check for ".so" library symlinks in non-dev packages
     """
 
-    if not name.endswith("-dev") and not name.endswith("-dbg") and not name.startswith("nativesdk-") and path.endswith(".so") and os.path.islink(path):
+    if not name.endswith("-dev") and not name.endswith("-dbg") and not name.endswith("-ptest") and not name.startswith("nativesdk-") and path.endswith(".so") and os.path.islink(path):
         messages.append("non -dev/-dbg/-nativesdk package contains symlink .so: %s path '%s'" % \
                  (name, package_qa_clean_path(path,d)))
 
@@ -229,7 +229,7 @@ def package_qa_check_staticdev(path, name, d, elf, messages):
     libgcc.a, libgcov.a will be skipped in their packages
     """
 
-    if not name.endswith("-pic") and not name.endswith("-staticdev") and path.endswith(".a") and not path.endswith("_nonshared.a"):
+    if not name.endswith("-pic") and not name.endswith("-staticdev") and not name.endswith("-ptest") and path.endswith(".a") and not path.endswith("_nonshared.a"):
         messages.append("non -staticdev package contains static .a library: %s path '%s'" % \
                  (name, package_qa_clean_path(path,d)))
 
@@ -273,7 +273,7 @@ def package_qa_check_dbg(path, name, d, elf, messages):
     Check for ".debug" files or directories outside of the dbg package
     """
 
-    if not "-dbg" in name:
+    if not "-dbg" in name and not "-ptest" in name:
         if '.debug' in path.split(os.path.sep):
             messages.append("non debug package contains .debug directory: %s path %s" % \
                      (name, package_qa_clean_path(path,d)))
meta/classes/ptest.bbclass +1 -8
index 6ab5ee4..69d2c68 100644
@@ -14,14 +14,7 @@ PTEST_ENABLED = "${@base_contains("DISTRO_FEATURES", "ptest", "1", "0", d)}"
 RDEPENDS_${PN}-ptest_virtclass-native = ""
 RDEPENDS_${PN}-ptest_virtclass-nativesdk = ""
 
-PACKAGES += "${@base_contains('DISTRO_FEATURES', 'ptest', '${PN}-ptest', '', d)}"
-
-FILES_${PN}-dbg += "${PTEST_PATH}/.debug \
-                    ${PTEST_PATH}/*/.debug \
-                    ${PTEST_PATH}/*/*/.debug \
-                    ${PTEST_PATH}/*/*/*/.debug \
-                    ${PTEST_PATH}/*/*/*/*/.debug \
-                   "
+PACKAGES =+ "${@base_contains('DISTRO_FEATURES', 'ptest', '${PN}-ptest', '', d)}"
 
 do_configure_ptest_base() {
     if [ ${PTEST_ENABLED} = 1 ]; then
Signature · 1 line
1.7.5.4
#2 lore raw

Re: [PATCH] ptest bugfix: Make all ptest files go into -ptest package

Quoted 6 lines · Björn Stenberg

Move ${PN}-ptest to start of PACKAGES to ensure all ptest files are packaged in the -ptest package.

Add QA exclusions to insane.bbclass to ensure -ptest packages can contain any files they need.

Quoted 7 lines

This patch fixes bug #4069.

---
 meta/classes/insane.bbclass |    6 +++---
 meta/classes/ptest.bbclass  |    9 +--------
 2 files changed, 4 insertions(+), 11 deletions(-)

Don't merge this, it's not fully working. Updated patch coming next week.

Signature · 1 line
Björn
#3 lore raw

PATCH v2 1/1 ptest bugfix: Make all ptest files go into -ptest package

Move ${PN}-ptest to start of PACKAGES to ensure all ptest files are packaged in the -ptest package.

Add QA exclusions to insane.bbclass to ensure -ptest packages can contain any files they need.

Bugfix: Add subshells for 'type' command.

This patch fixes bug #4069.

---
 meta/classes/insane.bbclass |    6 +++---
 meta/classes/ptest.bbclass  |   15 ++++-----------
 2 files changed, 7 insertions(+), 14 deletions(-)
meta/classes/insane.bbclass +3 -3
index 2f10688..5170580 100644
@@ -216,7 +216,7 @@ def package_qa_check_dev(path, name, d, elf, messages):
     Check for ".so" library symlinks in non-dev packages
     """
 
-    if not name.endswith("-dev") and not name.endswith("-dbg") and not name.startswith("nativesdk-") and path.endswith(".so") and os.path.islink(path):
+    if not name.endswith("-dev") and not name.endswith("-dbg") and not name.endswith("-ptest") and not name.startswith("nativesdk-") and path.endswith(".so") and os.path.islink(path):
         messages.append("non -dev/-dbg/-nativesdk package contains symlink .so: %s path '%s'" % \
                  (name, package_qa_clean_path(path,d)))
 
@@ -229,7 +229,7 @@ def package_qa_check_staticdev(path, name, d, elf, messages):
     libgcc.a, libgcov.a will be skipped in their packages
     """
 
-    if not name.endswith("-pic") and not name.endswith("-staticdev") and path.endswith(".a") and not path.endswith("_nonshared.a"):
+    if not name.endswith("-pic") and not name.endswith("-staticdev") and not name.endswith("-ptest") and path.endswith(".a") and not path.endswith("_nonshared.a"):
         messages.append("non -staticdev package contains static .a library: %s path '%s'" % \
                  (name, package_qa_clean_path(path,d)))
 
@@ -273,7 +273,7 @@ def package_qa_check_dbg(path, name, d, elf, messages):
     Check for ".debug" files or directories outside of the dbg package
     """
 
-    if not "-dbg" in name:
+    if not "-dbg" in name and not "-ptest" in name:
         if '.debug' in path.split(os.path.sep):
             messages.append("non debug package contains .debug directory: %s path %s" % \
                      (name, package_qa_clean_path(path,d)))
meta/classes/ptest.bbclass +4 -11
index 6ab5ee4..c51c4e6 100644
@@ -14,18 +14,11 @@ PTEST_ENABLED = "${@base_contains("DISTRO_FEATURES", "ptest", "1", "0", d)}"
 RDEPENDS_${PN}-ptest_virtclass-native = ""
 RDEPENDS_${PN}-ptest_virtclass-nativesdk = ""
 
-PACKAGES += "${@base_contains('DISTRO_FEATURES', 'ptest', '${PN}-ptest', '', d)}"
-
-FILES_${PN}-dbg += "${PTEST_PATH}/.debug \
-                    ${PTEST_PATH}/*/.debug \
-                    ${PTEST_PATH}/*/*/.debug \
-                    ${PTEST_PATH}/*/*/*/.debug \
-                    ${PTEST_PATH}/*/*/*/*/.debug \
-                   "
+PACKAGES =+ "${@base_contains('DISTRO_FEATURES', 'ptest', '${PN}-ptest', '', d)}"
 
 do_configure_ptest_base() {
     if [ ${PTEST_ENABLED} = 1 ]; then
-        if [ type -t do_configure_ptest = function ]; then
+        if [ $(type -t do_configure_ptest) = function ]; then
             do_configure_ptest
         fi
     fi
@@ -33,7 +26,7 @@ do_configure_ptest_base() {
 
 do_compile_ptest_base() {
     if [ ${PTEST_ENABLED} = 1 ]; then
-        if [ type -t do_compile_ptest = function ]; then
+        if [ $(type -t do_compile_ptest) = function ]; then
             do_compile_ptest
         fi
     fi
@@ -46,7 +39,7 @@ do_install_ptest_base() {
             if grep -q install-ptest: Makefile; then
                 oe_runmake DESTDIR=${D}${PTEST_PATH} install-ptest
             fi
-            if [ type -t do_install_ptest = function ]; then
+            if [ $(type -t do_install_ptest) = function ]; then
                 do_install_ptest
             fi
         fi
Signature · 1 line
1.7.5.4
#4 lore raw

Re: [PATCH v2] ptest bugfix: Make all ptest files go into -ptest package

Quoted 53 lines · Björn Stenberg

Move ${PN}-ptest to start of PACKAGES to ensure all ptest files are packaged in the -ptest package.

Add QA exclusions to insane.bbclass to ensure -ptest packages can contain any files they need.

Bugfix: Add subshells for 'type' command.

This patch fixes bug #4069.

---
 meta/classes/insane.bbclass |    6 +++---
 meta/classes/ptest.bbclass  |   15 ++++-----------
 2 files changed, 7 insertions(+), 14 deletions(-)
meta/classes/insane.bbclass +3 -3
index 2f10688..5170580 100644
@@ -216,7 +216,7 @@ def package_qa_check_dev(path, name, d, elf, messages):
     Check for ".so" library symlinks in non-dev packages
     """
 
-    if not name.endswith("-dev") and not name.endswith("-dbg") and not name.startswith("nativesdk-") and path.endswith(".so") and os.path.islink(path):
+    if not name.endswith("-dev") and not name.endswith("-dbg") and not name.endswith("-ptest") and not name.startswith("nativesdk-") and path.endswith(".so") and os.path.islink(path):
         messages.append("non -dev/-dbg/-nativesdk package contains symlink .so: %s path '%s'" % \
                  (name, package_qa_clean_path(path,d)))
 
@@ -229,7 +229,7 @@ def package_qa_check_staticdev(path, name, d, elf, messages):
     libgcc.a, libgcov.a will be skipped in their packages
     """
 
-    if not name.endswith("-pic") and not name.endswith("-staticdev") and path.endswith(".a") and not path.endswith("_nonshared.a"):
+    if not name.endswith("-pic") and not name.endswith("-staticdev") and not name.endswith("-ptest") and path.endswith(".a") and not path.endswith("_nonshared.a"):
         messages.append("non -staticdev package contains static .a library: %s path '%s'" % \
                  (name, package_qa_clean_path(path,d)))
 
@@ -273,7 +273,7 @@ def package_qa_check_dbg(path, name, d, elf, messages):
     Check for ".debug" files or directories outside of the dbg package
     """
 
-    if not "-dbg" in name:
+    if not "-dbg" in name and not "-ptest" in name:
         if '.debug' in path.split(os.path.sep):
             messages.append("non debug package contains .debug directory: %s path %s" % \
                      (name, package_qa_clean_path(path,d)))
meta/classes/ptest.bbclass +0 -0
index 6ab5ee4..c51c4e6 100644
@@ -14,18 +14,11 @@ PTEST_ENABLED = "${@base_contains("DISTRO_FEATURES", "ptest", "1", "0", d)}"
 RDEPENDS_${PN}-ptest_virtclass-native = ""

meta/classes/ptest.bbclass PTEST_ENABLED = "${@base_contains("DISTRO_FEATURES", "ptest", "1", "0", d)}" · line 24

Quoted 10 lines
meta/classes/ptest.bbclass +0 -8
 RDEPENDS_${PN}-ptest_virtclass-nativesdk = ""
 
-PACKAGES += "${@base_contains('DISTRO_FEATURES', 'ptest', '${PN}-ptest', '', d)}"
-
-FILES_${PN}-dbg += "${PTEST_PATH}/.debug \
-                    ${PTEST_PATH}/*/.debug \
-                    ${PTEST_PATH}/*/*/.debug \
-                    ${PTEST_PATH}/*/*/*/.debug \
-                    ${PTEST_PATH}/*/*/*/*/.debug \
-                   "

Why are we putting all the debug files into the -ptest package now? Does that make sense?

Cheers,

Richard

Quoted 27 lines
meta/classes/ptest.bbclass +4 -3
+PACKAGES =+ "${@base_contains('DISTRO_FEATURES', 'ptest', '${PN}-ptest', '', d)}"
 
 do_configure_ptest_base() {
     if [ ${PTEST_ENABLED} = 1 ]; then
-        if [ type -t do_configure_ptest = function ]; then
+        if [ $(type -t do_configure_ptest) = function ]; then
             do_configure_ptest
         fi
     fi
@@ -33,7 +26,7 @@ do_configure_ptest_base() {
 
 do_compile_ptest_base() {
     if [ ${PTEST_ENABLED} = 1 ]; then
-        if [ type -t do_compile_ptest = function ]; then
+        if [ $(type -t do_compile_ptest) = function ]; then
             do_compile_ptest
         fi
     fi
@@ -46,7 +39,7 @@ do_install_ptest_base() {
             if grep -q install-ptest: Makefile; then
                 oe_runmake DESTDIR=${D}${PTEST_PATH} install-ptest
             fi
-            if [ type -t do_install_ptest = function ]; then
+            if [ $(type -t do_install_ptest) = function ]; then
                 do_install_ptest
             fi
         fi
#5 lore raw

Re: [PATCH v2] ptest bugfix: Make all ptest files go into -ptest package

Quoted 2 lines · Richard Purdie

Why are we putting all the debug files into the -ptest package now? Does that make sense?

(Sorry, I missed this comment.)

My reasoning is that we want all ptest data to be contained in -ptest packages and not "leak" into other packages.

If we keep debug data in the normal -dbg packages, you will always get ptest debug data installed in your image when defining distro-feature "ptest", even if it doesn't include any -ptest packages. That feels wrong to me.

Signature · 1 line
Björn
#6 lore raw

PATCH v3 1/1 ptest bug fixes

In reply to Björn Stenberg

Move ${PN}-ptest to start of PACKAGES to ensure all ptest files are packaged in the -ptest package.

Add QA exclusions to insane.bbclass to ensure -ptest packages can contain any files they need.

Disable ptest for native packages.

---
 meta/classes/insane.bbclass |    6 +++---
 meta/classes/native.bbclass |    3 +++
 meta/classes/ptest.bbclass  |   15 ++++-----------
 3 files changed, 10 insertions(+), 14 deletions(-)
meta/classes/insane.bbclass +3 -3
index 2f10688..5170580 100644
@@ -216,7 +216,7 @@ def package_qa_check_dev(path, name, d, elf, messages):
     Check for ".so" library symlinks in non-dev packages
     """
 
-    if not name.endswith("-dev") and not name.endswith("-dbg") and not name.startswith("nativesdk-") and path.endswith(".so") and os.path.islink(path):
+    if not name.endswith("-dev") and not name.endswith("-dbg") and not name.endswith("-ptest") and not name.startswith("nativesdk-") and path.endswith(".so") and os.path.islink(path):
         messages.append("non -dev/-dbg/-nativesdk package contains symlink .so: %s path '%s'" % \
                  (name, package_qa_clean_path(path,d)))
 
@@ -229,7 +229,7 @@ def package_qa_check_staticdev(path, name, d, elf, messages):
     libgcc.a, libgcov.a will be skipped in their packages
     """
 
-    if not name.endswith("-pic") and not name.endswith("-staticdev") and path.endswith(".a") and not path.endswith("_nonshared.a"):
+    if not name.endswith("-pic") and not name.endswith("-staticdev") and not name.endswith("-ptest") and path.endswith(".a") and not path.endswith("_nonshared.a"):
         messages.append("non -staticdev package contains static .a library: %s path '%s'" % \
                  (name, package_qa_clean_path(path,d)))
 
@@ -273,7 +273,7 @@ def package_qa_check_dbg(path, name, d, elf, messages):
     Check for ".debug" files or directories outside of the dbg package
     """
 
-    if not "-dbg" in name:
+    if not "-dbg" in name and not "-ptest" in name:
         if '.debug' in path.split(os.path.sep):
             messages.append("non debug package contains .debug directory: %s path %s" % \
                      (name, package_qa_clean_path(path,d)))
meta/classes/native.bbclass +3 -0
index bfc9e4c..cd930f0 100644
@@ -50,6 +50,9 @@ TOOLCHAIN_OPTIONS = ""
 
 DEPENDS_GETTEXT = "gettext-native"
 
+# Don't build ptest natively
+PTEST_ENABLED = "0"
+
 # Don't use site files for native builds
 export CONFIG_SITE = ""
 
meta/classes/ptest.bbclass +4 -11
index 6ab5ee4..c51c4e6 100644
@@ -14,18 +14,11 @@ PTEST_ENABLED = "${@base_contains("DISTRO_FEATURES", "ptest", "1", "0", d)}"
 RDEPENDS_${PN}-ptest_virtclass-native = ""
 RDEPENDS_${PN}-ptest_virtclass-nativesdk = ""
 
-PACKAGES += "${@base_contains('DISTRO_FEATURES', 'ptest', '${PN}-ptest', '', d)}"
-
-FILES_${PN}-dbg += "${PTEST_PATH}/.debug \
-                    ${PTEST_PATH}/*/.debug \
-                    ${PTEST_PATH}/*/*/.debug \
-                    ${PTEST_PATH}/*/*/*/.debug \
-                    ${PTEST_PATH}/*/*/*/*/.debug \
-                   "
+PACKAGES =+ "${@base_contains('DISTRO_FEATURES', 'ptest', '${PN}-ptest', '', d)}"
 
 do_configure_ptest_base() {
     if [ ${PTEST_ENABLED} = 1 ]; then
-        if [ type -t do_configure_ptest = function ]; then
+        if [ $(type -t do_configure_ptest) = function ]; then
             do_configure_ptest
         fi
     fi
@@ -33,7 +26,7 @@ do_configure_ptest_base() {
 
 do_compile_ptest_base() {
     if [ ${PTEST_ENABLED} = 1 ]; then
-        if [ type -t do_compile_ptest = function ]; then
+        if [ $(type -t do_compile_ptest) = function ]; then
             do_compile_ptest
         fi
     fi
@@ -46,7 +39,7 @@ do_install_ptest_base() {
             if grep -q install-ptest: Makefile; then
                 oe_runmake DESTDIR=${D}${PTEST_PATH} install-ptest
             fi
-            if [ type -t do_install_ptest = function ]; then
+            if [ $(type -t do_install_ptest) = function ]; then
                 do_install_ptest
             fi
         fi
Signature · 1 line
1.7.5.4
#7 lore raw

PATCH v4 1/1 ptest bug fixes

In reply to Björn Stenberg

Move ${PN}-ptest to start of PACKAGES to ensure all ptest files are packaged in the -ptest package.

Add QA exclusions to insane.bbclass to ensure -ptest packages can contain any files they need.

Disable ptest for native packages.

Don't emit errors on missing _ptest functions.

---
 meta/classes/insane.bbclass |    6 +++---
 meta/classes/native.bbclass |    3 +++
 meta/classes/ptest.bbclass  |   17 +++++------------
 3 files changed, 11 insertions(+), 15 deletions(-)
meta/classes/insane.bbclass +3 -3
index 75db7a2..336beaa 100644
@@ -216,7 +216,7 @@ def package_qa_check_dev(path, name, d, elf, messages):
     Check for ".so" library symlinks in non-dev packages
     """
 
-    if not name.endswith("-dev") and not name.endswith("-dbg") and not name.startswith("nativesdk-") and path.endswith(".so") and os.path.islink(path):
+    if not name.endswith("-dev") and not name.endswith("-dbg") and not name.endswith("-ptest") and not name.startswith("nativesdk-") and path.endswith(".so") and os.path.islink(path):
         messages.append("non -dev/-dbg/-nativesdk package contains symlink .so: %s path '%s'" % \
                  (name, package_qa_clean_path(path,d)))
 
@@ -229,7 +229,7 @@ def package_qa_check_staticdev(path, name, d, elf, messages):
     libgcc.a, libgcov.a will be skipped in their packages
     """
 
-    if not name.endswith("-pic") and not name.endswith("-staticdev") and path.endswith(".a") and not path.endswith("_nonshared.a"):
+    if not name.endswith("-pic") and not name.endswith("-staticdev") and not name.endswith("-ptest") and path.endswith(".a") and not path.endswith("_nonshared.a"):
         messages.append("non -staticdev package contains static .a library: %s path '%s'" % \
                  (name, package_qa_clean_path(path,d)))
 
@@ -273,7 +273,7 @@ def package_qa_check_dbg(path, name, d, elf, messages):
     Check for ".debug" files or directories outside of the dbg package
     """
 
-    if not "-dbg" in name:
+    if not "-dbg" in name and not "-ptest" in name:
         if '.debug' in path.split(os.path.sep):
             messages.append("non debug package contains .debug directory: %s path %s" % \
                      (name, package_qa_clean_path(path,d)))
meta/classes/native.bbclass +3 -0
index bfc9e4c..cd930f0 100644
@@ -50,6 +50,9 @@ TOOLCHAIN_OPTIONS = ""
 
 DEPENDS_GETTEXT = "gettext-native"
 
+# Don't build ptest natively
+PTEST_ENABLED = "0"
+
 # Don't use site files for native builds
 export CONFIG_SITE = ""
 
meta/classes/ptest.bbclass +5 -12
index 6ab5ee4..37357e8 100644
@@ -7,25 +7,18 @@ DESCRIPTION_${PN}-ptest ?= "${DESCRIPTION}  \
 This package contains a test directory ${PTEST_PATH} for package test purposes."
 
 PTEST_PATH ?= "${libdir}/${PN}/ptest"
-FILES_${PN}-ptest = "${PTEST_PATH}/*"
+FILES_${PN}-ptest = "${PTEST_PATH}"
 SECTION_${PN}-ptest = "devel"
 ALLOW_EMPTY_${PN}-ptest = "1"
 PTEST_ENABLED = "${@base_contains("DISTRO_FEATURES", "ptest", "1", "0", d)}"
 RDEPENDS_${PN}-ptest_virtclass-native = ""
 RDEPENDS_${PN}-ptest_virtclass-nativesdk = ""
 
-PACKAGES += "${@base_contains('DISTRO_FEATURES', 'ptest', '${PN}-ptest', '', d)}"
-
-FILES_${PN}-dbg += "${PTEST_PATH}/.debug \
-                    ${PTEST_PATH}/*/.debug \
-                    ${PTEST_PATH}/*/*/.debug \
-                    ${PTEST_PATH}/*/*/*/.debug \
-                    ${PTEST_PATH}/*/*/*/*/.debug \
-                   "
+PACKAGES =+ "${@base_contains('DISTRO_FEATURES', 'ptest', '${PN}-ptest', '', d)}"
 
 do_configure_ptest_base() {
     if [ ${PTEST_ENABLED} = 1 ]; then
-        if [ type -t do_configure_ptest = function ]; then
+        if [ a$(type -t do_configure_ptest) = afunction ]; then
             do_configure_ptest
         fi
     fi
@@ -33,7 +26,7 @@ do_configure_ptest_base() {
 
 do_compile_ptest_base() {
     if [ ${PTEST_ENABLED} = 1 ]; then
-        if [ type -t do_compile_ptest = function ]; then
+        if [ a$(type -t do_compile_ptest) = afunction ]; then
             do_compile_ptest
         fi
     fi
@@ -46,7 +39,7 @@ do_install_ptest_base() {
             if grep -q install-ptest: Makefile; then
                 oe_runmake DESTDIR=${D}${PTEST_PATH} install-ptest
             fi
-            if [ type -t do_install_ptest = function ]; then
+            if [ a$(type -t do_install_ptest) = afunction ]; then
                 do_install_ptest
             fi
         fi
Signature · 1 line
1.7.10.4