insane: add checking to standardize how .bbappend files do FILESEXTRAPATHS

PATCH In review Hongxu Jia 3 participants 6 messages Jul 23, 2014 – Jul 25, 2014

lore

Share & tools

lore

Hongxu Jia

author

2014-07-23 03:33 UTC #0 lore raw

v3 cover insane: add checking to standardize how .bbappend files do FILESEXTRAPATHS

Change in V3:
- Make the error message more readable. The incorrect assignment info will be helpful for issue location.

Change in V2:
- It is more reasonable to move the checking to insance.bbclass rather than create a new bbclass.

Test Steps:

1) Add git://git.yoctoproject.org/meta-fsl-ppc layer
The openssl and cryptodev-fsl incorrectly assigned FILESEXTRAPATHS which forgot the colon.(I have sent the fix to meta-fsl-ppc layer mail list)
...
FILESEXTRAPATHS_prepend := "${THISDIR}/openssl-fsl"
...
FILESEXTRAPATHS_prepend := "${THISDIR}/cryptodev-fsl"
...
2) bitbake openssl
...
ERROR: FILESEXTRAPATHS-variable, must always use _prepend (or _append) type of assignment, and don't forget the colon.
Please assign it with the format of:
FILESEXTRAPATHS_append := ":${THISDIR}/Your_Files_Path" or FILESEXTRAPATHS_prepend := "${THISDIR}/Your_Files_Path:"
in your bbappend file

Your incorrect assignment is:
/home/jiahongxu/yocto/meta-openembedded/meta-fsl-ppc/recipes-connectivity/openssl/openssl-fsl__default:

ERROR: Failed to parse recipe: /home/jiahongxu/yocto/meta-openembedded/meta-fsl-ppc/recipes-connectivity/openssl/openssl_1.0.1g.bb
...

//Hongxu

The following changes since commit 4d2ac6f6df2b3ef98699dd4f7afadb2d994222bb:

rootfs: Remove the extraneous install directory (2014-07-19 00:18:21 +0100)

are available in the git repository at:

git://git.pokylinux.org/poky-contrib hongxu/filesextrapaths
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=hongxu/filesextrapaths

Hongxu Jia (1):
insane: add checking to standardize how .bbappend files do
FILESEXTRAPATHS

 meta/classes/insane.bbclass | 13 +++++++++++++
 meta/classes/utils.bbclass  |  2 ++
 meta/conf/bitbake.conf      |  2 ++
 3 files changed, 17 insertions(+)

Signature · 1 line

1.8.1.2

Hongxu Jia

author

2014-07-23 03:33 UTC #1 lore raw

PATCH v3 1/1 insane: add checking to standardize how .bbappend files do FILESEXTRAPATHS

When adding patches or config files from bbappend files, it requires the use of FILESEXTRAPATHS, which has been an issue and failure point for people starting to work with bitbake and oe-core.

We add checking to standardize how to use FILESEXTRAPATHS. Only the format of:
FILESEXTRAPATHS_append := ":${THISDIR}/Your_Files_Path" or FILESEXTRAPATHS_prepend := "${THISDIR}/Your_Files_Path:"
is acceptable.

[YOCTO #5412]

Signed-off-by Hongxu Jia <[email protected]>

---
 meta/classes/insane.bbclass | 13 +++++++++++++
 meta/classes/utils.bbclass  |  2 ++
 meta/conf/bitbake.conf      |  2 ++
 3 files changed, 17 insertions(+)

meta/classes/insane.bbclass +13 -0

index b26216e..6b1c9ad 100644
@@ -1022,6 +1022,19 @@ python () {
     # Check various variables
     ###########################################################################
 
+    # Checking ${FILESEXTRAPATHS}
+    extrapaths = (d.getVar("FILESEXTRAPATHS", True) or "")
+    if '__default' not in extrapaths.split(":"):
+        msg = "FILESEXTRAPATHS-variable, must always use _prepend (or _append)\n"
+        msg += "type of assignment, and don't forget the colon.\n"
+        msg += "Please assign it with the format of:\n"
+        msg += "  FILESEXTRAPATHS_append := \":${THISDIR}/Your_Files_Path\" or\n"
+        msg += "  FILESEXTRAPATHS_prepend := \"${THISDIR}/Your_Files_Path:\"\n"
+        msg += "in your bbappend file\n\n"
+        msg += "Your incorrect assignment is:\n"
+        msg += "%s\n" % extrapaths
+        bb.fatal(msg)
+
     if d.getVar('do_stage', True) is not None:
         bb.fatal("Legacy staging found for %s as it has a do_stage function. This will need conversion to a do_install or often simply removal to work with OE-core" % d.getVar("FILE", True))

meta/classes/utils.bbclass +2 -0

index 0ee13e0..0f2a484 100644
@@ -312,6 +312,8 @@ def explode_deps(s):
 def base_set_filespath(path, d):
     filespath = []
     extrapaths = (d.getVar("FILESEXTRAPATHS", True) or "")
+    # Remove default flag which was used for checking
+    extrapaths = extrapaths.replace("__default:", "")
     # Don't prepend empty strings to the path list
     if extrapaths != "":
         path = extrapaths.split(":") + path

meta/conf/bitbake.conf +2 -0

index f4870d5..6a0cf7a 100644
@@ -311,6 +311,8 @@ FILES_${PN}-locale = "${datadir}/locale"
 FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE'))}"
 # FILESPATH is set in base.bbclass
 #FILESPATH = "${FILE_DIRNAME}/${PF}:${FILE_DIRNAME}/${P}:${FILE_DIRNAME}/${PN}:${FILE_DIRNAME}/${BP}:${FILE_DIRNAME}/${BPN}:${FILE_DIRNAME}/files:${FILE_DIRNAME}"
+# This default was only used for checking
+FILESEXTRAPATHS ?= "__default:"
 
 ##################################################################
 # General work and output directories for the build system.

Signature · 1 line

1.8.1.2

Gary Thomas

2014-07-25 17:25 UTC #2 lore raw

Quoted 29 lines · Hongxu Jia

When adding patches or config files from bbappend files, it requires the use of FILESEXTRAPATHS, which has been an issue and failure point for people starting to work with bitbake and oe-core.
We add checking to standardize how to use FILESEXTRAPATHS. Only the format of:
   FILESEXTRAPATHS_append := ":${THISDIR}/Your_Files_Path" or
   FILESEXTRAPATHS_prepend := "${THISDIR}/Your_Files_Path:"
is acceptable.
[YOCTO #5412]
Signed-off-by Hongxu Jia <[email protected]>
---
  meta/classes/insane.bbclass | 13 +++++++++++++
  meta/classes/utils.bbclass  |  2 ++
  meta/conf/bitbake.conf      |  2 ++
  3 files changed, 17 insertions(+)
meta/classes/insane.bbclass +2 -0
index b26216e..6b1c9ad 100644
@@ -1022,6 +1022,19 @@ python () {
      # Check various variables
      ###########################################################################
 
+    # Checking ${FILESEXTRAPATHS}
+    extrapaths = (d.getVar("FILESEXTRAPATHS", True) or "")

meta/classes/insane.bbclass python () { · line 1036

Quoted 10 lines

meta/classes/insane.bbclass +10 -0

+    if '__default' not in extrapaths.split(":"):
+        msg = "FILESEXTRAPATHS-variable, must always use _prepend (or _append)\n"
+        msg += "type of assignment, and don't forget the colon.\n"
+        msg += "Please assign it with the format of:\n"
+        msg += "  FILESEXTRAPATHS_append := \":${THISDIR}/Your_Files_Path\" or\n"
+        msg += "  FILESEXTRAPATHS_prepend := \"${THISDIR}/Your_Files_Path:\"\n"
+        msg += "in your bbappend file\n\n"
+        msg += "Your incorrect assignment is:\n"
+        msg += "%s\n" % extrapaths
+        bb.fatal(msg)

It sure would have been nice (even polite?) to make this a warning when first added. As is, it's going to break a lot of builds and it will be very tedious to fix since the parser will die on the first one it finds :-(

Quoted 31 lines

meta/classes/insane.bbclass +1 -0

+
      if d.getVar('do_stage', True) is not None:
          bb.fatal("Legacy staging found for %s as it has a do_stage function. This will need conversion to a do_install or often simply removal to work with OE-core" % d.getVar("FILE", True))

meta/classes/utils.bbclass +2 -0

index 0ee13e0..0f2a484 100644
@@ -312,6 +312,8 @@ def explode_deps(s):
  def base_set_filespath(path, d):
      filespath = []
      extrapaths = (d.getVar("FILESEXTRAPATHS", True) or "")
+    # Remove default flag which was used for checking
+    extrapaths = extrapaths.replace("__default:", "")
      # Don't prepend empty strings to the path list
      if extrapaths != "":
          path = extrapaths.split(":") + path

meta/conf/bitbake.conf +2 -0

index f4870d5..6a0cf7a 100644
@@ -311,6 +311,8 @@ FILES_${PN}-locale = "${datadir}/locale"
  FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE'))}"
  # FILESPATH is set in base.bbclass
  #FILESPATH = "${FILE_DIRNAME}/${PF}:${FILE_DIRNAME}/${P}:${FILE_DIRNAME}/${PN}:${FILE_DIRNAME}/${BP}:${FILE_DIRNAME}/${BPN}:${FILE_DIRNAME}/files:${FILE_DIRNAME}"
+# This default was only used for checking
+FILESEXTRAPATHS ?= "__default:"
 
  ##################################################################
  # General work and output directories for the build system.

Signature · 4 lines

------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Richard Purdie

2014-07-25 18:35 UTC #3 lore raw

Quoted 28 lines · Gary Thomas

Quoted 27 lines · Hongxu Jia
When adding patches or config files from bbappend files, it requires the use of FILESEXTRAPATHS, which has been an issue and failure point for people starting to work with bitbake and oe-core.
We add checking to standardize how to use FILESEXTRAPATHS. Only the format of:
   FILESEXTRAPATHS_append := ":${THISDIR}/Your_Files_Path" or
   FILESEXTRAPATHS_prepend := "${THISDIR}/Your_Files_Path:"
is acceptable.
[YOCTO #5412]
Signed-off-by Hongxu Jia <[email protected]>
---
  meta/classes/insane.bbclass | 13 +++++++++++++
  meta/classes/utils.bbclass  |  2 ++
  meta/conf/bitbake.conf      |  2 ++
  3 files changed, 17 insertions(+)
meta/classes/insane.bbclass +0 -0
index b26216e..6b1c9ad 100644
@@ -1022,6 +1022,19 @@ python () {
      # Check various variables
      ###########################################################################
 

Quoted 16 lines

meta/classes/insane.bbclass python () { · line 1036
Quoted 12 lines
meta/classes/insane.bbclass +12 -0
+    # Checking ${FILESEXTRAPATHS}
+    extrapaths = (d.getVar("FILESEXTRAPATHS", True) or "")
+    if '__default' not in extrapaths.split(":"):
+        msg = "FILESEXTRAPATHS-variable, must always use _prepend (or _append)\n"
+        msg += "type of assignment, and don't forget the colon.\n"
+        msg += "Please assign it with the format of:\n"
+        msg += "  FILESEXTRAPATHS_append := \":${THISDIR}/Your_Files_Path\" or\n"
+        msg += "  FILESEXTRAPATHS_prepend := \"${THISDIR}/Your_Files_Path:\"\n"
+        msg += "in your bbappend file\n\n"
+        msg += "Your incorrect assignment is:\n"
+        msg += "%s\n" % extrapaths
+        bb.fatal(msg)
It sure would have been nice (even polite?) to make this a warning when first added. As is, it's going to break a lot of builds and it will be very tedious to fix since the parser will die on the first one it finds :-(

Agreed, I'll change this to a warning.

Cheers,

Richard

Gary Thomas

2014-07-25 19:02 UTC #4 lore raw

Quoted 46 lines · Richard Purdie

Quoted 28 lines · Gary Thomas
Quoted 27 lines · Hongxu Jia
When adding patches or config files from bbappend files, it requires the use of FILESEXTRAPATHS, which has been an issue and failure point for people starting to work with bitbake and oe-core.
We add checking to standardize how to use FILESEXTRAPATHS. Only the format of:
    FILESEXTRAPATHS_append := ":${THISDIR}/Your_Files_Path" or
    FILESEXTRAPATHS_prepend := "${THISDIR}/Your_Files_Path:"
is acceptable.
[YOCTO #5412]
Signed-off-by Hongxu Jia <[email protected]>
---
   meta/classes/insane.bbclass | 13 +++++++++++++
   meta/classes/utils.bbclass  |  2 ++
   meta/conf/bitbake.conf      |  2 ++
   3 files changed, 17 insertions(+)
meta/classes/insane.bbclass +0 -0
index b26216e..6b1c9ad 100644
@@ -1022,6 +1022,19 @@ python () {
       # Check various variables
       ###########################################################################
 
Quoted 16 lines
meta/classes/insane.bbclass python () { · line 1036
Quoted 12 lines
meta/classes/insane.bbclass +12 -0
+    # Checking ${FILESEXTRAPATHS}
+    extrapaths = (d.getVar("FILESEXTRAPATHS", True) or "")
+    if '__default' not in extrapaths.split(":"):
+        msg = "FILESEXTRAPATHS-variable, must always use _prepend (or _append)\n"
+        msg += "type of assignment, and don't forget the colon.\n"
+        msg += "Please assign it with the format of:\n"
+        msg += "  FILESEXTRAPATHS_append := \":${THISDIR}/Your_Files_Path\" or\n"
+        msg += "  FILESEXTRAPATHS_prepend := \"${THISDIR}/Your_Files_Path:\"\n"
+        msg += "in your bbappend file\n\n"
+        msg += "Your incorrect assignment is:\n"
+        msg += "%s\n" % extrapaths
+        bb.fatal(msg)
It sure would have been nice (even polite?) to make this a warning when first added. As is, it's going to break a lot of builds and it will be very tedious to fix since the parser will die on the first one it finds :-(

Quoted 1 line

Agreed, I'll change this to a warning.

Thanks :-)

Signature · 4 lines

------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Hongxu Jia

author

2014-07-25 08:23 UTC #5 lore raw

In reply to Hongxu Jia

Ping

The fix in meta-fsl-ppc has been merged. It will not break the autobuilder.

//Hongxu

Quoted 56 lines · Hongxu Jia

Change in V3:
- Make the error message more readable. The incorrect assignment info will be helpful for issue location.
Change in V2:
- It is more reasonable to move the checking to insance.bbclass rather than create a new bbclass.
Test Steps:
1) Add git://git.yoctoproject.org/meta-fsl-ppc layer
    The openssl and cryptodev-fsl incorrectly assigned
    FILESEXTRAPATHS which forgot the colon.(I have sent the
    fix to meta-fsl-ppc layer mail list)
    ...
    FILESEXTRAPATHS_prepend := "${THISDIR}/openssl-fsl"
    ...
    FILESEXTRAPATHS_prepend := "${THISDIR}/cryptodev-fsl"
    ...
2) bitbake openssl
...
ERROR: FILESEXTRAPATHS-variable, must always use _prepend (or _append) type of assignment, and don't forget the colon.
Please assign it with the format of:
   FILESEXTRAPATHS_append := ":${THISDIR}/Your_Files_Path" or
   FILESEXTRAPATHS_prepend := "${THISDIR}/Your_Files_Path:"
in your bbappend file
Your incorrect assignment is:
/home/jiahongxu/yocto/meta-openembedded/meta-fsl-ppc/recipes-connectivity/openssl/openssl-fsl__default:
ERROR: Failed to parse recipe: /home/jiahongxu/yocto/meta-openembedded/meta-fsl-ppc/recipes-connectivity/openssl/openssl_1.0.1g.bb
...
//Hongxu
The following changes since commit 4d2ac6f6df2b3ef98699dd4f7afadb2d994222bb:
rootfs: Remove the extraneous install directory (2014-07-19 00:18:21 +0100)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib hongxu/filesextrapaths
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=hongxu/filesextrapaths
Hongxu Jia (1):
insane: add checking to standardize how .bbappend files do
FILESEXTRAPATHS
  meta/classes/insane.bbclass | 13 +++++++++++++
  meta/classes/utils.bbclass  |  2 ++
  meta/conf/bitbake.conf      |  2 ++
  3 files changed, 17 insertions(+)