Skip to contentSkip to conversation

insane.bbclass: Enhance file-rdeps QA check

RFC PATCH In review Alejandro Hernandez 3 participants 4 messages Jan 28, 2015 – Jan 30, 2015

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 insane.bbclass: Enhance file-rdeps QA check

This is a proposed solution for [YOCTO #7126]

For the first item , I simply added the symlink-to-sysroot check to the QA_WARN

this gave me the QA warning as I belive it was supposed to.

For the second item, I changed filerdepends from set() to dict(), hence methods

for adding or deleting items had to change, I believe dicts are slower than sets

but I had to keep track of the key:value relationship, since sets are unordered

a better way didnt occur to me, all this to flag the issue along with the

offending file accordingly, due to this rdep_rprovides isnt needed anymore.

---
 meta/classes/insane.bbclass | 26 +++++++++++---------------
 1 file changed, 11 insertions(+), 15 deletions(-)
meta/classes/insane.bbclass +11 -15
index 143ec46..542346a 100644
@@ -34,7 +34,7 @@ WARN_QA ?= "ldflags useless-rpaths rpaths staticdev libdir xorg-driver-abi \
 ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la \
             perms dep-cmp pkgvarcheck perm-config perm-line perm-link \
             split-strip packages-list pkgv-undefined var-undefined \
-            version-going-backwards \
+            version-going-backwards  symlink-to-sysroot\
             "
 
 ALL_QA = "${WARN_QA} ${ERROR_QA}"
@@ -604,7 +604,6 @@ def package_qa_check_symlink_to_sysroot(path, name, d, elf, messages):
             if target.startswith(tmpdir):
                 trimmed = path.replace(os.path.join (d.getVar("PKGDEST", True), name), "")
                 messages["symlink-to-sysroot"] = "Symlink %s in %s points to TMPDIR" % (trimmed, name)
-
 def package_qa_check_license(workdir, d):
     """
     Check for changes in the license files 
@@ -803,13 +802,14 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d):
             if bb.data.inherits_class('nativesdk', d):
                 ignored_file_rdeps |= set(['/bin/bash', '/usr/bin/perl'])
             # For Saving the FILERDEPENDS
-            filerdepends = set()
+            filerdepends = {}
             rdep_data = oe.packagedata.read_subpkgdata(pkg, d)
             for key in rdep_data:
                 if key.startswith("FILERDEPENDS_"):
                     for subkey in rdep_data[key].split():
-                        filerdepends.add(subkey)
-            filerdepends -= ignored_file_rdeps
+                        if subkey not in ignored_file_rdeps:
+                            # We already know it starts with FILERDEPENDS_
+                            filerdepends[subkey] = key[13:]
 
             if filerdepends:
                 next = rdepends
@@ -841,31 +841,27 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d):
                 # case there is a RDEPENDS_pkg = "python" in the recipe.
                 for py in [ d.getVar('MLPREFIX', True) + "python", "python" ]:
                     if py in done:
-                        filerdepends.discard("/usr/bin/python")
+                        filerdepends.pop("/usr/bin/python",None)
                         done.remove(py)
                 for rdep in done:
                     # For Saving the FILERPROVIDES, RPROVIDES and FILES_INFO
-                    rdep_rprovides = set()
                     rdep_data = oe.packagedata.read_subpkgdata(rdep, d)
                     for key in rdep_data:
                         if key.startswith("FILERPROVIDES_") or key.startswith("RPROVIDES_"):
                             for subkey in rdep_data[key].split():
-                                rdep_rprovides.add(subkey)
+                                filerdepends.pop(subkey,None)
                         # Add the files list to the rprovides
                         if key == "FILES_INFO":
                             # Use eval() to make it as a dict
                             for subkey in eval(rdep_data[key]):
-                                rdep_rprovides.add(subkey)
-                    filerdepends -= rdep_rprovides
+                                filerdepends.pop(subkey,None)
                     if not filerdepends:
                         # Break if all the file rdepends are met
                         break
-                    else:
-                        # Clear it for the next loop
-                        rdep_rprovides.clear()
             if filerdepends:
-                error_msg = "%s requires %s, but no providers in its RDEPENDS" % \
-                            (pkg, ', '.join(str(e) for e in filerdepends))
+                for key in filerdepends:
+                    error_msg = "%s contained in package %s requires %s, but no providers found in its RDEPENDS" % \
+                            (filerdepends[key],pkg, key)
                 sane = package_qa_handle_error("file-rdeps", error_msg, d)
 
     return sane
Signature · 1 line
1.9.1
#1 lore raw
Quoted 10 lines · Alejandro Hernandez

This is a proposed solution for [YOCTO #7126]

For the first item , I simply added the symlink-to-sysroot check to the QA_WARN

this gave me the QA warning as I belive it was supposed to.

For the second item, I changed filerdepends from set() to dict(), hence methods

for adding or deleting items had to change, I believe dicts are slower than sets

Have you done before and after builds to assess the performance differences if any?

Quoted 6 lines

but I had to keep track of the key:value relationship, since sets are unordered

a better way didnt occur to me, all this to flag the issue along with the

offending file accordingly, due to this rdep_rprovides isnt needed anymore.

What tool are you using that might cause double spacing in your commit messages.

Quoted 10 lines
---
  meta/classes/insane.bbclass | 26 +++++++++++---------------
  1 file changed, 11 insertions(+), 15 deletions(-)
meta/classes/insane.bbclass +0 -0
index 143ec46..542346a 100644

meta/classes/insane.bbclass WARN_QA ?= "ldflags useless-rpaths rpaths staticdev libdir xorg-driver-abi \ · line 39

Quoted 7 lines
meta/classes/insane.bbclass +1 -1
@@ -34,7 +34,7 @@ WARN_QA ?= "ldflags useless-rpaths rpaths staticdev libdir xorg-driver-abi \
  ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la \
              perms dep-cmp pkgvarcheck perm-config perm-line perm-link \
              split-strip packages-list pkgv-undefined var-undefined \
-            version-going-backwards \
+            version-going-backwards  symlink-to-sysroot\
              "
 

We should also add this to the poky.conf, for the Poky distro since it overrides the ERROR_QA setting in meta-yocto/distro/conf/poky.conf. That patch should go to [email protected]

Quoted 33 lines
meta/classes/insane.bbclass +5 -5
  ALL_QA = "${WARN_QA} ${ERROR_QA}"
@@ -604,7 +604,6 @@ def package_qa_check_symlink_to_sysroot(path, name, d, elf, messages):
              if target.startswith(tmpdir):
                  trimmed = path.replace(os.path.join (d.getVar("PKGDEST", True), name), "")
                  messages["symlink-to-sysroot"] = "Symlink %s in %s points to TMPDIR" % (trimmed, name)
-
  def package_qa_check_license(workdir, d):
      """
      Check for changes in the license files
@@ -803,13 +802,14 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d):
              if bb.data.inherits_class('nativesdk', d):
                  ignored_file_rdeps |= set(['/bin/bash', '/usr/bin/perl'])
              # For Saving the FILERDEPENDS
-            filerdepends = set()
+            filerdepends = {}
              rdep_data = oe.packagedata.read_subpkgdata(pkg, d)
              for key in rdep_data:
                  if key.startswith("FILERDEPENDS_"):
                      for subkey in rdep_data[key].split():
-                        filerdepends.add(subkey)
-            filerdepends -= ignored_file_rdeps
+                        if subkey not in ignored_file_rdeps:
+                            # We already know it starts with FILERDEPENDS_
+                            filerdepends[subkey] = key[13:]
 
              if filerdepends:
                  next = rdepends
@@ -841,31 +841,27 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d):
                  # case there is a RDEPENDS_pkg = "python" in the recipe.
                  for py in [ d.getVar('MLPREFIX', True) + "python", "python" ]:
                      if py in done:
-                        filerdepends.discard("/usr/bin/python")
+                        filerdepends.pop("/usr/bin/python",None)

meta/classes/insane.bbclass def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d): · line 852

Quoted 10 lines
meta/classes/insane.bbclass +1 -2
                          done.remove(py)
                  for rdep in done:
                      # For Saving the FILERPROVIDES, RPROVIDES and FILES_INFO
-                    rdep_rprovides = set()
                      rdep_data = oe.packagedata.read_subpkgdata(rdep, d)
                      for key in rdep_data:
                          if key.startswith("FILERPROVIDES_") or key.startswith("RPROVIDES_"):
                              for subkey in rdep_data[key].split():
-                                rdep_rprovides.add(subkey)
+                                filerdepends.pop(subkey,None)

Should this be a pop here since you are removing a add()?

Quoted 23 lines
meta/classes/insane.bbclass +4 -7
                          # Add the files list to the rprovides
                          if key == "FILES_INFO":
                              # Use eval() to make it as a dict
                              for subkey in eval(rdep_data[key]):
-                                rdep_rprovides.add(subkey)
-                    filerdepends -= rdep_rprovides
+                                filerdepends.pop(subkey,None)
                      if not filerdepends:
                          # Break if all the file rdepends are met
                          break
-                    else:
-                        # Clear it for the next loop
-                        rdep_rprovides.clear()
              if filerdepends:
-                error_msg = "%s requires %s, but no providers in its RDEPENDS" % \
-                            (pkg, ', '.join(str(e) for e in filerdepends))
+                for key in filerdepends:
+                    error_msg = "%s contained in package %s requires %s, but no providers found in its RDEPENDS" % \
+                            (filerdepends[key],pkg, key)
                  sane = package_qa_handle_error("file-rdeps", error_msg, d)
 
      return sane
#2 lore raw
Quoted 1 line · Saul Wold

Have you done before and after builds to assess the performance differences if any?

I checked, found no actual difference on performance.

Quoted 1 line

What tool are you using that might cause double spacing in your commit messages.

Won't be happening again.

Quoted 17 lines
Quoted 14 lines
meta/classes/insane.bbclass +1 -1
index 143ec46..542346a 100644
@@ -34,7 +34,7 @@ WARN_QA ?= "ldflags useless-rpaths rpaths staticdev 
 libdir xorg-driver-abi \
  ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig 
 la \
              perms dep-cmp pkgvarcheck perm-config perm-line 
 perm-link \
              split-strip packages-list pkgv-undefined var-undefined \
-            version-going-backwards \
+            version-going-backwards  symlink-to-sysroot\
Quoted 2 lines
              "

We should also add this to the poky.conf, for the Poky distro since it overrides the ERROR_QA setting in meta-yocto/distro/conf/poky.conf. That patch should go to [email protected]

Yes, a patch to poky is also required, I intended to send it once this gets accepted.

Quoted 15 lines
Quoted 15 lines
meta/classes/insane.bbclass +0 -0
@@ -841,31 +841,27 @@ def package_qa_check_rdepends(pkg, pkgdest, 

skip, taskdeps, packages, d):

                  # case there is a RDEPENDS_pkg = "python" in the 
recipe.
                  for py in [ d.getVar('MLPREFIX', True) + "python", 
"python" ]:
                      if py in done:
-                        filerdepends.discard("/usr/bin/python")
+ filerdepends.pop("/usr/bin/python",None)
                          done.remove(py)
                  for rdep in done:
                      # For Saving the FILERPROVIDES, RPROVIDES and 
FILES_INFO
-                    rdep_rprovides = set()
                      rdep_data = 
Quoted 8 lines

meta/classes/insane.bbclass

Quoted 7 lines

oe.packagedata.read_subpkgdata(rdep, d)
for key in rdep_data:
if key.startswith("FILERPROVIDES_") or
key.startswith("RPROVIDES_"):
for subkey in rdep_data[key].split():

meta/classes/insane.bbclass +1 -1
-                                rdep_rprovides.add(subkey)
+                                filerdepends.pop(subkey,None)

Should this be a pop here since you are removing a add()?

Yes, it should be a pop, because it was adding subkey to rdep_rprovides, which were substracted from filerdepends afterwards, this worked on sets but not on dictionaries, the same result is achieved this way.

#3 lore raw
Quoted 7 lines · Alejandro Hernandez

We should also add this to the poky.conf, for the Poky distro since it

Quoted 3 lines

overrides the ERROR_QA setting in meta-yocto/distro/conf/poky.conf. That patch should go to [email protected]

Yes, a patch to poky is also required, I intended to send it once this gets accepted.

Good news is that I'm sending a series that means we don't need to do this anymore (poky.conf edits the defaults, instead of replacing).

Ross