Skip to contentSkip to conversation

distrodata.bbclass/distro_check.py: Fix PN mapping

PATCH In review Beth Flanagan 2 participants 9 messages Mar 31, 2015

lore
Share & tools
lore

Beth Flanagan

author
#0 lore raw

cover distrodata.bbclass/distro_check.py: Fix PN mapping

Last night while running distrodata, I noticed some incorrect results that had to do with how we were trying to get the mapping to the base PN name of packages.

Part of this is due to how distrodata was splitting PNs to remove nativesdk-, cross-, crosssdk-, etc. This occurred on packages that had the need to strip out multiple of these strings (nativesdk-cross- for example).

The first patch ensures that in these cases we grab the distrodate from the base PN.

The final two patches support -dummy and buildtools- packages, so that they also map.

Beth Flanagan (3):
distrodata.bbclass/distro_check.py: Fix splits distrodata.bbclass/distro_check.py: Support -dummy mapping of PN distrodata.bbclass/distro_check.py: Support buildtools PN mapping

 meta/classes/distrodata.bbclass | 127 ++++++++++++++++++++++------------------
 meta/lib/oe/distro_check.py     |  54 +++++++++--------
 2 files changed, 98 insertions(+), 83 deletions(-)
Signature · 11 lines
1.9.1

-------------------------------------------------------------
Intel Ireland Limited (Branch)
Collinstown Industrial Park, Leixlip, County Kildare, Ireland
Registered Number: E902934

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
author
#1 lore raw

PATCH 1/3 distrodata.bbclass/distro_check.py: Fix splits

When we're trying to get the name of the package for nativesdk/cross/crosssdk this bit of code fails to do this right for things like nativesdk-foo-cross.

This ensures that a package like nativesdk-glibc-initial maps to glibc and not to nativesdk-glibc

Signed-off-by Beth Flanagan <[email protected]>
---
 meta/classes/distrodata.bbclass | 105 ++++++++++++++++++----------------------
 meta/lib/oe/distro_check.py     |  46 ++++++++----------
 2 files changed, 67 insertions(+), 84 deletions(-)
meta/classes/distrodata.bbclass +46 -59
index 83aa381..10e59d3 100644
@@ -25,37 +25,30 @@ python do_distrodata_np() {
         distro_check_dir = os.path.join(tmpdir, "distro_check")
         datetime = localdata.getVar('DATETIME', True)
         dist_check.update_distro_data(distro_check_dir, datetime)
+        pnstripped = pn
 
-        if pn.find("-native") != -1:
-            pnstripped = pn.split("-native")
+        if pnstripped.find("-native") != -1:
+            pnstripped = pnstripped.split("-native")[0]
             bb.note("Native Split: %s" % pnstripped)
-            localdata.setVar('OVERRIDES', "pn-" + pnstripped[0] + ":" + d.getVar('OVERRIDES', True))
-            bb.data.update_data(localdata)
 
-        if pn.find("-cross") != -1:
-            pnstripped = pn.split("-cross")
+        if pnstripped.find("-cross") != -1:
+            pnstripped = pnstripped.split("-cross")[0]
             bb.note("cross Split: %s" % pnstripped)
-            localdata.setVar('OVERRIDES', "pn-" + pnstripped[0] + ":" + d.getVar('OVERRIDES', True))
-            bb.data.update_data(localdata)
 
-        if pn.find("-crosssdk") != -1:
-            pnstripped = pn.split("-crosssdk")
-            bb.note("cross Split: %s" % pnstripped)
-            localdata.setVar('OVERRIDES', "pn-" + pnstripped[0] + ":" + d.getVar('OVERRIDES', True))
-            bb.data.update_data(localdata)
+        if pnstripped.find("-crosssdk") != -1:
+            pnstripped = pnstripped.split("-crosssdk")[0]
+            bb.note("crosssdk Split: %s" % pnstripped)
 
-        if pn.startswith("nativesdk-"):
-            pnstripped = pn.replace("nativesdk-", "")
+        if pnstripped.startswith("nativesdk-"):
+            pnstripped = pnstripped.replace("nativesdk-", "")
             bb.note("NativeSDK Split: %s" % pnstripped)
-            localdata.setVar('OVERRIDES', "pn-" + pnstripped + ":" + d.getVar('OVERRIDES', True))
-            bb.data.update_data(localdata)
-
 
-        if pn.find("-initial") != -1:
-            pnstripped = pn.split("-initial")
+        if pnstripped.find("-initial") != -1:
+            pnstripped = pnstripped.split("-initial")[0]
             bb.note("initial Split: %s" % pnstripped)
-            localdata.setVar('OVERRIDES', "pn-" + pnstripped[0] + ":" + d.getVar('OVERRIDES', True))
-            bb.data.update_data(localdata)
+
+        localdata.setVar('OVERRIDES', "pn-" + pnstripped + ":" + d.getVar('OVERRIDES', True))
+        bb.data.update_data(localdata)
 
         """generate package information from .bb file"""
         pname = localdata.getVar('PN', True)

@@ -112,36 +105,30 @@ python do_distrodata() {
 
         pn = d.getVar("PN", True)
         bb.note("Package Name: %s" % pn)
+        pnstripped = pn
 
-        if pn.find("-native") != -1:
-            pnstripped = pn.split("-native")
+        if pnstripped.find("-native") != -1:
+            pnstripped = pnstripped.split("-native")[0]
             bb.note("Native Split: %s" % pnstripped)
-            localdata.setVar('OVERRIDES', "pn-" + pnstripped[0] + ":" + d.getVar('OVERRIDES', True))
-            bb.data.update_data(localdata)
 
-        if pn.startswith("nativesdk-"):
-            pnstripped = pn.replace("nativesdk-", "")
+        if pnstripped.startswith("nativesdk-"):
+            pnstripped = pnstripped.replace("nativesdk-", "")
             bb.note("NativeSDK Split: %s" % pnstripped)
-            localdata.setVar('OVERRIDES', "pn-" + pnstripped + ":" + d.getVar('OVERRIDES', True))
-            bb.data.update_data(localdata)
 
-        if pn.find("-cross") != -1:
-            pnstripped = pn.split("-cross")
+        if pnstripped.find("-cross") != -1:
+            pnstripped = pnstripped.split("-cross")[0]
             bb.note("cross Split: %s" % pnstripped)
-            localdata.setVar('OVERRIDES', "pn-" + pnstripped[0] + ":" + d.getVar('OVERRIDES', True))
-            bb.data.update_data(localdata)
 
-        if pn.find("-crosssdk") != -1:
-            pnstripped = pn.split("-crosssdk")
-            bb.note("cross Split: %s" % pnstripped)
-            localdata.setVar('OVERRIDES', "pn-" + pnstripped[0] + ":" + d.getVar('OVERRIDES', True))
-            bb.data.update_data(localdata)
+        if pnstripped.find("-crosssdk") != -1:
+            pnstripped = pnstripped.split("-crosssdk")[0]
+            bb.note("crosssdk Split: %s" % pnstripped)
 
-        if pn.find("-initial") != -1:
-            pnstripped = pn.split("-initial")
+        if pnstripped.find("-initial") != -1:
+            pnstripped = pnstripped.split("-initial")[0]
             bb.note("initial Split: %s" % pnstripped)
-            localdata.setVar('OVERRIDES', "pn-" + pnstripped[0] + ":" + d.getVar('OVERRIDES', True))
-            bb.data.update_data(localdata)
+
+        localdata.setVar('OVERRIDES', "pn-" + pnstripped + ":" + d.getVar('OVERRIDES', True))
+        bb.data.update_data(localdata)
 
         """generate package information from .bb file"""
         pname = localdata.getVar('PN', True)

@@ -280,34 +267,34 @@ python do_checkpkg() {
 
         """generate package information from .bb file"""
         pname = d.getVar('PN', True)
+        pnstripped = pname
 
-        if pname.find("-native") != -1:
+        if pnstripped.find("-native") != -1:
             if d.getVar('BBCLASSEXTEND', True):
                     return
-            pnstripped = pname.split("-native")
+            pnstripped = pnstripped.split("-native")[0]
             bb.note("Native Split: %s" % pnstripped)
-            localdata.setVar('OVERRIDES', "pn-" + pnstripped[0] + ":" + d.getVar('OVERRIDES', True))
-            bb.data.update_data(localdata)
 
-        if pname.startswith("nativesdk-"):
+        if pnstripped.startswith("nativesdk-"):
             if d.getVar('BBCLASSEXTEND', True):
                     return
-            pnstripped = pname.replace("nativesdk-", "")
+            pnstripped = pnstripped.replace("nativesdk-", "")
             bb.note("NativeSDK Split: %s" % pnstripped)
-            localdata.setVar('OVERRIDES', "pn-" + pnstripped + ":" + d.getVar('OVERRIDES', True))
-            bb.data.update_data(localdata)
 
-        if pname.find("-cross") != -1:
-            pnstripped = pname.split("-cross")
+        if pnstripped.find("-cross") != -1:
+            pnstripped = pnstripped.split("-cross")[0]
             bb.note("cross Split: %s" % pnstripped)
-            localdata.setVar('OVERRIDES', "pn-" + pnstripped[0] + ":" + d.getVar('OVERRIDES', True))
-            bb.data.update_data(localdata)
 
-        if pname.find("-initial") != -1:
-            pnstripped = pname.split("-initial")
+        if pnstripped.find("-crosssdk") != -1:
+            pnstripped = pnstripped.split("-cross")[0]
+            bb.note("crosssdk Split: %s" % pnstripped)
+
+        if pnstripped.find("-initial") != -1:
+            pnstripped = pnstripped.split("-initial")[0]
             bb.note("initial Split: %s" % pnstripped)
-            localdata.setVar('OVERRIDES', "pn-" + pnstripped[0] + ":" + d.getVar('OVERRIDES', True))
-            bb.data.update_data(localdata)
+
+        localdata.setVar('OVERRIDES', "pn-" + pnstripped + ":" + d.getVar('OVERRIDES', True))
+        bb.data.update_data(localdata)
 
         pdesc = localdata.getVar('DESCRIPTION', True)
         pgrp = localdata.getVar('SECTION', True)
meta/lib/oe/distro_check.py +21 -25

53 lines not on this page · raw message

Signature · 11 lines
1.9.1

-------------------------------------------------------------
Intel Ireland Limited (Branch)
Collinstown Industrial Park, Leixlip, County Kildare, Ireland
Registered Number: E902934

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#2 lore raw

Re: [PATCH 1/3] distrodata.bbclass/distro_check.py: Fix splits

Quoted 2 lines · Beth Flanagan

+ if pnstripped.find("-native") != -1:

Shouldn't this be .endswith() or .startswith() to prevent false-positives?

Ross

#3 lore raw

Re: [PATCH 1/3] distrodata.bbclass/distro_check.py: Fix splits

Quoted 9 lines · Ross
Quoted 2 lines · Beth Flanagan

+ if pnstripped.find("-native") != -1:

Shouldn't this be .endswith() or .startswith() to prevent false-positives?

Also this all clearly says "put me in a function". But what you need is *almost* the BPN variable but with some prefix stripping that the BPN-logic doesn't do (BPN only has suffix stripped, oddly).

I'd argue that the BPN logic should also strip nativesdk- prefixes, so by fixing that the distrotools class could just get PN and BPN from the data store and be done with it.

Ross

author
#4 lore raw

Re: [PATCH 1/3] distrodata.bbclass/distro_check.py: Fix splits

In reply to Burton, Ross
Quoted 9 lines · Ross
Quoted 2 lines · Beth Flanagan

+ if pnstripped.find("-native") != -1:

Shouldn't this be .endswith() or .startswith() to prevent false-positives?

Not that we'd ever end up with something like foo-native-dummy (watch someone prove that wrong), but we wouldn't want .endswith() here as that would end up with bad data as well. I'm by no means happy with the solution here, but for the time being it gets packagedata and check_package working and not spitting out incorrect data.

I think we probably need to take a better look at packagedata and check_package in 1.9, especially with how we map this to the actual PN, but in this case, I was of the opinion that better was the enemy of good enough.

Quoted 12 lines

Ross

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

Signature · 3 lines
Elizabeth Flanagan
Yocto Project
Build and Release
author
#5 lore raw

PATCH 2/3 distrodata.bbclass/distro_check.py: Support -dummy mapping of PN

In reply to Beth Flanagan

Strip out -dummy in order to map to the base PN of dummy packages

Signed-off-by Beth Flanagan <[email protected]>
---
 meta/classes/distrodata.bbclass | 12 ++++++++++++
 meta/lib/oe/distro_check.py     |  7 ++++++-
 2 files changed, 18 insertions(+), 1 deletion(-)
meta/classes/distrodata.bbclass +12 -0
index 10e59d3..6562c86 100644
@@ -47,6 +47,10 @@ python do_distrodata_np() {
             pnstripped = pnstripped.split("-initial")[0]
             bb.note("initial Split: %s" % pnstripped)
 
+        if pnstripped.find("-dummy") != -1:
+            pnstripped = pnstripped.split("-dummy")
+            bb.note("dummy Split: %s" % pnstripped)
+
         localdata.setVar('OVERRIDES', "pn-" + pnstripped + ":" + d.getVar('OVERRIDES', True))
         bb.data.update_data(localdata)
 

@@ -127,6 +131,10 @@ python do_distrodata() {
             pnstripped = pnstripped.split("-initial")[0]
             bb.note("initial Split: %s" % pnstripped)
 
+        if pnstripped.find("-dummy") != -1:
+            pnstripped = pnstripped.split("-dummy")[0]
+            bb.note("dummy Split: %s" % pnstripped)
+
         localdata.setVar('OVERRIDES', "pn-" + pnstripped + ":" + d.getVar('OVERRIDES', True))
         bb.data.update_data(localdata)
 

@@ -293,6 +301,10 @@ python do_checkpkg() {
             pnstripped = pnstripped.split("-initial")[0]
             bb.note("initial Split: %s" % pnstripped)
 
+        if pnstripped.find("-dummy") != -1:
+            pnstripped = pnstripped.split("-dummy")[0]
+            bb.note("dummy Split: %s" % pnstripped)
+
         localdata.setVar('OVERRIDES', "pn-" + pnstripped + ":" + d.getVar('OVERRIDES', True))
         bb.data.update_data(localdata)
meta/lib/oe/distro_check.py +6 -1
index f754991..d13783b 100644
@@ -279,7 +279,9 @@ def compare_in_distro_packages_list(distro_check_dir, d):
     recipe_name = d.getVar('PN', True)
     bb.note("Checking: %s" % pn)
     pnstripped = pn
-    trim_dict = dict({"-native":"-native", "-cross":"-cross", "-crosssdk":"-crosssdk" ,"-initial":"-initial"})
+    trim_dict = dict({"-native":"-native", "-cross":"-cross", \
+                      "-crosssdk":"-crosssdk" ,"-initial":"-initial", \
+                      "-dummy":"-dummy"})
 
     if pnstripped.find("-native") != -1:
         pnstripped = pnstripped.split("-native")[0]

@@ -296,6 +298,9 @@ def compare_in_distro_packages_list(distro_check_dir, d):
     if pnstripped.find("-initial") != -1:
         pnstripped = pnstripped.split("-initial")[0]
 
+    if pnstripped.find("-dummy") != -1:
+        pnstripped = pnstripped.split("-dummy")[0]
+
     localdata.setVar('OVERRIDES', "pn-" + pnstripped + ":" + d.getVar('OVERRIDES', True))
 
     bb.data.update_data(localdata)
Signature · 11 lines
1.9.1

-------------------------------------------------------------
Intel Ireland Limited (Branch)
Collinstown Industrial Park, Leixlip, County Kildare, Ireland
Registered Number: E902934

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
author
#6 lore raw

PATCH 3/3 distrodata.bbclass/distro_check.py: Support buildtools PN mapping

In reply to Beth Flanagan

We need to be able to map buildtools- to the original PN.

Signed-off-by Beth Flanagan <[email protected]>
---
 meta/classes/distrodata.bbclass | 12 ++++++++++++
 meta/lib/oe/distro_check.py     |  3 +++
 2 files changed, 15 insertions(+)
meta/classes/distrodata.bbclass +12 -0
index 6562c86..f540170 100644
@@ -43,6 +43,10 @@ python do_distrodata_np() {
             pnstripped = pnstripped.replace("nativesdk-", "")
             bb.note("NativeSDK Split: %s" % pnstripped)
 
+        if pnstripped.find("buildtools-") != -1:
+            pnstripped = pnstripped.split("buildtools-")[1]
+            bb.note("buildtools Split: %s" % pnstripped)
+
         if pnstripped.find("-initial") != -1:
             pnstripped = pnstripped.split("-initial")[0]
             bb.note("initial Split: %s" % pnstripped)

@@ -119,6 +123,10 @@ python do_distrodata() {
             pnstripped = pnstripped.replace("nativesdk-", "")
             bb.note("NativeSDK Split: %s" % pnstripped)
 
+        if pnstripped.find("buildtools-") != -1:
+            pnstripped = pnstripped.split("buildtools-")[1]
+            bb.note("buildtools Split: %s" % pnstripped)
+
         if pnstripped.find("-cross") != -1:
             pnstripped = pnstripped.split("-cross")[0]
             bb.note("cross Split: %s" % pnstripped)

@@ -289,6 +297,10 @@ python do_checkpkg() {
             pnstripped = pnstripped.replace("nativesdk-", "")
             bb.note("NativeSDK Split: %s" % pnstripped)
 
+        if pnstripped.find("buildtools-") != -1:
+            pnstripped = pnstripped.split("buildtools-")[1]
+            bb.note("buildtools Split: %s" % pnstripped)
+
         if pnstripped.find("-cross") != -1:
             pnstripped = pnstripped.split("-cross")[0]
             bb.note("cross Split: %s" % pnstripped)
meta/lib/oe/distro_check.py +3 -0
index d13783b..88474d0 100644
@@ -289,6 +289,9 @@ def compare_in_distro_packages_list(distro_check_dir, d):
     if pnstripped.startswith("nativesdk-"):
         pnstripped = pnstripped.split("nativesdk-")[1]
 
+    if pnstripped.find("buildtools-") != -1:
+        pnstripped = pnstripped.split("buildtools-")[1]
+
     if pnstripped.find("-cross") != -1:
         pnstripped = pnstripped.split("-cross")[0]
Signature · 11 lines
1.9.1

-------------------------------------------------------------
Intel Ireland Limited (Branch)
Collinstown Industrial Park, Leixlip, County Kildare, Ireland
Registered Number: E902934

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#7 lore raw

Re: [PATCH 3/3] distrodata.bbclass/distro_check.py: Support buildtools PN mapping

Quoted 2 lines · Beth Flanagan

We need to be able to map buildtools- to the original PN.

What packages are prefixed buildtools-? I can only see a buildtools-tarball and nativesdk-buildtools-perl-dummy, neither of which can be considered to have a "real PN" of tarball or perl really.

Ross

author
#8 lore raw

Re: [PATCH 3/3] distrodata.bbclass/distro_check.py: Support buildtools PN mapping

Quoted 4 lines · Ross
Quoted 1 line · Beth Flanagan
Quoted 8 lines
Quoted 1 line

We need to be able to map buildtools- to the original PN.

What packages are prefixed buildtools-? I can only see a buildtools-tarball and nativesdk-buildtools-perl-dummy, neither of which can be considered to have a "real PN" of tarball or perl really.

Ross

Agreed. Looking at the two recipes that use this, I think it's probably safe to just exclude packages that have buildtools- prefixes.

-b

Quoted 11 lines

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

Signature · 3 lines
Elizabeth Flanagan
Yocto Project
Build and Release