Skip to contentSkip to conversation

base.bbclass: fix COMPATIBLE_MACHINE

PATCH In review Robert Yang 4 participants 12 messages Apr 10, 2016 – Apr 11, 2016

lore
Share & tools
lore

Robert Yang

author
#0 lore raw

cover base.bbclass: fix COMPATIBLE_MACHINE

The following changes since commit d2734fbfe209fcb8bd908b55e64d1ea8712d02c1:

local.conf.sample: Make it possible to override EXTRA_IMAGE_FEATURES (2016-04-09 07:56:57 +0100)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib rbt/base
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/base

Robert Yang (1):
base.bbclass: fix COMPATIBLE_MACHINE

 meta/classes/base.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Signature · 1 line
2.8.0
author
#1 lore raw

PATCH 1/1 base.bbclass: fix COMPATIBLE_MACHINE

It mismatched such as qemux86 and qemux86-64 which was incorrect, for example:
COMPATIBLE_MACHINE = "(qemux86)"
But it treated MACHINE = "qemux86-64" as matched. The similar to others.

This patch fixes the problem.

Signed-off-by Robert Yang <[email protected]>
---
 meta/classes/base.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/base.bbclass +1 -1
index f9697a9..f376478 100644
@@ -469,7 +469,7 @@ python () {
         import re
         compat_machines = (d.getVar('MACHINEOVERRIDES', True) or "").split(":")
         for m in compat_machines:
-            if re.match(need_machine, m):
+            if re.match('^' + need_machine + '$', m):
                 break
         else:
             raise bb.parse.SkipPackage("incompatible with machine %s (not in COMPATIBLE_MACHINE)" % d.getVar('MACHINE', True))
Signature · 1 line
2.8.0
#2 lore raw
Quoted 9 lines · Robert Yang

It mismatched such as qemux86 and qemux86-64 which was incorrect, for example:
COMPATIBLE_MACHINE = "(qemux86)"
But it treated MACHINE = "qemux86-64" as matched. The similar to others.

This patch fixes the problem.

Signed-off-by Robert Yang <[email protected]>

Did you verify that no recipes are in fact relying on this behavior? This variable has always been a regex, so this has always been the case. Also the ^ is unnecessary, re.match always matches at the beginning of the string, re.search is the one that does not.

author
#3 lore raw
Quoted 8 lines · Christopher Larson

On Sun, Apr 10, 2016 at 7:16 AM Robert Yang <[email protected] <mailto:[email protected]>> wrote:

It mismatched such as qemux86 and qemux86-64 which was incorrect, for example:
COMPATIBLE_MACHINE = "(qemux86)"
But it treated MACHINE = "qemux86-64" as matched. The similar to others.

Quoted 7 lines

This patch fixes the problem.

Signed-off-by Robert Yang <[email protected]

<mailto:[email protected]>>

Did you verify that no recipes are in fact relying on this behavior? This

This recipe:
meta/recipes-kernel/linux/linux-yocto-tiny_4.4.bb:
COMPATIBLE_MACHINE = "(qemux86)"

And only when MACHINE="qemux86" works well, if MACHINE="qemux86-64", there is no errors or warnings when building, but a lot of unexpected errors when running.

Quoted 1 line

variable has always been a regex, so this has always been the case. Also the ^

Ah, yes, '^' is not needed, it's a little late last night, so I was confused. Updated in the repo:

git://git.openembedded.org/openembedded-core-contrib rbt/base
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/base

Robert Yang (1):
base.bbclass: fix COMPATIBLE_MACHINE

meta/classes/base.bbclass +1 -1
index f9697a9..dc43406 100644
@@ -469,7 +469,7 @@ python () {
         import re
         compat_machines = (d.getVar('MACHINEOVERRIDES', True) or "").split(":")
         for m in compat_machines:
-            if re.match(need_machine, m):
+            if re.match(need_machine + '$', m):
                 break
         else:
             raise bb.parse.SkipPackage("incompatible with machine %s (not in

COMPATIBLE_MACHINE)" % d.getVar('MACHINE', True))

// Robert

Quoted 2 lines

is unnecessary, re.match always matches at the beginning of the string, re.search is the one that does not.

#4 lore raw
In reply to Robert Yang
Quoted 3 lines · Robert Yang

It mismatched such as qemux86 and qemux86-64 which was incorrect, for example:
COMPATIBLE_MACHINE = "(qemux86)"

That will match qemux86 and qemux86-64 and is by design! It's a regular expression, not an exact string match! There are plenty of recipes (especially outside of oe-core) that depend on this behavior, so please don't break it!

Quoted 30 lines

But it treated MACHINE = "qemux86-64" as matched. The similar to others.

This patch fixes the problem.

Signed-off-by Robert Yang <[email protected]>
---
 meta/classes/base.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/base.bbclass +1 -1
index f9697a9..f376478 100644
@@ -469,7 +469,7 @@ python () {
         import re
         compat_machines = (d.getVar('MACHINEOVERRIDES', True) or "").split(":")
         for m in compat_machines:
-            if re.match(need_machine, m):
+            if re.match('^' + need_machine + '$', m):
                 break
         else:
             raise bb.parse.SkipPackage("incompatible with machine %s (not in COMPATIBLE_MACHINE)" % d.getVar('MACHINE', True))
Signature · 7 lines
2.8.0

-- 
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
author
#5 lore raw
Quoted 6 lines · Denys Dmytriyenko
Quoted 3 lines · Robert Yang

It mismatched such as qemux86 and qemux86-64 which was incorrect, for example:
COMPATIBLE_MACHINE = "(qemux86)"

That will match qemux86 and qemux86-64 and is by design! It's a regular

I'm afraid no, please see my last reply, for others such as MACHINE_OVERRIDES, they never design to work in such a way, so I don't think that COMPATIBLE_MACHINE should work in this way. If you really want to match more, I think that you can set it as "(qemux86.*)" or something familiar.

Quoted 3 lines

expression, not an exact string match! There are plenty of recipes (especially outside of oe-core) that depend on this behavior, so please don't break it!

I checked oe-core, there isn't any wrong when set MACHINE to:
"qemuarm"
"qemuarm64"
"qemumips"
"qemumips64"
"qemuppc"
"qemux86"
"qemux86-64"

and run bitbake world -g.

Would you please tell me which recipes outside of oe-core relay on this? I think that we should fix the recipe.

// Robert

Quoted 33 lines
Quoted 32 lines

But it treated MACHINE = "qemux86-64" as matched. The similar to others.

This patch fixes the problem.

Signed-off-by Robert Yang <[email protected]>
---
  meta/classes/base.bbclass | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/base.bbclass +1 -2
index f9697a9..f376478 100644
@@ -469,7 +469,7 @@ python () {
          import re
          compat_machines = (d.getVar('MACHINEOVERRIDES', True) or "").split(":")
          for m in compat_machines:
-            if re.match(need_machine, m):
+            if re.match('^' + need_machine + '$', m):
                  break
          else:
              raise bb.parse.SkipPackage("incompatible with machine %s (not in COMPATIBLE_MACHINE)" % d.getVar('MACHINE', True))
--

2.8.0

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

#6 lore raw
Quoted 8 lines · Robert Yang
Quoted 5 lines · Denys Dmytriyenko
Quoted 3 lines · Robert Yang

It mismatched such as qemux86 and qemux86-64 which was incorrect, for example:
COMPATIBLE_MACHINE = "(qemux86)"

Quoted 8 lines
Quoted 1 line

That will match qemux86 and qemux86-64 and is by design! It's a regular

I'm afraid no, please see my last reply, for others such as MACHINE_OVERRIDES, they never design to work in such a way, so I don't think that COMPATIBLE_MACHINE should work in this way. If you really want to match more, I think that you can set it as "(qemux86.*)" or something familiar.

That's an apples and oranges comparison. MACHINEOVERRIDES is part of OVERRIDES, which has *completely* different semantics than COMPATIBLE_*. COMPATIBLE_MACHINE is a regex variable more like BBMASK than anything else, and it's been that way since we introduced it. OVERRIDES has nothing to do with regular expressions.

author
#7 lore raw
Quoted 20 lines · Christopher Larson

On Sun, Apr 10, 2016 at 6:17 PM Robert Yang <[email protected] <mailto:[email protected]>> wrote:

On 04/11/2016 03:06 AM, Denys Dmytriyenko wrote:
> On Sun, Apr 10, 2016 at 07:14:56AM -0700, Robert Yang wrote:
>> It mismatched such as qemux86 and qemux86-64 which was incorrect, for >> example:
>> COMPATIBLE_MACHINE = "(qemux86)"
>
> That will match qemux86 and qemux86-64 and is by design! It's a regular

I'm afraid no, please see my last reply, for others such as MACHINE_OVERRIDES, they never design to work in such a way, so I don't think that COMPATIBLE_MACHINE should work in this way. If you really want to match more, I think that you can set it as "(qemux86.*)" or something familiar.

Quoted 4 lines

That's an apples and oranges comparison. MACHINEOVERRIDES is part of OVERRIDES, which has *completely* different semantics than COMPATIBLE_*. COMPATIBLE_MACHINE is a regex variable more like BBMASK than anything else, and it's been that way since we introduced it. OVERRIDES has nothing to do with regular expressions.

Since introduced ? I did a grep in oe-core and meta-openembedded, it seems that no ?

In oe-core:

$ grep 'COMPATIBLE_MACHINE.*qemux86' meta/recipes* -r
linux-yocto_4.4.bb:COMPATIBLE_MACHINE = "qemuarm|qemuarm64|qemux86|qemuppc|qemumips|qemumips64|qemux86-64"
linux-yocto-tiny_4.4.bb:COMPATIBLE_MACHINE = "(qemux86)"
linux-yocto-dev.bb:COMPATIBLE_MACHINE = "(qemuarm|qemux86|qemuppc|qemumips|qemumips64|qemux86-64)"
linux-yocto-tiny_4.1.bb:COMPATIBLE_MACHINE = "(qemux86)"
linux-yocto-rt_4.1.bb:COMPATIBLE_MACHINE = "(qemux86|qemux86-64|qemuarm|qemuppc|qemumips)"
linux-yocto-rt_4.4.bb:COMPATIBLE_MACHINE = "(qemux86|qemux86-64|qemuarm|qemuppc|qemumips)"
linux-yocto_4.1.bb:COMPATIBLE_MACHINE = "qemuarm|qemuarm64|qemux86|qemuppc|qemumips|qemumips64|qemux86-64"

In meta-openembedded:
vboxguestdrivers_4.3.30.bb:COMPATIBLE_MACHINE = "(qemux86|qemux86-64)"

We can see that the only one sets qemux86 but no set qemux86-64 is linux-yocto-tiny:
COMPATIBLE_MACHINE = "(qemux86)"

And it really doesn't work with qemux86-64.

// Robert

#8 lore raw
Quoted 21 lines · Robert Yang
Quoted 20 lines · Christopher Larson

On Sun, Apr 10, 2016 at 6:17 PM Robert Yang <[email protected] <mailto:[email protected]>> wrote:

Quoted 6 lines · Denys Dmytriyenko
Quoted 3 lines · Robert Yang

It mismatched such as qemux86 and qemux86-64 which was incorrect, for example:
COMPATIBLE_MACHINE = "(qemux86)"

That will match qemux86 and qemux86-64 and is by design! It's a regular

I'm afraid no, please see my last reply, for others such as MACHINE_OVERRIDES, they never design to work in such a way, so I don't think that COMPATIBLE_MACHINE should work in this way. If you really want to match more, I think that you can set it as "(qemux86.*)" or something familiar.

Quoted 4 lines
Quoted 4 lines

That's an apples and oranges comparison. MACHINEOVERRIDES is part of OVERRIDES, which has *completely* different semantics than COMPATIBLE_*. COMPATIBLE_MACHINE is a regex variable more like BBMASK than anything else, and it's been that way since we introduced it. OVERRIDES has nothing to do with regular expressions.

I completely agree with Chris here! And Robert, what is with this "I think it should work this way" reasoning? I'm arguing with you in another thread exactly due to this, where your only explanation was "I think that one recipe should only have one -dev package, I'm not sure whether this is right or not" and hence you summarize "it should"...

Quoted 2 lines

Since introduced ? I did a grep in oe-core and meta-openembedded, it seems that no ?

Not really sure what you are trying to say here... Are you arguing that it was never used as a regular expression? I beg to differ and I can provide multiple examples for that:

http://cgit.openembedded.org/meta-openembedded/tree/meta-oe/recipes-devtools/nodejs/nodejs_4.4.1.bb#n8
http://arago-project.org/git/?p=meta-arago.git;a=blob;f=meta-arago-distro/recipes-core/images/arago-console-image.bb;hb=HEAD#l6

It was like that for 15+ years, since OpenEmbedded Classic days...

Quoted 16 lines

In oe-core:

$ grep 'COMPATIBLE_MACHINE.*qemux86' meta/recipes* -r
linux-yocto_4.4.bb:COMPATIBLE_MACHINE =
"qemuarm|qemuarm64|qemux86|qemuppc|qemumips|qemumips64|qemux86-64"
linux-yocto-tiny_4.4.bb:COMPATIBLE_MACHINE = "(qemux86)"
linux-yocto-dev.bb:COMPATIBLE_MACHINE =
"(qemuarm|qemux86|qemuppc|qemumips|qemumips64|qemux86-64)"
linux-yocto-tiny_4.1.bb:COMPATIBLE_MACHINE = "(qemux86)"
linux-yocto-rt_4.1.bb:COMPATIBLE_MACHINE =
"(qemux86|qemux86-64|qemuarm|qemuppc|qemumips)"
linux-yocto-rt_4.4.bb:COMPATIBLE_MACHINE =
"(qemux86|qemux86-64|qemuarm|qemuppc|qemumips)"
linux-yocto_4.1.bb:COMPATIBLE_MACHINE =
"qemuarm|qemuarm64|qemux86|qemuppc|qemumips|qemumips64|qemux86-64"
Quoted 6 lines

In meta-openembedded:
vboxguestdrivers_4.3.30.bb:COMPATIBLE_MACHINE = "(qemux86|qemux86-64)"

We can see that the only one sets qemux86 but no set qemux86-64 is linux-yocto-tiny:
COMPATIBLE_MACHINE = "(qemux86)"

Do you know that the world doesn't end with oe-core and meta-oe?

Quoted 1 line

And it really doesn't work with qemux86-64.

Well, then fix it by setting it to "qemu86$"

Signature · 1 line
Denys
#9 lore raw
Quoted 11 lines
Quoted 2 lines

Since introduced ? I did a grep in oe-core and meta-openembedded, it seems that no ?

Not really sure what you are trying to say here... Are you arguing that it was never used as a regular expression? I beg to differ and I can provide multiple examples for that:

http://cgit.openembedded.org/meta-openembedded/tree/meta-oe/recipes-devtools/nodejs/nodejs_4.4.1.bb#n8
http://arago-project.org/git/?p=meta-arago.git;a=blob;f=meta-arago-distro/recipes-core/images/arago-console-image.bb;hb=HEAD#l6

It was like that for 15+ years, since OpenEmbedded Classic days...

Oops, that does sound more impressive, but I meant 10+ years - my math failed me that late at night... :)

Signature · 1 line
Denys
#10 lore raw
In reply to Robert Yang
Quoted 61 lines · Robert Yang
Quoted 26 lines · Christopher Larson

On Sun, Apr 10, 2016 at 6:17 PM Robert Yang <
[email protected]
<mailto:[email protected]>> wrote:

    On 04/11/2016 03:06 AM, Denys Dmytriyenko wrote:
     > On Sun, Apr 10, 2016 at 07:14:56AM -0700, Robert Yang wrote:
     >> It mismatched such as qemux86 and qemux86-64 which was
incorrect, for
     >> example:
     >> COMPATIBLE_MACHINE = "(qemux86)"
     >
     > That will match qemux86 and qemux86-64 and is by design!

It's a regular

I'm afraid no, please see my last reply, for others such as MACHINE_OVERRIDES, they never design to work in such a way, so
I don't
think that COMPATIBLE_MACHINE should work in this way. If you
really
want to match more, I think that you can set it as
"(qemux86.*)" or
something familiar.

Quoted 8 lines

That's an apples and oranges comparison. MACHINEOVERRIDES is part of OVERRIDES,
which has *completely* different semantics than COMPATIBLE_*. COMPATIBLE_MACHINE
is a regex variable more like BBMASK than anything else, and it's been that way
since we introduced it. OVERRIDES has nothing to do with regular expressions.

Since introduced ? I did a grep in oe-core and meta-openembedded, it seems
that no ?

In oe-core:

$ grep 'COMPATIBLE_MACHINE.*qemux86' meta/recipes* -r
linux-yocto_4.4.bb:COMPATIBLE_MACHINE = 
"qemuarm|qemuarm64|qemux86|qemuppc|qemumips|qemumips64|qemux86-64"
linux-yocto-tiny_4.4.bb:COMPATIBLE_MACHINE = "(qemux86)"
linux-yocto-dev.bb:COMPATIBLE_MACHINE = 
"(qemuarm|qemux86|qemuppc|qemumips|qemumips64|qemux86-64)"
linux-yocto-tiny_4.1.bb:COMPATIBLE_MACHINE = "(qemux86)"
linux-yocto-rt_4.1.bb:COMPATIBLE_MACHINE = 
"(qemux86|qemux86-64|qemuarm|qemuppc|qemumips)"
linux-yocto-rt_4.4.bb:COMPATIBLE_MACHINE = 
"(qemux86|qemux86-64|qemuarm|qemuppc|qemumips)"
linux-yocto_4.1.bb:COMPATIBLE_MACHINE = 
"qemuarm|qemuarm64|qemux86|qemuppc|qemumips|qemumips64|qemux86-64"

In meta-openembedded:
vboxguestdrivers_4.3.30.bb:COMPATIBLE_MACHINE = "(qemux86|qemux86 -64)"

Quoted 5 lines

We can see that the only one sets qemux86 but no set qemux86-64 is linux-yocto-tiny:
COMPATIBLE_MACHINE = "(qemux86)"

And it really doesn't work with qemux86-64.

The trouble is as others have mentioned, COMPATIBLE_MACHINE has always worked this way. We can't really change that without a wider discussion and at this point in the release in particular, I can't take such a patch.

We do really need to think about the behaviour as it does do unexpected things and catch people out.

I'd suggest you add a '$' to the end of the regex in linux-yocto-tiny to resolve this. We could also add '$' to the end of the other usages and make it clear it is a regex?

Cheers,

Richard

author
#11 lore raw
Quoted 73 lines · Richard Purdie
Quoted 61 lines · Robert Yang
Quoted 26 lines · Christopher Larson

On Sun, Apr 10, 2016 at 6:17 PM Robert Yang <
[email protected]
<mailto:[email protected]>> wrote:

     On 04/11/2016 03:06 AM, Denys Dmytriyenko wrote:
      > On Sun, Apr 10, 2016 at 07:14:56AM -0700, Robert Yang wrote:
      >> It mismatched such as qemux86 and qemux86-64 which was
incorrect, for
      >> example:
      >> COMPATIBLE_MACHINE = "(qemux86)"
      >
      > That will match qemux86 and qemux86-64 and is by design!

It's a regular

I'm afraid no, please see my last reply, for others such as MACHINE_OVERRIDES, they never design to work in such a way, so
I don't
think that COMPATIBLE_MACHINE should work in this way. If you
really
want to match more, I think that you can set it as
"(qemux86.*)" or
something familiar.

Quoted 8 lines

That's an apples and oranges comparison. MACHINEOVERRIDES is part of OVERRIDES,
which has *completely* different semantics than COMPATIBLE_*. COMPATIBLE_MACHINE
is a regex variable more like BBMASK than anything else, and it's been that way
since we introduced it. OVERRIDES has nothing to do with regular expressions.

Since introduced ? I did a grep in oe-core and meta-openembedded, it seems
that no ?

In oe-core:

$ grep 'COMPATIBLE_MACHINE.*qemux86' meta/recipes* -r
linux-yocto_4.4.bb:COMPATIBLE_MACHINE =
"qemuarm|qemuarm64|qemux86|qemuppc|qemumips|qemumips64|qemux86-64"
linux-yocto-tiny_4.4.bb:COMPATIBLE_MACHINE = "(qemux86)"
linux-yocto-dev.bb:COMPATIBLE_MACHINE =
"(qemuarm|qemux86|qemuppc|qemumips|qemumips64|qemux86-64)"
linux-yocto-tiny_4.1.bb:COMPATIBLE_MACHINE = "(qemux86)"
linux-yocto-rt_4.1.bb:COMPATIBLE_MACHINE =
"(qemux86|qemux86-64|qemuarm|qemuppc|qemumips)"
linux-yocto-rt_4.4.bb:COMPATIBLE_MACHINE =
"(qemux86|qemux86-64|qemuarm|qemuppc|qemumips)"
linux-yocto_4.1.bb:COMPATIBLE_MACHINE =
"qemuarm|qemuarm64|qemux86|qemuppc|qemumips|qemumips64|qemux86-64"

In meta-openembedded:
vboxguestdrivers_4.3.30.bb:COMPATIBLE_MACHINE = "(qemux86|qemux86 -64)"

Quoted 5 lines

We can see that the only one sets qemux86 but no set qemux86-64 is linux-yocto-tiny:
COMPATIBLE_MACHINE = "(qemux86)"

And it really doesn't work with qemux86-64.

The trouble is as others have mentioned, COMPATIBLE_MACHINE has always worked this way. We can't really change that without a wider discussion and at this point in the release in particular, I can't take such a patch.

Quoted 4 lines

We do really need to think about the behaviour as it does do unexpected things and catch people out.

I'd suggest you add a '$' to the end of the regex in linux-yocto-tiny

Thanks, I will send a new patch it.

Quoted 2 lines

to resolve this. We could also add '$' to the end of the other usages and make it clear it is a regex?

I'd like to only fix linux-yocto-tiny atm in case of cause other errors, maybe we can do that for other recipes in YP 2.2.

// Robert

Quoted 7 lines

Cheers,

Richard