Skip to contentSkip to conversation

base.bbclass: Check BuildStarted for HOSTTOOLS

PATCH Superseded Robert Yang 2 participants 6 messages Feb 14, 2019

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover base.bbclass: Check BuildStarted for HOSTTOOLS

The following changes since commit 430bd28093fb743ba3bd032e11e40b4dd8e5bd1f:

linux-yocto: add baseline ARC support (2019-02-12 14:04:24 +0000)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib rbt/ht
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/ht

Robert Yang (1):
base.bbclass: Check BuildStarted for HOSTTOOLS

 meta/classes/base.bbclass | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Signature · 1 line
2.7.4
#1 lore raw

PATCH 1/1 base.bbclass: Check BuildStarted for HOSTTOOLS

There might be no bb.event.ConfigParsed event bitbake server is running, so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed when BuildStarted.

Fixed:

$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native
$ rm -fr tmp
$ bitbake quilt-native
ERROR: Error running gcc  --version: /bin/sh: gcc: command not found

This error is caused by enable_uninative(), it runs twice (ConfigParsed and BuildStarted), the error would happen when there is no ConfigParsed event (no hosttools is created), but BuildStarted. This patch can fix the problem.

[YOCTO #13022]

---
 meta/classes/base.bbclass | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
meta/classes/base.bbclass +5 -1
index f1a3c0e..283cb0f 100644
@@ -224,7 +224,11 @@ base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.MultiConfigParsed
 python base_eventhandler() {
     import bb.runqueue
 
-    if isinstance(e, bb.event.ConfigParsed):
+    # There might be no bb.event.ConfigParsed event bitbake server is running,
+    # so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed when
+    # BuildStarted.
+    if isinstance(e, bb.event.ConfigParsed) or \
+            (isinstance(e, bb.event.BuildStarted) and not os.path.exists(d.getVar('HOSTTOOLS_DIR'))):
         if not d.getVar("NATIVELSBSTRING", False):
             d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
         d.setVar('BB_VERSION', bb.__version__)
Signature · 1 line
2.7.4
#2 lore raw
Quoted 43 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Robert Yang
Sent: den 14 februari 2019 03:59
To: [email protected]
Subject: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for HOSTTOOLS

There might be no bb.event.ConfigParsed event bitbake server is running,
so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed when BuildStarted.

Fixed:

$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native
$ rm -fr tmp
$ bitbake quilt-native
ERROR: Error running gcc  --version: /bin/sh: gcc: command not found

This error is caused by enable_uninative(), it runs twice (ConfigParsed and
BuildStarted), the error would happen when there is no ConfigParsed event
(no hosttools is created), but BuildStarted. This patch can fix the problem.

[YOCTO #13022]

---
 meta/classes/base.bbclass | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
meta/classes/base.bbclass +0 -0
index f1a3c0e..283cb0f 100644
@@ -224,7 +224,11 @@ base_eventhandler[eventmask] =

"bb.event.ConfigParsed bb.event.MultiConfigParsed
python base_eventhandler() {
import bb.runqueue

meta/classes/base.bbclass base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.MultiConfigParsed · line 231

Quoted 6 lines
meta/classes/base.bbclass +5 -1
-    if isinstance(e, bb.event.ConfigParsed):
+    # There might be no bb.event.ConfigParsed event bitbake server is running,
+    # so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed when
+    # BuildStarted.
+    if isinstance(e, bb.event.ConfigParsed) or \
+            (isinstance(e, bb.event.BuildStarted) and not os.path.exists(d.getVar('HOSTTOOLS_DIR'))):

I don't think you want to do this here since not all of this if statement should be done for the BuildStarted event. I think it is better to add a separate if statement just for the HOSTTOOLS part (see below). Also, I don't think you should be checking if the HOSTTOOLS_DIR exists, because the contents of ${HOSTTOOLS} and ${HOSTTOOLS_NONFATAL} may have changed, in which case any new tools need to be added to the directory.

Quoted 5 lines

if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)
--
2.7.4

This is my suggestion for how to split the current handling of bb.event.ConfigParsed (also note that I corrected the comment before the new if statement):

if isinstance(e, bb.event.ConfigParsed):
if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)

# There might be no bb.event.ConfigParsed event if the bitbake server is # running, so check bb.event.BuildStarted too to make sure ${HOSTTOOLS_DIR} # exists.
if isinstance(e, bb.event.ConfigParsed) or isinstance(e, bb.event.BuildStarted):
# Works with the line in layer.conf which changes PATH to point here setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS', d) setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS_NONFATAL', d, fatal=False)

//Peter

#3 lore raw
Quoted 43 lines · Peter Kjellerstedt
Quoted 43 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Robert Yang
Sent: den 14 februari 2019 03:59
To: [email protected]
Subject: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for HOSTTOOLS

There might be no bb.event.ConfigParsed event bitbake server is running,
so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed when BuildStarted.

Fixed:

$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native
$ rm -fr tmp
$ bitbake quilt-native
ERROR: Error running gcc  --version: /bin/sh: gcc: command not found

This error is caused by enable_uninative(), it runs twice (ConfigParsed and
BuildStarted), the error would happen when there is no ConfigParsed event
(no hosttools is created), but BuildStarted. This patch can fix the problem.

[YOCTO #13022]

---
  meta/classes/base.bbclass | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
meta/classes/base.bbclass +0 -0
index f1a3c0e..283cb0f 100644
@@ -224,7 +224,11 @@ base_eventhandler[eventmask] =

"bb.event.ConfigParsed bb.event.MultiConfigParsed
python base_eventhandler() {
import bb.runqueue

Quoted 8 lines

meta/classes/base.bbclass base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.MultiConfigParsed · line 231

Quoted 6 lines
meta/classes/base.bbclass +5 -1
-    if isinstance(e, bb.event.ConfigParsed):
+    # There might be no bb.event.ConfigParsed event bitbake server is running,
+    # so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed when
+    # BuildStarted.
+    if isinstance(e, bb.event.ConfigParsed) or \
+            (isinstance(e, bb.event.BuildStarted) and not os.path.exists(d.getVar('HOSTTOOLS_DIR'))):

I don't think you want to do this here since not all of this if statement

That can make the code simple and avoid duplicated, the only code which may run twice is "d.setVar('BB_VERSION', bb.__version__)", I don't think it's a problem since it is very light.

Quoted 5 lines

should be done for the BuildStarted event. I think it is better to add a separate if statement just for the HOSTTOOLS part (see below). Also, I don't think you should be checking if the HOSTTOOLS_DIR exists, because the contents of ${HOSTTOOLS} and ${HOSTTOOLS_NONFATAL} may have changed, in which case any new tools need to be added to the directory.

You don't have to worry about this since bitbake server can handle it when HOSTTOOLS are changed in conf files, you can try this:

$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native

Add HOSTTOOLS += "passwd" to conf/local.conf, and:

$ bitbake quilt-native
$ ls tmp/hosttools/ | grep passwd
passwd

The passwd will be created.

Quoted 11 lines
Quoted 6 lines

if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)
--
2.7.4

This is my suggestion for how to split the current handling of bb.event.ConfigParsed (also note that I corrected the comment before the new if statement):

Quoted 6 lines

if isinstance(e, bb.event.ConfigParsed):
if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)

# There might be no bb.event.ConfigParsed event if the bitbake server is

Thanks for the correction.

Quoted 3 lines

# running, so check bb.event.BuildStarted too to make sure ${HOSTTOOLS_DIR} # exists.
if isinstance(e, bb.event.ConfigParsed) or isinstance(e, bb.event.BuildStarted):

I'm afraid that it doesn't work since it would make hosttools generate twice when bitbake sever is not running, which seems not a good idea.

// Robert

Quoted 7 lines

# Works with the line in layer.conf which changes PATH to point here setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS', d) setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS_NONFATAL', d, fatal=False)

//Peter

#4 lore raw
Quoted 64 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Robert Yang
Sent: den 14 februari 2019 04:22
To: Peter Kjellerstedt <[email protected]>; openembedded- [email protected]
Subject: Re: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for HOSTTOOLS

Quoted 2 lines · Peter Kjellerstedt
Quoted 2 lines

-----Original Message-----
From: [email protected]

<openembedded-

Quoted 9 lines
Quoted 9 lines

[email protected]> On Behalf Of Robert Yang
Sent: den 14 februari 2019 03:59
To: [email protected]
Subject: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for HOSTTOOLS

There might be no bb.event.ConfigParsed event bitbake server is running,
so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed

when

Quoted 10 lines
Quoted 10 lines

BuildStarted.

Fixed:

$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native
$ rm -fr tmp
$ bitbake quilt-native
ERROR: Error running gcc  --version: /bin/sh: gcc: command not found

This error is caused by enable_uninative(), it runs twice

(ConfigParsed

Quoted 22 lines
Quoted 22 lines

and
BuildStarted), the error would happen when there is no ConfigParsed event
(no hosttools is created), but BuildStarted. This patch can fix the problem.

[YOCTO #13022]

---
  meta/classes/base.bbclass | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
meta/classes/base.bbclass +0 -0
index f1a3c0e..283cb0f 100644
@@ -224,7 +224,11 @@ base_eventhandler[eventmask] =

"bb.event.ConfigParsed bb.event.MultiConfigParsed
python base_eventhandler() {
import bb.runqueue

Quoted 2 lines

meta/classes/base.bbclass

Quoted 2 lines
meta/classes/base.bbclass +1 -1
-    if isinstance(e, bb.event.ConfigParsed):
+    # There might be no bb.event.ConfigParsed event bitbake server

is running,

Quoted 1 line
Quoted 1 line

+ # so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR

existed when

Quoted 3 lines

meta/classes/base.bbclass base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.MultiConfigParsed

Quoted 3 lines
meta/classes/base.bbclass +3 -0
+    # BuildStarted.
+    if isinstance(e, bb.event.ConfigParsed) or \
+            (isinstance(e, bb.event.BuildStarted) and not
Quoted 8 lines

os.path.exists(d.getVar('HOSTTOOLS_DIR'))):

Quoted 2 lines

I don't think you want to do this here since not all of this if

statement

That can make the code simple and avoid duplicated, the only code which may run twice is "d.setVar('BB_VERSION', bb.__version__)", I don't think it's a problem since it is very light.

Well, true, but I still don't think you should mix them. Have one case for what only needs to be done for bb.event.ConfigParsed, one for only bb.event.BuildStarted and one for either.

Quoted 41 lines
Quoted 1 line

should be done for the BuildStarted event. I think it is better to

add a

Quoted 1 line

separate if statement just for the HOSTTOOLS part (see below). Also,

I don't

Quoted 2 lines

think you should be checking if the HOSTTOOLS_DIR exists, because the contents of ${HOSTTOOLS} and ${HOSTTOOLS_NONFATAL} may have changed,

in

Quoted 1 line

which case any new tools need to be added to the directory.

You don't have to worry about this since bitbake server can handle it when HOSTTOOLS are changed in conf files, you can try this:

$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native

Add HOSTTOOLS += "passwd" to conf/local.conf, and:

$ bitbake quilt-native
$ ls tmp/hosttools/ | grep passwd
passwd

The passwd will be created.

Quoted 11 lines
Quoted 6 lines

if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)
--
2.7.4

This is my suggestion for how to split the current handling of bb.event.ConfigParsed (also note that I corrected the comment before the new if statement):

Quoted 6 lines

if isinstance(e, bb.event.ConfigParsed):
if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)

# There might be no bb.event.ConfigParsed event if the bitbake server is

Quoted 8 lines

Thanks for the correction.

Quoted 3 lines

# running, so check bb.event.BuildStarted too to make sure ${HOSTTOOLS_DIR} # exists.
if isinstance(e, bb.event.ConfigParsed) or isinstance(e, bb.event.BuildStarted):

I'm afraid that it doesn't work since it would make hosttools generate twice when bitbake sever is not running, which seems not a good idea.

I don't think that's a problem in practice since the generation of ${HOSTTOLS_DIR} should be pretty fast if it is already populated, but I guess you could use your original if statement to avoid that:

if isinstance(e, bb.event.ConfigParsed) or \
(isinstance(e, bb.event.BuildStarted) and not os.path.exists(d.getVar('HOSTTOOLS_DIR'))):

Quoted 14 lines

// Robert

Quoted 7 lines

# Works with the line in layer.conf which changes PATH to point here setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS', d) setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS_NONFATAL', d, fatal=False)

//Peter

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

#5 lore raw
Quoted 69 lines · Peter Kjellerstedt
Quoted 69 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Robert Yang
Sent: den 14 februari 2019 04:22
To: Peter Kjellerstedt <[email protected]>; openembedded- [email protected]
Subject: Re: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for HOSTTOOLS

Quoted 2 lines · Peter Kjellerstedt
Quoted 2 lines

-----Original Message-----
From: [email protected]

<openembedded-

Quoted 9 lines
Quoted 9 lines

[email protected]> On Behalf Of Robert Yang
Sent: den 14 februari 2019 03:59
To: [email protected]
Subject: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for HOSTTOOLS

There might be no bb.event.ConfigParsed event bitbake server is running,
so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed

when

Quoted 10 lines
Quoted 10 lines

BuildStarted.

Fixed:

$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native
$ rm -fr tmp
$ bitbake quilt-native
ERROR: Error running gcc  --version: /bin/sh: gcc: command not found

This error is caused by enable_uninative(), it runs twice

(ConfigParsed

Quoted 22 lines
Quoted 22 lines

and
BuildStarted), the error would happen when there is no ConfigParsed event
(no hosttools is created), but BuildStarted. This patch can fix the problem.

[YOCTO #13022]

---
   meta/classes/base.bbclass | 6 +++++-
   1 file changed, 5 insertions(+), 1 deletion(-)
meta/classes/base.bbclass +0 -0
index f1a3c0e..283cb0f 100644
@@ -224,7 +224,11 @@ base_eventhandler[eventmask] =

"bb.event.ConfigParsed bb.event.MultiConfigParsed
python base_eventhandler() {
import bb.runqueue

Quoted 2 lines

meta/classes/base.bbclass

Quoted 2 lines
meta/classes/base.bbclass +1 -1
-    if isinstance(e, bb.event.ConfigParsed):
+    # There might be no bb.event.ConfigParsed event bitbake server

is running,

Quoted 1 line
Quoted 1 line

+ # so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR

existed when

Quoted 3 lines

meta/classes/base.bbclass base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.MultiConfigParsed

Quoted 3 lines
meta/classes/base.bbclass +3 -0
+    # BuildStarted.
+    if isinstance(e, bb.event.ConfigParsed) or \
+            (isinstance(e, bb.event.BuildStarted) and not

os.path.exists(d.getVar('HOSTTOOLS_DIR'))):

Quoted 2 lines

I don't think you want to do this here since not all of this if

statement

Quoted 7 lines
Quoted 3 lines

That can make the code simple and avoid duplicated, the only code which may run twice is "d.setVar('BB_VERSION', bb.__version__)", I don't think it's a problem since it is very light.

Well, true, but I still don't think you should mix them. Have one case for what only needs to be done for bb.event.ConfigParsed, one for only bb.event.BuildStarted and one for either.

Yeah, it's not a big problem, I will update it in V2.

Quoted 44 lines
Quoted 44 lines
Quoted 2 lines

should be done for the BuildStarted event. I think it is better to

add a

Quoted 1 line

separate if statement just for the HOSTTOOLS part (see below). Also,

I don't

Quoted 2 lines

think you should be checking if the HOSTTOOLS_DIR exists, because the contents of ${HOSTTOOLS} and ${HOSTTOOLS_NONFATAL} may have changed,

in

Quoted 1 line

which case any new tools need to be added to the directory.

You don't have to worry about this since bitbake server can handle it when HOSTTOOLS are changed in conf files, you can try this:

$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native

Add HOSTTOOLS += "passwd" to conf/local.conf, and:

$ bitbake quilt-native
$ ls tmp/hosttools/ | grep passwd
passwd

The passwd will be created.

Quoted 11 lines
Quoted 6 lines

if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)
--
2.7.4

This is my suggestion for how to split the current handling of bb.event.ConfigParsed (also note that I corrected the comment before the new if statement):

Quoted 6 lines

if isinstance(e, bb.event.ConfigParsed):
if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)

# There might be no bb.event.ConfigParsed event if the bitbake server is

Thanks for the correction.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

# running, so check bb.event.BuildStarted too to make sure ${HOSTTOOLS_DIR} # exists.
if isinstance(e, bb.event.ConfigParsed) or isinstance(e, bb.event.BuildStarted):

I'm afraid that it doesn't work since it would make hosttools generate twice when bitbake sever is not running, which seems not a good idea.

I don't think that's a problem in practice since the generation of

It may cause a race issue when tools in tmp/hosttools are being used, but BuildStarted removes them at the same time.

// Robert

Quoted 21 lines

${HOSTTOLS_DIR} should be pretty fast if it is already populated, but I guess you could use your original if statement to avoid that:

if isinstance(e, bb.event.ConfigParsed) or \
(isinstance(e, bb.event.BuildStarted) and not os.path.exists(d.getVar('HOSTTOOLS_DIR'))):

Quoted 14 lines

// Robert

Quoted 7 lines

# Works with the line in layer.conf which changes PATH to point here setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS', d) setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS_NONFATAL', d, fatal=False)

//Peter

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core