Skip to contentSkip to conversation

Robert Yang : perf: fix for rebuilding

PATCH In review Martin Jansa 4 participants 15 messages Sep 2, 2015 – Sep 22, 2015

lore
Share & tools
lore
#0 lore raw

I'm still seeing issues when perf is rebuilt

The original issue was:
OSError: [Errno 2] No such file or directory: 'BUILD/work/qemux86-webos-linux/perf/1.0-r9/perf-1.0'

as described here:
http://patches.openembedded.org/patch/94719/

First part of fix is now backported to dizzy:
commit 40aeb23c427e0c56f7073fc54ea63bf0eabbe092

Author: Robert Yang <[email protected]>
Date:   Wed Mar 11 01:35:38 2015 -0700

license.bbclass: set dirs for do_populate_lic_setscene

but perf is still failing in some builds with the same error when do_configure and do_populate_lic are executed at the same time:

NOTE: recipe perf-1.0-r9: task do_configure: Started
NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
File "bitbake/lib/bb/build.py", line 497, in exec_task
return _exec_task(fn, task, d, quieterr)
File "bitbake/lib/bb/build.py", line 440, in _exec_task
exec_func(func, localdata)
File "bitbake/lib/bb/build.py", line 212, in exec_func
exec_func_python(func, d, runfile, cwd=adir)
File "bitbake/lib/bb/build.py", line 237, in exec_func_python
os.chdir(cwd)
OSError: [Errno 2] No such file or directory: '....../perf/1.0-r9/perf-1.0'

NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 3826 (../oe-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded

So is there some 2nd/3rd part of this fix or is it broken in master?

I haven't found any other commit related to this perf configure or populate_lic in master and fido.

Regards,

Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#1 lore raw
Quoted 32 lines · Martin Jansa

I'm still seeing issues when perf is rebuilt

The original issue was:
OSError: [Errno 2] No such file or directory: 'BUILD/work/qemux86-webos-linux/perf/1.0-r9/perf-1.0'

as described here:
http://patches.openembedded.org/patch/94719/

First part of fix is now backported to dizzy:
commit 40aeb23c427e0c56f7073fc54ea63bf0eabbe092

Author: Robert Yang <[email protected]>
Date:   Wed Mar 11 01:35:38 2015 -0700

license.bbclass: set dirs for do_populate_lic_setscene

but perf is still failing in some builds with the same error when do_configure and do_populate_lic are executed at the same time:

NOTE: recipe perf-1.0-r9: task do_configure: Started
NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
File "bitbake/lib/bb/build.py", line 497, in exec_task
return _exec_task(fn, task, d, quieterr)
File "bitbake/lib/bb/build.py", line 440, in _exec_task
exec_func(func, localdata)
File "bitbake/lib/bb/build.py", line 212, in exec_func
exec_func_python(func, d, runfile, cwd=adir)
File "bitbake/lib/bb/build.py", line 237, in exec_func_python
os.chdir(cwd)
OSError: [Errno 2] No such file or directory: '....../perf/1.0-r9/perf-1.0'

Quoted 8 lines

NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 3826 (../oe-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded

So is there some 2nd/3rd part of this fix or is it broken in master?

I haven't found any other commit related to this perf configure or populate_lic in master and fido.

It's still broken in master.

Easy way to reproduce this is:
OE qemux86@ ~/build/oe-core $ for i in seq -w 1 100; do bitbake -c cleansstate perf; bitbake -c build perf | tee log.perf.$i; done

On slow machine I got first failure on 60th iteration, on much faster machine I got 4 failures in first 20 iterations.

NOTE: recipe perf-1.0-r9: task do_configure: Started
NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
ERROR: Function failed: do_configure (log file is located at /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/log.do_configure.10700)
ERROR: Logfile of failure stored in: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/log.do_configure.10700 Log data follows:

| DEBUG: Executing python function sysroot_cleansstate
| DEBUG: Python function sysroot_cleansstate finished
| DEBUG: Executing shell function do_configure
| mkdir: cannot create directory '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0/': File exists
| WARNING: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/run.do_configure.10700:1 exit 1 from
|   mkdir /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0/
| ERROR: Function failed: do_configure (log file is located at /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/log.do_configure.10700)

NOTE: recipe perf-1.0-r9: task do_configure: Failed
ERROR: Task 5 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_configure) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_populate_lic: Succeeded
NOTE: Tasks Summary: Attempted 833 tasks of which 828 didn't need to be rerun and 1 failed.

Can you please fix it properly instead of removing whole ${B}?

do_configure_prepend () {
    # Fix for rebuilding
    rm -rf ${B}/
    mkdir ${B}/

...

I wish there is easier way to undo this commit in .bbappend, we very rarely rebuild perf in the same build directory, but too often we see normal clean builds failing in perf do_configure or do_populate_lic tasks :/.

Regards and thanks for feedback (it's broken for 8 months and no reply after it was reported 3 months ago).

Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#2 lore raw

PATCH 1/1 Revert "perf: fix for rebuilding"

* This reverts commit 9dafa571ed0a40d21a886dec7704c31150b21942.

* The "fix" is causing more issues then it's fixing and there was no feedback in many months.
* More info:
http://lists.openembedded.org/pipermail/openembedded-core/2015-June/105684.html
http://lists.openembedded.org/pipermail/openembedded-core/2015-September/109870.html
http://lists.openembedded.org/pipermail/openembedded-core/2015-September/110338.html

---
 meta/recipes-kernel/perf/perf.bb | 4 ----
 1 file changed, 4 deletions(-)
meta/recipes-kernel/perf/perf.bb +0 -4
index adb3a2c..f1d7894 100644
@@ -118,10 +118,6 @@ do_install() {
 }
 
 do_configure_prepend () {
-    # Fix for rebuilding
-    rm -rf ${B}/
-    mkdir ${B}/
-
     # If building a multlib based perf, the incorrect library path will be
     # detected by perf, since it triggers via: ifeq ($(ARCH),x86_64). In a 32 bit
     # build, with a 64 bit multilib, the arch won't match and the detection of a 
Signature · 1 line
2.5.1
#3 lore raw

Re: [PATCH][master][fido][dizzy] Revert "perf: fix for rebuilding"

Hi Martin,

Sorry, I'm blocked by mutitlib SDK issues recently, I will fix it sooner. Revert the patch is fine to me if it is urgent.

// Robert

Quoted 30 lines · Martin Jansa

* This reverts commit 9dafa571ed0a40d21a886dec7704c31150b21942.

* The "fix" is causing more issues then it's fixing and there was no feedback in many months.
* More info:

   http://lists.openembedded.org/pipermail/openembedded-core/2015-June/105684.html
   http://lists.openembedded.org/pipermail/openembedded-core/2015-September/109870.html
   http://lists.openembedded.org/pipermail/openembedded-core/2015-September/110338.html
---
  meta/recipes-kernel/perf/perf.bb | 4 ----
  1 file changed, 4 deletions(-)
meta/recipes-kernel/perf/perf.bb +0 -4
index adb3a2c..f1d7894 100644
@@ -118,10 +118,6 @@ do_install() {
  }
 
  do_configure_prepend () {
-    # Fix for rebuilding
-    rm -rf ${B}/
-    mkdir ${B}/
-
      # If building a multlib based perf, the incorrect library path will be
      # detected by perf, since it triggers via: ifeq ($(ARCH),x86_64). In a 32 bit
      # build, with a 64 bit multilib, the arch won't match and the detection of a
#4 lore raw

Re: [PATCH][master][fido][dizzy] Revert "perf: fix for rebuilding"

Hi Martin,

I've figured out the reason, this is because the task's default cwd is ${B}, the easier way to fix the problem is use mkdir -p rather than mkdir, autotools.bbclass also has this problem, but I'd like to fix exec_func, I will send a patch to bitbake-devel after more testing.

// Robert

Quoted 46 lines · Robert Yang

Hi Martin,

Sorry, I'm blocked by mutitlib SDK issues recently, I will fix it sooner. Revert the patch is fine to me if it is urgent.

// Robert

Quoted 37 lines · Martin Jansa

* This reverts commit 9dafa571ed0a40d21a886dec7704c31150b21942.

* The "fix" is causing more issues then it's fixing and there was no feedback in many months.
* More info:

http://lists.openembedded.org/pipermail/openembedded-core/2015-June/105684.html

http://lists.openembedded.org/pipermail/openembedded-core/2015-September/109870.html

http://lists.openembedded.org/pipermail/openembedded-core/2015-September/110338.html

---
  meta/recipes-kernel/perf/perf.bb | 4 ----
  1 file changed, 4 deletions(-)
meta/recipes-kernel/perf/perf.bb +0 -4
index adb3a2c..f1d7894 100644
@@ -118,10 +118,6 @@ do_install() {
  }
 
  do_configure_prepend () {
-    # Fix for rebuilding
-    rm -rf ${B}/
-    mkdir ${B}/
-
      # If building a multlib based perf, the incorrect library path will be
      # detected by perf, since it triggers via: ifeq ($(ARCH),x86_64). In a

32 bit
# build, with a 64 bit multilib, the arch won't match and the detection
of a

#5 lore raw

Re: [PATCH][master][fido][dizzy] Revert "perf: fix for rebuilding"

Quoted 5 lines · Robert Yang

I've figured out the reason, this is because the task's default cwd is ${B}, the easier way to fix the problem is use mkdir -p rather than mkdir, autotools.bbclass also has this problem, but I'd like to fix exec_func, I will send a patch to bitbake-devel after more testing.

Changing the default $B of [dirs] is not something we'll be doing post-M3. I've a branch that changes that default and have fixed all the obvious breakage in oe-core already so this is something on my plan for immediately after 2.1 branches off.

Ross

#6 lore raw

Re: [PATCH][master][fido][dizzy] Revert "perf: fix for rebuilding"

Quoted 11 lines · Ross

On 16 September 2015 at 08:48, Robert Yang <[email protected] <mailto:[email protected]>> wrote:

I've figured out the reason, this is because the task's default cwd is ${B}, the easier way to fix the problem is use mkdir -p rather than mkdir, autotools.bbclass also has this problem, but I'd like to fix exec_func, I will send a patch to bitbake-devel after more testing.

Changing the default $B of [dirs] is not something we'll be doing post-M3. I've

Thanks, so let's simply use "mkdir -p" to fix the problem, atm ?

Quoted 3 lines

a branch that changes that default and have fixed all the obvious breakage in oe-core already so this is something on my plan for immediately after 2.1 branches off.

I made 2 patches just now, one is for bitbake, and one for insane.bbclass,
maybe we are doing the similar things, I can drop them then.

build.py: default exec_func's cwd to WORKDIR

This can fix a few problems:
- The ${B} was nearly always created in the past after any tak runs, for example, the ${B} exists after do_clean or do_cleansstate (first removed, then created), but ${B} is useless and confused end user in this case, the similar to a lot of prefuncs and postfuncs which also create ${B}, but not used. This patch fixes the problem.

- This can fix race issue when we use the following commands in other tasks:
rm -rf ${B}
mkdir ${B}
such as autotools.bbclass and perf.bb.

When the 'dirs' or 'cleandirs' is not specified by task, which means that they are not important, so default to WORKDIR which is more common and usually existed.

bitbake/lib/bb/build.py +3 -2
index 948c395..a413c35 100644
@@ -182,8 +182,9 @@ def exec_func(func, d, dirs = None):
             bb.utils.mkdirhier(adir)
         adir = dirs[-1]
     else:
-        adir = d.getVar('B', True)
-        bb.utils.mkdirhier(adir)
+        adir = d.getVar('WORKDIR', True)
+        if not os.path.exists(adir):
+            bb.utils.mkdirhier(adir)
 
     ispython = flags.get('python')
 

===================

insane.bbclass: only do_qa_unpack warn when SRC_URI is not null

If SRC_URI is null, then no source is needed to be unpacked, thus no warn.

meta/classes/insane.bbclass +1 -1
index 5c8629a..e22e8a0 100644
@@ -1214,7 +1214,7 @@ python do_qa_unpack() {
     bb.note("Checking has ${S} been created")
 
     s_dir = d.getVar('S', True)
-    if not os.path.exists(s_dir):
+    if not os.path.exists(s_dir) and d.getVar('SRC_URI', True):
         bb.warn('%s: the directory %s (%s) pointed to by the S variable 

doesn\'t exist - please set S within the recipe to point to where the source has been unpacked to' % (d.getVar('PN', T
}

// Robert

Quoted 2 lines

Ross

#7 lore raw

Re: [PATCH][master][fido][dizzy] Revert "perf: fix for rebuilding"

In reply to Robert Yang
Quoted 7 lines · Robert Yang

Hi Martin,

I've figured out the reason, this is because the task's default cwd is ${B}, the easier way to fix the problem is use mkdir -p rather than mkdir, autotools.bbclass also has this problem, but I'd like to fix exec_func, I will send a patch to bitbake-devel after more testing.

Unfortunately "mkdir -p" fixes only part of this problem, it still fails quite often when bitbake tries to "os.chdir(cwd)" in populate_lic after do_configure removed ${B} and before do_configure re-created it with "mkdir -p".

So 3 from 50 iterations (if I use the same for cycle to test this) fail with:

NOTE: recipe perf-1.0-r9: task do_configure: Started
NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 553, in exec_task
return _exec_task(fn, task, d, quieterr)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 493, in _exec_task
exec_func(func, localdata)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 227, in exec_func
exec_func_python(func, d, runfile, cwd=adir)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 252, in exec_func_python
os.chdir(cwd)
OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'

NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded

or even longer exception:

NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
NOTE: recipe perf-1.0-r9: task do_configure: Started
ERROR: Error executing a python function in /OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb:

The stack trace of python calls that resulted in this exception/failure was:
File: 'sstate_task_postfunc', lineno: 14, function: <module>

     0010:    sstate_package(shared_state, d)
     0011:    os.umask(omask)
     0012:
     0013:
 *** 0014:sstate_task_postfunc(d)
     0015:
File: 'sstate_task_postfunc', lineno: 4, function: sstate_task_postfunc
     0001:
     0002:def sstate_task_postfunc(d):
     0003:    shared_state = sstate_state_fromvars(d)

*** 0004: sstate_install(shared_state, d)
0005: for intercept in shared_state['interceptfuncs']:
0006: bb.build.exec_func(intercept, d)
0007: omask = os.umask(002)
0008: if omask != 002:
File: 'sstate.bbclass', lineno: 113, function: sstate_install

     0109:        if os.path.exists(state[1]):
     0110:            oe.path.copyhardlinktree(state[1], state[2])
     0111:
     0112:    for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split():
 *** 0113:        bb.build.exec_func(postinst, d)
     0114:
     0115:    for lock in locks:
     0116:        bb.utils.unlockfile(lock)
     0117:

File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 227, function: exec_func
0223: pass
0224:
0225: with bb.utils.fileslocked(lockfiles):
0226: if ispython:
*** 0227: exec_func_python(func, d, runfile, cwd=adir)

    0228:        else:
     0229:            exec_func_shell(func, d, runfile, cwd=adir)
     0230:
     0231:_functionfmt = """

File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 252, function: exec_func_python
0248: try:
0249: olddir = os.getcwd()
0250: except OSError:
0251: olddir = None
*** 0252: os.chdir(cwd)

     0253:
     0254:    bb.debug(2, "Executing python function %s" % func)
     0255:
     0256:    try:

Exception: OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'

ERROR: Function failed: sstate_task_postfunc
ERROR: Logfile of failure stored in: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/log.do_populate_lic.16588
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded
NOTE: Tasks Summary: Attempted 840 tasks of which 835 didn't need to be rerun and 1 failed.

So this revert (or more complete fix) is still needed in master, fido and dizzy.

Regards,

Quoted 50 lines

// Robert

Quoted 46 lines · Robert Yang

Hi Martin,

Sorry, I'm blocked by mutitlib SDK issues recently, I will fix it sooner. Revert the patch is fine to me if it is urgent.

// Robert

Quoted 37 lines · Martin Jansa

* This reverts commit 9dafa571ed0a40d21a886dec7704c31150b21942.

* The "fix" is causing more issues then it's fixing and there was no feedback in many months.
* More info:

http://lists.openembedded.org/pipermail/openembedded-core/2015-June/105684.html

http://lists.openembedded.org/pipermail/openembedded-core/2015-September/109870.html

http://lists.openembedded.org/pipermail/openembedded-core/2015-September/110338.html

---
  meta/recipes-kernel/perf/perf.bb | 4 ----
  1 file changed, 4 deletions(-)
meta/recipes-kernel/perf/perf.bb +0 -4
index adb3a2c..f1d7894 100644
@@ -118,10 +118,6 @@ do_install() {
  }
 
  do_configure_prepend () {
-    # Fix for rebuilding
-    rm -rf ${B}/
-    mkdir ${B}/
-
      # If building a multlib based perf, the incorrect library path will be
      # detected by perf, since it triggers via: ifeq ($(ARCH),x86_64). In a

32 bit
# build, with a 64 bit multilib, the arch won't match and the detection
of a

Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#8 lore raw

RFC PATCH 1/1 perf: don't re-create ${B}

* otherwise there is race-condition between do_configure rm+mkdir and bitbake trying to use it as CWD for do_populate_lic task, which results in errors like this:

NOTE: recipe perf-1.0-r9: task do_configure: Started
NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 553, in exec_task
return _exec_task(fn, task, d, quieterr)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 493, in _exec_task
exec_func(func, localdata)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 227, in exec_func
exec_func_python(func, d, runfile, cwd=adir)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 252, in exec_func_python
os.chdir(cwd)
OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded

or even longer exception:

NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
NOTE: recipe perf-1.0-r9: task do_configure: Started
ERROR: Error executing a python function in /OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb:

The stack trace of python calls that resulted in this exception/failure was:
File: 'sstate_task_postfunc', lineno: 14, function: <module>

     0010:    sstate_package(shared_state, d)
     0011:    os.umask(omask)
     0012:
     0013:
 *** 0014:sstate_task_postfunc(d)
     0015:
File: 'sstate_task_postfunc', lineno: 4, function: sstate_task_postfunc
     0001:
     0002:def sstate_task_postfunc(d):
     0003:    shared_state = sstate_state_fromvars(d)

*** 0004: sstate_install(shared_state, d)
0005: for intercept in shared_state['interceptfuncs']:
0006: bb.build.exec_func(intercept, d)
0007: omask = os.umask(002)
0008: if omask != 002:
File: 'sstate.bbclass', lineno: 113, function: sstate_install

     0109:        if os.path.exists(state[1]):
     0110:            oe.path.copyhardlinktree(state[1], state[2])
     0111:
     0112:    for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split():
 *** 0113:        bb.build.exec_func(postinst, d)
     0114:
     0115:    for lock in locks:
     0116:        bb.utils.unlockfile(lock)
     0117:

File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 227, function: exec_func
0223: pass
0224:
0225: with bb.utils.fileslocked(lockfiles):
0226: if ispython:
*** 0227: exec_func_python(func, d, runfile, cwd=adir)

    0228:        else:
     0229:            exec_func_shell(func, d, runfile, cwd=adir)
     0230:
     0231:_functionfmt = """

File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 252, function: exec_func_python
0248: try:
0249: olddir = os.getcwd()
0250: except OSError:
0251: olddir = None
*** 0252: os.chdir(cwd)

     0253:
     0254:    bb.debug(2, "Executing python function %s" % func)
     0255:
     0256:    try:

Exception: OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'

ERROR: Function failed: sstate_task_postfunc
ERROR: Logfile of failure stored in: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/log.do_populate_lic.16588
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded
NOTE: Tasks Summary: Attempted 840 tasks of which 835 didn't need to be rerun and 1 failed.

---
 meta/recipes-kernel/perf/perf.bb | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
meta/recipes-kernel/perf/perf.bb +1 -2
index 22bd3c8..7f5c3b3 100644
@@ -119,8 +119,7 @@ do_install() {
 
 do_configure_prepend () {
     # Fix for rebuilding
-    rm -rf ${B}/
-    mkdir -p ${B}/
+    find ${B} -mindepth 1 -maxdepth 1 -delete
 
     # If building a multlib based perf, the incorrect library path will be
     # detected by perf, since it triggers via: ifeq ($(ARCH),x86_64). In a 32 bit
Signature · 1 line
2.5.1
#9 lore raw

RFC PATCH v2 1/1 cmake.bbclass, perf: don't re-create ${B}

In reply to Martin Jansa

* otherwise there is race-condition between do_configure rm+mkdir and bitbake trying to use it as CWD for do_populate_lic task, which results in errors like this:

NOTE: recipe perf-1.0-r9: task do_configure: Started
NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 553, in exec_task
return _exec_task(fn, task, d, quieterr)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 493, in _exec_task
exec_func(func, localdata)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 227, in exec_func
exec_func_python(func, d, runfile, cwd=adir)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 252, in exec_func_python
os.chdir(cwd)
OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded

or even longer exception:

NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
NOTE: recipe perf-1.0-r9: task do_configure: Started
ERROR: Error executing a python function in /OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb:

The stack trace of python calls that resulted in this exception/failure was:
File: 'sstate_task_postfunc', lineno: 14, function: <module>

     0010:    sstate_package(shared_state, d)
     0011:    os.umask(omask)
     0012:
     0013:
 *** 0014:sstate_task_postfunc(d)
     0015:
File: 'sstate_task_postfunc', lineno: 4, function: sstate_task_postfunc
     0001:
     0002:def sstate_task_postfunc(d):
     0003:    shared_state = sstate_state_fromvars(d)

*** 0004: sstate_install(shared_state, d)
0005: for intercept in shared_state['interceptfuncs']:
0006: bb.build.exec_func(intercept, d)
0007: omask = os.umask(002)
0008: if omask != 002:
File: 'sstate.bbclass', lineno: 113, function: sstate_install

     0109:        if os.path.exists(state[1]):
     0110:            oe.path.copyhardlinktree(state[1], state[2])
     0111:
     0112:    for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split():
 *** 0113:        bb.build.exec_func(postinst, d)
     0114:
     0115:    for lock in locks:
     0116:        bb.utils.unlockfile(lock)
     0117:

File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 227, function: exec_func
0223: pass
0224:
0225: with bb.utils.fileslocked(lockfiles):
0226: if ispython:
*** 0227: exec_func_python(func, d, runfile, cwd=adir)

    0228:        else:
     0229:            exec_func_shell(func, d, runfile, cwd=adir)
     0230:
     0231:_functionfmt = """

File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 252, function: exec_func_python
0248: try:
0249: olddir = os.getcwd()
0250: except OSError:
0251: olddir = None
*** 0252: os.chdir(cwd)

     0253:
     0254:    bb.debug(2, "Executing python function %s" % func)
     0255:
     0256:    try:

Exception: OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'

ERROR: Function failed: sstate_task_postfunc
ERROR: Logfile of failure stored in: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/log.do_populate_lic.16588
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded
NOTE: Tasks Summary: Attempted 840 tasks of which 835 didn't need to be rerun and 1 failed.

---
 meta/classes/cmake.bbclass       | 3 +--
 meta/recipes-kernel/perf/perf.bb | 3 +--
 2 files changed, 2 insertions(+), 4 deletions(-)
meta/classes/cmake.bbclass +1 -2
index ae3cc02..e31799a 100644
@@ -91,8 +91,7 @@ cmake_do_configure() {
 	fi
 
 	if [ "${S}" != "${B}" ]; then
-		rm -rf ${B}
-		mkdir -p ${B}
+		find ${B} -mindepth 1 -maxdepth 1 | xargs rm -rf
 		cd ${B}
 	else
 		find ${B} -name CMakeFiles -or -name Makefile -or -name cmake_install.cmake -or -name CMakeCache.txt -delete
meta/recipes-kernel/perf/perf.bb +1 -2
index 22bd3c8..a5a04af 100644
@@ -119,8 +119,7 @@ do_install() {
 
 do_configure_prepend () {
     # Fix for rebuilding
-    rm -rf ${B}/
-    mkdir -p ${B}/
+    find ${B} -mindepth 1 -maxdepth 1 | xargs rm -rf
 
     # If building a multlib based perf, the incorrect library path will be
     # detected by perf, since it triggers via: ifeq ($(ARCH),x86_64). In a 32 bit
Signature · 1 line
2.5.1
#10 lore raw

Re: [RFC][PATCHv2] cmake.bbclass, perf: don't re-create ${B}

Quoted 4 lines · Martin Jansa

* otherwise there is race-condition between do_configure rm+mkdir and bitbake trying to use it as CWD for do_populate_lic task, which results in errors like this:

It took me a few minutes to understand this race:

Task A executing perf:do_configure and deleting ${B}
Task B finishing perf:do_populate_lic and so running the sstate generation logic and the functions listed in SSTATEPOSTINSTFUNCS, which get executed with the default cwd of ${B}.

I'll note that autotools.bbclass also does rmdir/mkdir ${B} so I'm curious why you see it more in perf and cmake and not everything else.

Would a better long-term fix be to pass cwd=${WORKDIR} to exec_func in the sstate code to stop the functions being executed in ${B} (after vetting the functions, obviously). FWIW I'm the owner of the bug to stop bitbake knowing about ${B} which is leading to exactly this sort of problem.

Ross

#11 lore raw

Re: [RFC][PATCHv2] cmake.bbclass, perf: don't re-create ${B}

Quoted 7 lines · Ross
Quoted 4 lines · Martin Jansa

* otherwise there is race-condition between do_configure rm+mkdir and bitbake trying to use it as CWD for do_populate_lic task, which results in errors like this:

Quoted 6 lines

It took me a few minutes to understand this race:

Task A executing perf:do_configure and deleting ${B}
Task B finishing perf:do_populate_lic and so running the sstate generation logic and the functions listed in SSTATEPOSTINSTFUNCS, which get executed with the default cwd of ${B}.

Thanks for more detailed explanation, sorry if I wasn't clear on this one in my previous e-mails.

Quoted 2 lines

I'll note that autotools.bbclass also does rmdir/mkdir ${B} so I'm curious why you see it more in perf and cmake and not everything else.

perf for some reason shows this most often, most our internal components are using cmake, that's probably cmake was easier to find in our logs, but I didn't grep all of them, maybe there are similar exceptions in autotools based components. I'll send v3 with autotools.bbclass fix as well.

Quoted 4 lines

Would a better long-term fix be to pass cwd=${WORKDIR} to exec_func in the sstate code to stop the functions being executed in ${B} (after vetting the functions, obviously). FWIW I'm the owner of the bug to stop bitbake knowing about ${B} which is leading to exactly this sort of problem.

Yes, I agree it would be better, but as you said, such change is too late even for master branch and I need to fix this long-standing issue even in fido and dizzy, so smaller, simpler change will be easier to get backported.

My for cycle finished 469 iterations over night and no exception since
this fix:
OE @ ~/build/oe-core $ grep "ERROR: Task" log.perf.00*
log.perf.0003:ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1' log.perf.0004:ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1' log.perf.0031:ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1' log.perf.0055:ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1' log.perf.0062:ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'

Regards,

Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#12 lore raw

Re: [RFC][PATCHv2] cmake.bbclass, perf: don't re-create ${B}

In reply to Martin Jansa
Quoted 45 lines · Martin Jansa

* otherwise there is race-condition between do_configure rm+mkdir and bitbake trying to use it as CWD for do_populate_lic task, which results in errors like this:

NOTE: recipe perf-1.0-r9: task do_configure: Started
NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 553, in exec_task
return _exec_task(fn, task, d, quieterr)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 493, in _exec_task
exec_func(func, localdata)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 227, in exec_func
exec_func_python(func, d, runfile, cwd=adir)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 252, in exec_func_python
os.chdir(cwd)
OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded

or even longer exception:

NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
NOTE: recipe perf-1.0-r9: task do_configure: Started
ERROR: Error executing a python function in /OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb:

The stack trace of python calls that resulted in this exception/failure was:
File: 'sstate_task_postfunc', lineno: 14, function: <module>

     0010:    sstate_package(shared_state, d)
     0011:    os.umask(omask)
     0012:
     0013:
 *** 0014:sstate_task_postfunc(d)
     0015:
File: 'sstate_task_postfunc', lineno: 4, function: sstate_task_postfunc
     0001:
     0002:def sstate_task_postfunc(d):
     0003:    shared_state = sstate_state_fromvars(d)

*** 0004: sstate_install(shared_state, d)
0005: for intercept in shared_state['interceptfuncs']:
0006: bb.build.exec_func(intercept, d)
0007: omask = os.umask(002)
0008: if omask != 002:
File: 'sstate.bbclass', lineno: 113, function: sstate_install

Quoted 9 lines
     0109:        if os.path.exists(state[1]):
     0110:            oe.path.copyhardlinktree(state[1], state[2])
     0111:
     0112:    for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split():
 *** 0113:        bb.build.exec_func(postinst, d)
     0114:
     0115:    for lock in locks:
     0116:        bb.utils.unlockfile(lock)
     0117:

The longer stack trace is actually really helpful. I believe the code here should inherit the working directory of the current task rather than rely on the default of ${B} which bitbake assigns if there is nothing else.

The trouble is that if the function does specify a dirs flag, it should really use that instead of a supplied parameter and if we use the parameter to exec_func(), it won't do that.

There aren't that many uses of SSTATEPOSTINSTFUNCS in our metadata so the easiest way to fix this short term is probably to go through them and add specific [dirs] options to the functions.

Cheers,

Richard

#13 lore raw

Re: [RFC][PATCHv2] cmake.bbclass, perf: don't re-create ${B}

Quoted 5 lines · Richard Purdie

The longer stack trace is actually really helpful. I believe the code here should inherit the working directory of the current task rather than rely on the default of ${B} which bitbake assigns if there is nothing else.

Quoted 8 lines

The trouble is that if the function does specify a dirs flag, it should really use that instead of a supplied parameter and if we use the parameter to exec_func(), it won't do that.

There aren't that many uses of SSTATEPOSTINSTFUNCS in our metadata so the easiest way to fix this short term is probably to go through them and add specific [dirs] options to the functions.

There's enough instances of SSTATEPOSTINSTFUNCS dotted around even just oe-core, but they're all going to be using SSTATE_INSTDIR as the working directory. Whilst you have a valid point that the functions should respect [dirs] if set, for the case of sstate post-install hooks I think we should just force the right path so the recipe creators don't have to worry about this.

Ross

#14 lore raw

RFC PATCH v3 1/1 autotools.bbclass, cmake.bbclass, perf: don't re-create ${B}

In reply to Martin Jansa

* otherwise there is race-condition between do_configure rm+mkdir and bitbake trying to use it as CWD for do_populate_lic task, which results in errors like this:

NOTE: recipe perf-1.0-r9: task do_configure: Started
NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 553, in exec_task
return _exec_task(fn, task, d, quieterr)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 493, in _exec_task
exec_func(func, localdata)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 227, in exec_func
exec_func_python(func, d, runfile, cwd=adir)
File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 252, in exec_func_python
os.chdir(cwd)
OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded

or even longer exception:

NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
NOTE: recipe perf-1.0-r9: task do_configure: Started
ERROR: Error executing a python function in /OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb:

The stack trace of python calls that resulted in this exception/failure was:
File: 'sstate_task_postfunc', lineno: 14, function: <module>

     0010:    sstate_package(shared_state, d)
     0011:    os.umask(omask)
     0012:
     0013:
 *** 0014:sstate_task_postfunc(d)
     0015:
File: 'sstate_task_postfunc', lineno: 4, function: sstate_task_postfunc
     0001:
     0002:def sstate_task_postfunc(d):
     0003:    shared_state = sstate_state_fromvars(d)

*** 0004: sstate_install(shared_state, d)
0005: for intercept in shared_state['interceptfuncs']:
0006: bb.build.exec_func(intercept, d)
0007: omask = os.umask(002)
0008: if omask != 002:
File: 'sstate.bbclass', lineno: 113, function: sstate_install

     0109:        if os.path.exists(state[1]):
     0110:            oe.path.copyhardlinktree(state[1], state[2])
     0111:
     0112:    for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split():
 *** 0113:        bb.build.exec_func(postinst, d)
     0114:
     0115:    for lock in locks:
     0116:        bb.utils.unlockfile(lock)
     0117:

File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 227, function: exec_func
0223: pass
0224:
0225: with bb.utils.fileslocked(lockfiles):
0226: if ispython:
*** 0227: exec_func_python(func, d, runfile, cwd=adir)

    0228:        else:
     0229:            exec_func_shell(func, d, runfile, cwd=adir)
     0230:
     0231:_functionfmt = """

File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 252, function: exec_func_python
0248: try:
0249: olddir = os.getcwd()
0250: except OSError:
0251: olddir = None
*** 0252: os.chdir(cwd)

     0253:
     0254:    bb.debug(2, "Executing python function %s" % func)
     0255:
     0256:    try:

Exception: OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'

ERROR: Function failed: sstate_task_postfunc
ERROR: Logfile of failure stored in: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/log.do_populate_lic.16588
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded
NOTE: Tasks Summary: Attempted 840 tasks of which 835 didn't need to be rerun and 1 failed.

---
 meta/classes/autotools.bbclass   | 3 +--
 meta/classes/cmake.bbclass       | 3 +--
 meta/recipes-kernel/perf/perf.bb | 3 +--
 3 files changed, 3 insertions(+), 6 deletions(-)
meta/classes/autotools.bbclass +1 -2
index 819045a..67e1e3a 100644
@@ -104,8 +104,7 @@ autotools_preconfigure() {
 		if [ "`cat ${CONFIGURESTAMPFILE}`" != "${BB_TASKHASH}" ]; then
 			if [ "${S}" != "${B}" ]; then
 				echo "Previously configured separate build directory detected, cleaning ${B}"
-				rm -rf ${B}
-				mkdir -p ${B}
+				find ${B} -mindepth 1 -maxdepth 1 | xargs rm -rf
 			else
 				# At least remove the .la files since automake won't automatically
 				# regenerate them even if CFLAGS/LDFLAGS are different
meta/classes/cmake.bbclass +1 -2
index ae3cc02..e31799a 100644
@@ -91,8 +91,7 @@ cmake_do_configure() {
 	fi
 
 	if [ "${S}" != "${B}" ]; then
-		rm -rf ${B}
-		mkdir -p ${B}
+		find ${B} -mindepth 1 -maxdepth 1 | xargs rm -rf
 		cd ${B}
 	else
 		find ${B} -name CMakeFiles -or -name Makefile -or -name cmake_install.cmake -or -name CMakeCache.txt -delete
meta/recipes-kernel/perf/perf.bb +1 -2
index 22bd3c8..a5a04af 100644
@@ -119,8 +119,7 @@ do_install() {
 
 do_configure_prepend () {
     # Fix for rebuilding
-    rm -rf ${B}/
-    mkdir -p ${B}/
+    find ${B} -mindepth 1 -maxdepth 1 | xargs rm -rf
 
     # If building a multlib based perf, the incorrect library path will be
     # detected by perf, since it triggers via: ifeq ($(ARCH),x86_64). In a 32 bit
Signature · 1 line
2.5.1