Skip to contentSkip to conversation

Fixes for addtask and deltask

PATCH Superseded Robert Yang 6 participants 14 messages Apr 25, 2019 – May 5, 2019

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Fixes for addtask and deltask

The following changes since commit 7f7a7cba4dcd99d7b36a0d747bfd1bba3f9c2288:

bitbake: bb: siggen: Print more info when basehash are mis-matched (2019-04-23 23:30:45 +0100)

are available in the git repository at:

git://git.pokylinux.org/poky-contrib rbt/addtask
http://git.pokylinux.org/cgit.cgi//log/?h=rbt/addtask

Robert Yang (2):
bitbake: BBHandler: Fix addtask and deltask
bitbake: build.py: check dependendent task for addtask

 bitbake/lib/bb/build.py                    |  3 +++
 bitbake/lib/bb/parse/parse_py/BBHandler.py | 25 ++++++++++++++++++++++++-
 2 files changed, 27 insertions(+), 1 deletion(-)
Signature · 1 line
2.7.4
#1 lore raw

PATCH 1/2 bitbake: BBHandler: Fix addtask and deltask

The following commands are not supported, but they were ignored silently, that may suprise users:

* addtask task1 task2
task2 is ignored

* addtask task1 before task2 before task3
Should be: addtask task1 before task2 task3

* addtask task1 after task2 after task3
Should be: addtask task1 after task2 task3

* deltask task1 task2
task2 is ignore

This patch can check and warn for them.

[YOCTO #13282]

---
 bitbake/lib/bb/parse/parse_py/BBHandler.py | 25 ++++++++++++++++++++++++-
 1 file changed, 24 insertions(+), 1 deletion(-)
bitbake/lib/bb/parse/parse_py/BBHandler.py +24 -1
index 9dba5f2..4d90ff0 100644
@@ -42,7 +42,7 @@ __func_start_regexp__    = re.compile(r"(((?P<py>python)|(?P<fr>fakeroot))\s*)*(
 __inherit_regexp__       = re.compile(r"inherit\s+(.+)" )
 __export_func_regexp__   = re.compile(r"EXPORT_FUNCTIONS\s+(.+)" )
 __addtask_regexp__       = re.compile(r"addtask\s+(?P<func>\w+)\s*((before\s*(?P<before>((.*(?=after))|(.*))))|(after\s*(?P<after>((.*(?=before))|(.*)))))*")
-__deltask_regexp__       = re.compile(r"deltask\s+(?P<func>\w+)")
+__deltask_regexp__       = re.compile(r"deltask\s+(?P<func>\w+)(?P<ignores>.*)")
 __addhandler_regexp__    = re.compile(r"addhandler\s+(.+)" )
 __def_regexp__           = re.compile(r"def\s+(\w+).*:" )
 __python_func_regexp__   = re.compile(r"(\s+.*)|(^$)|(^#)" )
@@ -236,11 +236,34 @@ def feeder(lineno, s, fn, root, statements, eof=False):
 
     m = __addtask_regexp__.match(s)
     if m:
+        if len(m.group().split()) == 2:
+            # Check and warn for "addtask task1 task2"
+            m2 = re.match(r"addtask\s+(?P<func>\w+)(?P<ignores>.*)", s)
+            if m2 and m2.group('ignores'):
+                logger.warning('addtask ignored: "%s"' % m2.group('ignores'))
+
+        # Check and warn for "addtask task1 before task2 before task3", the
+        # similar to "after"
+        dup_after = 0
+        dup_before = 0
+        for tmp in s.split():
+            if 'before' == tmp:
+                dup_before += 1
+            if 'after' == tmp:
+                dup_after += 1
+        if dup_after > 1:
+            logger.warning('addtask found more than 1 keyword "after": "%s"' % s)
+        if dup_before > 1:
+            logger.warning('addtask found more than 1 keyword "before": "%s"' % s)
+
         ast.handleAddTask(statements, fn, lineno, m)
         return
 
     m = __deltask_regexp__.match(s)
     if m:
+        # Check and warn "for deltask task1 task2"
+        if m.group('ignores'):
+            logger.warning('deltask ignored: "%s"' % m.group('ignores'))
         ast.handleDelTask(statements, fn, lineno, m)
         return
 
Signature · 1 line
2.7.4
#4 lore raw

PATCH 2/2 bitbake: build.py: check dependendent task for addtask

In reply to Robert Yang

The following command is incorrect, but was ignored silently, that may suprise users:

addtask task after task_not_existed

This patch can check and warn for it. It would be better to also check "before" tasks, but there is no easier way to do it.

[YOCTO #13282]

---
 bitbake/lib/bb/build.py | 3 +++
 1 file changed, 3 insertions(+)
bitbake/lib/bb/build.py +3 -0
index 7571421..861e9a9 100644
@@ -815,6 +815,9 @@ def add_tasks(tasklist, d):
         task_deps['parents'][task] = []
         if 'deps' in flags:
             for dep in flags['deps']:
+                # Check and warn for "addtask task after foo" while foo does not exist
+                if not dep in tasklist:
+                    bb.warn('%s: dependent task %s does not exist!' % (d.getVar('PN'), dep))
                 dep = d.expand(dep)
                 task_deps['parents'][task].append(dep)
 
Signature · 1 line
2.7.4
#2 lore raw

Re: [PATCH 1/2] bitbake: BBHandler: Fix addtask and deltask

In reply to Robert Yang
Quoted 59 lines · Robert Yang

The following commands are not supported, but they were ignored silently, that
may suprise users:

* addtask task1 task2
task2 is ignored

* addtask task1 before task2 before task3
Should be: addtask task1 before task2 task3

* addtask task1 after task2 after task3
Should be: addtask task1 after task2 task3

* deltask task1 task2
task2 is ignore

This patch can check and warn for them.

[YOCTO #13282]

---
 bitbake/lib/bb/parse/parse_py/BBHandler.py | 25
++++++++++++++++++++++++-
 1 file changed, 24 insertions(+), 1 deletion(-)
bitbake/lib/bb/parse/parse_py/BBHandler.py +0 -0
index 9dba5f2..4d90ff0 100644
@@ -42,7 +42,7 @@ __func_start_regexp__    =

re.compile(r"(((?P<py>python)|(?P<fr>fakeroot))\s*)*(

 __inherit_regexp__       = re.compile(r"inherit\s+(.+)" )
 __export_func_regexp__   = re.compile(r"EXPORT_FUNCTIONS\s+(.+)" )
 __addtask_regexp__       =

re.compile(r"addtask\s+(?P<func>\w+)\s*((before\s*(?P<before>((.*(?=a fter))|(.*))))|(after\s*(?P<after>((.*(?=before))|(.*)))))*")

bitbake/lib/bb/parse/parse_py/BBHandler.py +1 -1
-__deltask_regexp__       = re.compile(r"deltask\s+(?P<func>\w+)")
+__deltask_regexp__       =

re.compile(r"deltask\s+(?P<func>\w+)(?P<ignores>.*)")

 __addhandler_regexp__    = re.compile(r"addhandler\s+(.+)" )
 __def_regexp__           = re.compile(r"def\s+(\w+).*:" )
 __python_func_regexp__   = re.compile(r"(\s+.*)|(^$)|(^#)" )
bitbake/lib/bb/parse/parse_py/BBHandler.py +0 -0
@@ -236,11 +236,34 @@ def feeder(lineno, s, fn, root, statements,

eof=False):

     m = __addtask_regexp__.match(s)
     if m:
bitbake/lib/bb/parse/parse_py/BBHandler.py +3 -0
+        if len(m.group().split()) == 2:
+            # Check and warn for "addtask task1 task2"
+            m2 = re.match(r"addtask\s+(?P<func>\w+)(?P<ignores>.*)",

s)

bitbake/lib/bb/parse/parse_py/BBHandler.py +2 -0
+            if m2 and m2.group('ignores'):
+                logger.warning('addtask ignored: "%s"' %

m2.group('ignores'))

bitbake/lib/bb/parse/parse_py/BBHandler.py +2 -0
+
+        # Check and warn for "addtask task1 before task2 before

task3", the

bitbake/lib/bb/parse/parse_py/BBHandler.py def feeder(lineno, s, fn, root, statements, eof=False): · line 254

Quoted 9 lines
bitbake/lib/bb/parse/parse_py/BBHandler.py +9 -0
+        # similar to "after"
+        dup_after = 0
+        dup_before = 0
+        for tmp in s.split():
+            if 'before' == tmp:
+                dup_before += 1
+            if 'after' == tmp:
+                dup_after += 1
+        if dup_after > 1:

taskexpression = s.split()
if taskexpression.count("after") > 1:

logger.warning("X")
if taskexpression.count("before") > 1:

logger.warning("Y")

Quoted 2 lines

+ logger.warning('addtask found more than 1 keyword
"after": "%s"' % s)

How about:

addtask contained multiple 'after' keywords, only one is supported

Quoted 3 lines
bitbake/lib/bb/parse/parse_py/BBHandler.py +2 -0
+        if dup_before > 1:
+            logger.warning('addtask found more than 1 keyword

"before": "%s"' % s)

and:

addtask contained multiple 'before' keywords, only one is supported

Quoted 9 lines

ast.handleAddTask(statements, fn, lineno, m) return

m = __deltask_regexp__.match(s)
if m:

bitbake/lib/bb/parse/parse_py/BBHandler.py +3 -0
+        # Check and warn "for deltask task1 task2"
+        if m.group('ignores'):
+            logger.warning('deltask ignored: "%s"' %

m.group('ignores'))

deltask listing multiple tasks is not supported

I did wonder if we should support multiple tasks for deltask?

Also, we might want to consider adding something to lib/bb/tests/ for this?

Cheers,

Richard

#5 lore raw

Re: [PATCH 2/2] bitbake: build.py: check dependendent task for addtask

In reply to Robert Yang
Quoted 20 lines · Robert Yang

The following command is incorrect, but was ignored silently, that may suprise users:

addtask task after task_not_existed

This patch can check and warn for it. It would be better to also check "before" tasks, but there is no easier way to do it.

[YOCTO #13282]

---
 bitbake/lib/bb/build.py | 3 +++
 1 file changed, 3 insertions(+)
bitbake/lib/bb/build.py +0 -0
index 7571421..861e9a9 100644

bitbake/lib/bb/build.py def add_tasks(tasklist, d): · line 822

Quoted 8 lines
bitbake/lib/bb/build.py +3 -0
@@ -815,6 +815,9 @@ def add_tasks(tasklist, d):
         task_deps['parents'][task] = []
         if 'deps' in flags:
             for dep in flags['deps']:
+                # Check and warn for "addtask task after foo" while foo does not exist
+                if not dep in tasklist:
+                    bb.warn('%s: dependent task %s does not exist!' % (d.getVar('PN'), dep))
                 dep = d.expand(dep)
                 task_deps['parents'][task].append(dep)

I can't help wonder if this change has races, depending on the order of the addtask and deltask expressions. Do most layers parse cleanly with this?

At one point bitbake did support "floating" invalid tasks, I can't remember if we changed that or not though.

Cheers,

Richard

#6 lore raw

Re: [PATCH 2/2] bitbake: build.py: check dependendent task for addtask

Quoted 25 lines · Richard Purdie
Quoted 24 lines · Robert Yang

The following command is incorrect, but was ignored silently, that may suprise users:

addtask task after task_not_existed

This patch can check and warn for it. It would be better to also check "before" tasks, but there is no easier way to do it.

[YOCTO #13282]

---
  bitbake/lib/bb/build.py | 3 +++
  1 file changed, 3 insertions(+)
bitbake/lib/bb/build.py +1 -0
index 7571421..861e9a9 100644
@@ -815,6 +815,9 @@ def add_tasks(tasklist, d):
          task_deps['parents'][task] = []
          if 'deps' in flags:
              for dep in flags['deps']:
+                # Check and warn for "addtask task after foo" while foo does not exist
Quoted 8 lines

bitbake/lib/bb/build.py def add_tasks(tasklist, d): · line 822

Quoted 4 lines
bitbake/lib/bb/build.py +2 -0
+                if not dep in tasklist:
+                    bb.warn('%s: dependent task %s does not exist!' % (d.getVar('PN'), dep))
                  dep = d.expand(dep)
                  task_deps['parents'][task].append(dep)

I can't help wonder if this change has races, depending on the order of the addtask and deltask expressions. Do most layers parse cleanly with this?

The bb.build.add_tasks() is only called by ast.py's finalize(), I think that everything should be ready in finalize stage, and there should be no races. I tested this with meta-oe, python perl, selinux and others, didn't see any problems (unless I depended on an invalid task).

Quoted 3 lines

At one point bitbake did support "floating" invalid tasks, I can't remember if we changed that or not though.

This warning is good for checking typos during development, I think that user can call "bb.build.addtask()" dynamically in a function to fix the warning for floating tasks?

// Robert

Quoted 6 lines

Cheers,

Richard

#3 lore raw

Re: [PATCH 1/2] bitbake: BBHandler: Fix addtask and deltask

In reply to Richard Purdie
Quoted 85 lines · Richard Purdie
Quoted 59 lines · Robert Yang

The following commands are not supported, but they were ignored silently, that
may suprise users:

* addtask task1 task2
task2 is ignored

* addtask task1 before task2 before task3
Should be: addtask task1 before task2 task3

* addtask task1 after task2 after task3
Should be: addtask task1 after task2 task3

* deltask task1 task2
task2 is ignore

This patch can check and warn for them.

[YOCTO #13282]

---
  bitbake/lib/bb/parse/parse_py/BBHandler.py | 25
++++++++++++++++++++++++-
  1 file changed, 24 insertions(+), 1 deletion(-)
bitbake/lib/bb/parse/parse_py/BBHandler.py +0 -0
index 9dba5f2..4d90ff0 100644
@@ -42,7 +42,7 @@ __func_start_regexp__    =

re.compile(r"(((?P<py>python)|(?P<fr>fakeroot))\s*)*(

  __inherit_regexp__       = re.compile(r"inherit\s+(.+)" )
  __export_func_regexp__   = re.compile(r"EXPORT_FUNCTIONS\s+(.+)" )
  __addtask_regexp__       =

re.compile(r"addtask\s+(?P<func>\w+)\s*((before\s*(?P<before>((.*(?=a fter))|(.*))))|(after\s*(?P<after>((.*(?=before))|(.*)))))*")

bitbake/lib/bb/parse/parse_py/BBHandler.py +1 -1
-__deltask_regexp__       = re.compile(r"deltask\s+(?P<func>\w+)")
+__deltask_regexp__       =

re.compile(r"deltask\s+(?P<func>\w+)(?P<ignores>.*)")

  __addhandler_regexp__    = re.compile(r"addhandler\s+(.+)" )
  __def_regexp__           = re.compile(r"def\s+(\w+).*:" )
  __python_func_regexp__   = re.compile(r"(\s+.*)|(^$)|(^#)" )
bitbake/lib/bb/parse/parse_py/BBHandler.py +0 -0
@@ -236,11 +236,34 @@ def feeder(lineno, s, fn, root, statements,

eof=False):

      m = __addtask_regexp__.match(s)
      if m:
bitbake/lib/bb/parse/parse_py/BBHandler.py +3 -0
+        if len(m.group().split()) == 2:
+            # Check and warn for "addtask task1 task2"
+            m2 = re.match(r"addtask\s+(?P<func>\w+)(?P<ignores>.*)",

s)

bitbake/lib/bb/parse/parse_py/BBHandler.py +2 -0
+            if m2 and m2.group('ignores'):
+                logger.warning('addtask ignored: "%s"' %

m2.group('ignores'))

bitbake/lib/bb/parse/parse_py/BBHandler.py +2 -0
+
+        # Check and warn for "addtask task1 before task2 before

task3", the

bitbake/lib/bb/parse/parse_py/BBHandler.py def feeder(lineno, s, fn, root, statements, eof=False): · line 254

Quoted 9 lines
bitbake/lib/bb/parse/parse_py/BBHandler.py +9 -0
+        # similar to "after"
+        dup_after = 0
+        dup_before = 0
+        for tmp in s.split():
+            if 'before' == tmp:
+                dup_before += 1
+            if 'after' == tmp:
+                dup_after += 1
+        if dup_after > 1:

taskexpression = s.split()
if taskexpression.count("after") > 1:

logger.warning("X")
if taskexpression.count("before") > 1:

logger.warning("Y")

Quoted 2 lines

+ logger.warning('addtask found more than 1 keyword
"after": "%s"' % s)

How about:

addtask contained multiple 'after' keywords, only one is supported

Quoted 7 lines
Quoted 3 lines
bitbake/lib/bb/parse/parse_py/BBHandler.py +2 -0
+        if dup_before > 1:
+            logger.warning('addtask found more than 1 keyword

"before": "%s"' % s)

and:

addtask contained multiple 'before' keywords, only one is supported

Thanks for all the replies above, I will fix them.

Quoted 11 lines
Quoted 10 lines

ast.handleAddTask(statements, fn, lineno, m) return

m = __deltask_regexp__.match(s)
if m:

bitbake/lib/bb/parse/parse_py/BBHandler.py +3 -0
+        # Check and warn "for deltask task1 task2"
+        if m.group('ignores'):
+            logger.warning('deltask ignored: "%s"' %

m.group('ignores'))

Quoted 3 lines

deltask listing multiple tasks is not supported

I did wonder if we should support multiple tasks for deltask?

Yes, that's a good idea, but if we support "deltask multiple tasks", I think that we should also support "addtask multiple tasks" since they are a pair, but I have other concerns on addtask, for example, the following code is hard to read and maintain:

__addtask_regexp__ = re.compile(r"addtask\s+(?P<func>\w+)\s*((before\s*(?P<before>((.*(?=after))|(.*))))|(after\s*(?P<after>((.*(?=before))|(.*)))))*")

I think that we need break it into small pieces. so I gave it up with these concerns which looks risky.

Quoted 3 lines

Also, we might want to consider adding something to lib/bb/tests/ for this?

Thanks, I will add it.

// Robert

Quoted 6 lines

Cheers,

Richard

#7 lore raw

Re: [PATCH 2/2] bitbake: build.py: check dependendent task for addtask

In reply to Robert Yang

I just got few thousands of warnings thanks to this, they might be legitimate as write_bom_data and write_abi_xml_data are our own tasks added in
https://github.com/webosose/meta-webosose/blob/master/meta-webos/classes/webos_base.bbclass
very
long time ago (based on fetchall task which was added in 2006 :)) - there might be better way to add them now, I'll check how to do it better or else I might need to disable this warning locally.

Based on "utility-tasks: Drop fetchall and checkuriall tasks" I guess the right way to do the same now will be to convert our scripts to call it with --runall.

It's pity that the warning is shown repeatedly.

grep -c WARNING:.*dependent.task BUILD/log/cooker/m16p3/console-latest.log 30409

and that was only very small build not even finished yet:
Currently 7 running tasks (1142 of 2514)

..
WARNING: cross-localedef-native: dependent task do_deploy does not exist
WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist
WARNING: cross-localedef-native: dependent task do_deploy does not exist
WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist
WARNING: cross-localedef-native: dependent task write_bom_data does not exist
WARNING: cross-localedef-native: dependent task write_abi_xml_data does not exist
WARNING: autoconf-archive-native: dependent task do_deploy does not exist
WARNING: autoconf-archive-native: dependent task do_package_write_ipk does not exist
WARNING: autoconf-archive-native: dependent task do_deploy does not exist
WARNING: autoconf-archive-native: dependent task do_package_write_ipk does not exist
WARNING: autoconf-archive-native: dependent task write_bom_data does not exist
WARNING: autoconf-archive-native: dependent task write_abi_xml_data does not exist
WARNING: cross-localedef-native: dependent task do_deploy does not exist
WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist
WARNING: cross-localedef-native: dependent task do_deploy does not exist
WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist
WARNING: cross-localedef-native: dependent task write_bom_data does not exist
WARNING: cross-localedef-native: dependent task write_abi_xml_data does not exist
WARNING: cross-localedef-native: dependent task do_deploy does not exist
WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist
WARNING: cross-localedef-native: dependent task do_deploy does not exist
WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist
WARNING: cross-localedef-native: dependent task write_bom_data does not exist
WARNING: cross-localedef-native: dependent task write_abi_xml_data does not exist
WARNING: cross-localedef-native: dependent task do_deploy does not exist
WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist
WARNING: cross-localedef-native: dependent task do_deploy does not exist
WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist
WARNING: cross-localedef-native: dependent task write_bom_data does not exist
WARNING: cross-localedef-native: dependent task write_abi_xml_data does not exist
..

Quoted 70 lines · Robert Yang
Quoted 2 lines · Richard Purdie
Quoted 1 line · Robert Yang

The following command is incorrect, but was ignored silently, that may

suprise

Quoted 5 lines
Quoted 5 lines

users:

addtask task after task_not_existed

This patch can check and warn for it. It would be better to also check

"before"

Quoted 9 lines
Quoted 9 lines

tasks, but there is no easier way to do it.

[YOCTO #13282]

---
  bitbake/lib/bb/build.py | 3 +++
  1 file changed, 3 insertions(+)
Quoted 9 lines

bitbake/lib/bb/build.py def add_tasks(tasklist, d): · line 818

Quoted 9 lines
bitbake/lib/bb/build.py +1 -0
index 7571421..861e9a9 100644
@@ -815,6 +815,9 @@ def add_tasks(tasklist, d):
          task_deps['parents'][task] = []
          if 'deps' in flags:
              for dep in flags['deps']:
+                # Check and warn for "addtask task after foo" while

foo does not exist

Quoted 2 lines

bitbake/lib/bb/build.py def add_tasks(tasklist, d): · line 820

Quoted 2 lines
bitbake/lib/bb/build.py +2 -0
+                if not dep in tasklist:
+                    bb.warn('%s: dependent task %s does not exist!' %

(d.getVar('PN'), dep))

Quoted 6 lines

bitbake/lib/bb/build.py def add_tasks(tasklist, d): · line 822

Quoted 2 lines
bitbake/lib/bb/build.py +0 -0
                  dep = d.expand(dep)
                  task_deps['parents'][task].append(dep)

I can't help wonder if this change has races, depending on the order of the addtask and deltask expressions. Do most layers parse cleanly with this?

The bb.build.add_tasks() is only called by ast.py's finalize(), I think that
everything should be ready in finalize stage, and there should be no races. I tested this with meta-oe, python perl, selinux and others, didn't see any problems (unless I depended on an invalid task).

Quoted 3 lines

At one point bitbake did support "floating" invalid tasks, I can't remember if we changed that or not though.

This warning is good for checking typos during development, I think that user can call "bb.build.addtask()" dynamically in a function to fix the warning
for floating tasks?

// Robert

Quoted 6 lines

Cheers,

Richard

--
_______________________________________________
bitbake-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/bitbake-devel

#8 lore raw

Re: [PATCH 2/2] bitbake: build.py: check dependendent task for addtask

In reply to Robert Yang

Hi,

Quoted 44 lines · Robert Yang
Quoted 25 lines · Richard Purdie
Quoted 24 lines · Robert Yang

The following command is incorrect, but was ignored silently, that may suprise users:

addtask task after task_not_existed

This patch can check and warn for it. It would be better to also check "before" tasks, but there is no easier way to do it.

[YOCTO #13282]

---
  bitbake/lib/bb/build.py | 3 +++
  1 file changed, 3 insertions(+)
bitbake/lib/bb/build.py +1 -0
index 7571421..861e9a9 100644
@@ -815,6 +815,9 @@ def add_tasks(tasklist, d):
          task_deps['parents'][task] = []
          if 'deps' in flags:
              for dep in flags['deps']:
+                # Check and warn for "addtask task after foo" while foo does not exist
Quoted 8 lines

bitbake/lib/bb/build.py def add_tasks(tasklist, d): · line 822

Quoted 4 lines
bitbake/lib/bb/build.py +2 -0
+                if not dep in tasklist:
+                    bb.warn('%s: dependent task %s does not exist!' % (d.getVar('PN'), dep))
                  dep = d.expand(dep)
                  task_deps['parents'][task].append(dep)

I can't help wonder if this change has races, depending on the order of the addtask and deltask expressions. Do most layers parse cleanly with this?

The bb.build.add_tasks() is only called by ast.py's finalize(), I think that everything should be ready in finalize stage, and there should be no races. I tested this with meta-oe, python perl, selinux and others, didn't see any problems (unless I depended on an invalid task).

Quoted 1 line
Quoted 7 lines
Quoted 2 lines

At one point bitbake did support "floating" invalid tasks, I can't remember if we changed that or not though.

This warning is good for checking typos during development, I think that user can call "bb.build.addtask()" dynamically in a function to fix the warning for floating tasks?

I'm getting a lot of these warnings after removing tmp/ and rebuilding my image from sstate cache.
FWIW I use rm_work.bbclass, and regularly use the sstate-cache-management,sh script to prune old cache.

/Jacob

#9 lore raw

Re: [PATCH 2/2] bitbake: build.py: check dependendent task for addtask

Quoted 4 lines · Jacob Kroon

I'm getting a lot of these warnings after removing tmp/ and rebuilding my image from sstate cache.
FWIW I use rm_work.bbclass, and regularly use the sstate-cache-management,sh script to prune old cache.

There's a patch I hacked up in master-next that shows where these warnings are coming from. Yes, this patch produces two warnings per recipe with rm_work enabled.

Ross

#10 lore raw

Re: [PATCH 2/2] bitbake: build.py: check dependendent task for addtask

Quoted 10 lines · Ross
Quoted 4 lines · Jacob Kroon

I'm getting a lot of these warnings after removing tmp/ and rebuilding my image from sstate cache.
FWIW I use rm_work.bbclass, and regularly use the sstate-cache-management,sh script to prune old cache.

There's a patch I hacked up in master-next that shows where these warnings are coming from. Yes, this patch produces two warnings per recipe with rm_work enabled.

argh... i hit the same issue and came up with the exact same conclusion.. i should have checked the mailing list earlier today ;)

I am getting thousands of warnings like that with rm_work. I had modified bitbake so that it shows the 'task' name like Ross, in ().

WARNING: zip-native: dependent task do_deploy (do_rm_work) does not exist
WARNING: zip-native: dependent task do_package_write_ipk (do_rm_work) does not exist
WARNING: zip-native: dependent task do_deploy (do_build_without_rm_work) does not exist
WARNING: zip-native: dependent task do_package_write_ipk (do_build_without_rm_work) does not exist
WARNING: zip: dependent task do_deploy (do_rm_work) does not exist
WARNING: zip: dependent task do_deploy (do_build_without_rm_work) does not exist
WARNING: zip-native: dependent task do_deploy (do_rm_work) does not exist
WARNING: zip-native: dependent task do_package_write_ipk (do_rm_work) does not exist
WARNING: zip-native: dependent task do_deploy (do_build_without_rm_work) does not exist
WARNING: zip-native: dependent task do_package_write_ipk (do_build_without_rm_work) does not exist
WARNING: zip: dependent task do_deploy (do_rm_work) does not exist
WARNING: zip: dependent task do_deploy (do_build_without_rm_work) does not exist
WARNING: zip-native: dependent task do_deploy (do_rm_work) does not exist
WARNING: zip-native: dependent task do_package_write_ipk (do_rm_work) does not exist
WARNING: zip-native: dependent task do_deploy (do_build_without_rm_work) does not exist
WARNING: zip-native: dependent task do_package_write_ipk (do_build_without_rm_work) does not exist
WARNING: python-cython: dependent task do_deploy (do_rm_work) does not exist
WARNING: python-cython: dependent task do_deploy (do_build_without_rm_work) does not exist
WARNING: python-cython-native: dependent task do_deploy (do_rm_work) does not exist
WARNING: python-cython-native: dependent task do_package_write_ipk (do_rm_work) does not exist
WARNING: python-cython-native: dependent task do_deploy (do_build_without_rm_work) does not exist
WARNING: python-cython-native: dependent task do_package_write_ipk (do_build_without_rm_work) does not exist

Quoted 7 lines

Ross
--
_______________________________________________
bitbake-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/bitbake-devel

#11 lore raw

Re: [PATCH 2/2] bitbake: build.py: check dependendent task for addtask

Quoted 15 lines · Nicolas Dechesne
Quoted 8 lines · Ross
Quoted 5 lines · Jacob Kroon

I'm getting a lot of these warnings after removing tmp/ and rebuilding
my image from sstate cache.
FWIW I use rm_work.bbclass, and regularly use the sstate-cache-management,sh script to prune old cache.

Quoted 4 lines

There's a patch I hacked up in master-next that shows where these warnings are coming from. Yes, this patch produces two warnings per
recipe with rm_work enabled.

Quoted 5 lines

argh... i hit the same issue and came up with the exact same conclusion.. i should have checked the mailing list earlier today ;)

I am getting thousands of warnings like that with rm_work. I had modified bitbake so that it shows the 'task' name like Ross, in ().

I suspect we're going to need to rethink that patch, and/or perhaps rm_work...

I did have concerns about it but was assured it was widely tested. We also probably need an rm_work test case on the autobuilder (need a bug for that so we don't forget?).

Cheers,

Richard

#12 lore raw

Re: [PATCH 2/2] bitbake: build.py: check dependendent task for addtask

Quoted 16 lines · Richard Purdie
Quoted 15 lines · Nicolas Dechesne
Quoted 12 lines · Ross
Quoted 5 lines · Jacob Kroon

I'm getting a lot of these warnings after removing tmp/ and rebuilding
my image from sstate cache.
FWIW I use rm_work.bbclass, and regularly use the sstate-cache-management,sh script to prune old cache.

There's a patch I hacked up in master-next that shows where these warnings are coming from. Yes, this patch produces two warnings per
recipe with rm_work enabled.

Quoted 8 lines
Quoted 5 lines

argh... i hit the same issue and came up with the exact same conclusion.. i should have checked the mailing list earlier today ;)

I am getting thousands of warnings like that with rm_work. I had modified bitbake so that it shows the 'task' name like Ross, in ().

I suspect we're going to need to rethink that patch, and/or perhaps rm_work...

Sorry for the warnings, I just came back from the International Labour Day Holiday. Something is wrong with rm_work, I will send a patch to fix it.

Quoted 4 lines

I did have concerns about it but was assured it was widely tested. We also probably need an rm_work test case on the autobuilder (need a bug for that so we don't forget?).

Rather than add a testcae on autobuilder, how about add a testcase to oe-selftest, please ?

// Robert

Quoted 7 lines

Cheers,

Richard

#13 lore raw

Re: [PATCH 2/2] bitbake: build.py: check dependendent task for addtask

Hi RP,

After more investigations, these warnings are caused by recrdeptask, the rm_work calls:

addtask after 'deps + recrdeptask' before do_build

Then we will get the warnings, and we can divide these warnings into 3 kinds:

1) deltask only removes task from 'deps', but doesn't remove it from 'recrdeptask', so there are warnings for native recipes like dependent task do_package_write_rpm, although native.bbclass has called "deltask do_package_write_rpm', such a recrdeptask doesn't make any sense for native recipes. But we can't make deltask remove recrdeptask since we may need keep recrdeptask sometimes, for example, the image recipes don't need do_package_write_rpm, but their do_rootfs tasks require all the dependent rpms are done. I think that we need a command such as delrecrdeptask to fix it, but I'm not sure whether it is worth or not.

2) base.bbclass has set 'do_build[recrdeptask] += "do_deploy"', but do_deploy doesn't exist for most of the recipes, so we get the warnings of not do_deploy task, I don't know why do_deploy should be a recrdeptask, and maybe we need set it in rootfs_xxx.bbclass or image.bbclass, or set it conditionally in a func.

3) gcc-source doesn't have do_populate_sysroot or do_populate_lic, but do_build_without_rm_work always depends on them:

bb.build.addtask('do_build_without_rm_work', '', ' '.join(deps), d)

so we get warnings that gcc-source has no dependent task do_populate_sysroot or do_populate_lic. I think that we can make do_build_without_rm_work depend on them conditionally to fix the problem.

Any comments are appreciated.

// Robert

Quoted 49 lines · Robert Yang
Quoted 16 lines · Richard Purdie
Quoted 15 lines · Nicolas Dechesne
Quoted 12 lines · Ross
Quoted 5 lines · Jacob Kroon

I'm getting a lot of these warnings after removing tmp/ and rebuilding
my image from sstate cache.
FWIW I use rm_work.bbclass, and regularly use the sstate-cache-management,sh script to prune old cache.

There's a patch I hacked up in master-next that shows where these warnings are coming from.  Yes, this patch produces two warnings per
recipe with rm_work enabled.

Quoted 8 lines
Quoted 5 lines

argh... i hit the same issue and came up with the exact same conclusion.. i should have checked the mailing list earlier today ;)

I am getting thousands of warnings like that with rm_work. I had modified bitbake so that it shows the 'task' name like Ross, in ().

I suspect we're going to need to rethink that patch, and/or perhaps rm_work...

Sorry for the warnings, I just came back from the International Labour Day Holiday. Something is wrong with rm_work, I will send a patch to fix it.

Quoted 4 lines

I did have concerns about it but was assured it was widely tested. We also probably need an rm_work test case on the autobuilder (need a bug for that so we don't forget?).

Rather than add a testcae on autobuilder, how about add a testcase to oe-selftest, please ?

// Robert

Quoted 7 lines

Cheers,

Richard