Skip to contentSkip to conversation

runqueue: add mc to pending hash index

PATCH In review [email protected] 2 participants 4 messages Nov 28, 2019 – Nov 29, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 runqueue: add mc to pending hash index

From: Kai Kang <[email protected]>

When build multiconfig target by following steps in section "Enabling Multiple Configuration Build Dependencies" in yocto dev-manual which adds dependency between multiconfig targets:

do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs"

it shows a lot of warnings, such as:

| WARNING: Deferring
|   mc:x86:virtual:native:/path/to/meta/recipes-support/nss/nss_3.45.bb:do_populate_sysroot
|   after
|   mc:arm:virtual:native:/path/to/meta/recipes-support/nss/nss_3.45.bb:do_populate_sysroot

In function pending_hash_index it only checks PN, taskname and hash. But they are same for multiconfig native and allarch packages. Add multiconfig to pending hash index too to make the index unique.

Ref:
https://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#dev-enabling-multiple-configuration-build-dependencies

---
 bitbake/lib/bb/runqueue.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
bitbake/lib/bb/runqueue.py +2 -2
index 485d9341f5..059eb9e283 100644
@@ -69,12 +69,12 @@ def build_tid(mc, fn, taskname):
     return fn + ":" + taskname
 
 # Index used to pair up potentially matching multiconfig tasks
-# We match on PN, taskname and hash being equal
+# We match on mc, PN, taskname and hash being equal
 def pending_hash_index(tid, rqdata):
     (mc, fn, taskname, taskfn) = split_tid_mcfn(tid)
     pn = rqdata.dataCaches[mc].pkg_fn[taskfn]
     h = rqdata.runtaskentries[tid].unihash
-    return pn + ":" + "taskname" + h
+    return mc + ":" + pn + ":" + "taskname" + h
 
 class RunQueueStats:
     """
Signature · 1 line
2.17.1
#1 lore raw
Quoted 23 lines · On Thu, 2019-11-28 at 17:27 +0800, [email protected] wrote:

From: Kai Kang <[email protected]>

When build multiconfig target by following steps in section "Enabling Multiple Configuration Build Dependencies" in yocto dev-manual which adds dependency between multiconfig targets:

do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs"

it shows a lot of warnings, such as:

Quoted 6 lines

WARNING: Deferring
mc:x86:virtual:native:/path/to/meta/recipes-
support/nss/nss_3.45.bb:do_populate_sysroot
after
mc:arm:virtual:native:/path/to/meta/recipes-
support/nss/nss_3.45.bb:do_populate_sysroot

In function pending_hash_index it only checks PN, taskname and hash. But
they are same for multiconfig native and allarch packages. Add multiconfig
to pending hash index too to make the index unique.

Quoted 4 lines

Ref:
https://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#dev-enabling-multiple-configuration-build-dependencies

This patch completes defeats the point of this code, disabling it. Its the wrong thing to do, we want this code to be active.

The bug is that this should not be a warning, probably just a note.

We need it to be a note so that we can tell the mechanism is in action from build output logs.

Cheers,

Richard

Quoted 26 lines
---
 bitbake/lib/bb/runqueue.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
bitbake/lib/bb/runqueue.py +2 -2
index 485d9341f5..059eb9e283 100644
@@ -69,12 +69,12 @@ def build_tid(mc, fn, taskname):
     return fn + ":" + taskname
 
 # Index used to pair up potentially matching multiconfig tasks
-# We match on PN, taskname and hash being equal
+# We match on mc, PN, taskname and hash being equal
 def pending_hash_index(tid, rqdata):
     (mc, fn, taskname, taskfn) = split_tid_mcfn(tid)
     pn = rqdata.dataCaches[mc].pkg_fn[taskfn]
     h = rqdata.runtaskentries[tid].unihash
-    return pn + ":" + "taskname" + h
+    return mc + ":" + pn + ":" + "taskname" + h
 
 class RunQueueStats:
     """
Signature · 1 line
2.17.1
author
#2 lore raw
Quoted 26 lines · Richard Purdie
Quoted 25 lines · On Thu, 2019-11-28 at 17:27 +0800, [email protected] wrote:

From: Kai Kang <[email protected]>

When build multiconfig target by following steps in section "Enabling Multiple Configuration Build Dependencies" in yocto dev-manual which adds dependency between multiconfig targets:

do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs"

it shows a lot of warnings, such as:

Quoted 6 lines

WARNING: Deferring
mc:x86:virtual:native:/path/to/meta/recipes-
support/nss/nss_3.45.bb:do_populate_sysroot
after
mc:arm:virtual:native:/path/to/meta/recipes-
support/nss/nss_3.45.bb:do_populate_sysroot

In function pending_hash_index it only checks PN, taskname and hash. But
they are same for multiconfig native and allarch packages. Add multiconfig
to pending hash index too to make the index unique.

Ref:
https://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#dev-enabling-multiple-configuration-build-dependencies

Quoted 8 lines
Quoted 1 line

This patch completes defeats the point of this code, disabling it. Its the wrong thing to do, we want this code to be active.

The bug is that this should not be a warning, probably just a note.

We need it to be a note so that we can tell the mechanism is in action from build output logs.

I have sent a patch to change from warning to note. But without comment,  I thought it should be fixed in this function. Would you like to review the patch "runqueue.py: not show warning for deferred multiconfig task" on Oct 17, please?

Thanks,
Kai

Quoted 32 lines

Cheers,

Richard

Quoted 26 lines
---
  bitbake/lib/bb/runqueue.py | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
bitbake/lib/bb/runqueue.py +2 -2
index 485d9341f5..059eb9e283 100644
@@ -69,12 +69,12 @@ def build_tid(mc, fn, taskname):
      return fn + ":" + taskname
 
  # Index used to pair up potentially matching multiconfig tasks
-# We match on PN, taskname and hash being equal
+# We match on mc, PN, taskname and hash being equal
  def pending_hash_index(tid, rqdata):
      (mc, fn, taskname, taskfn) = split_tid_mcfn(tid)
      pn = rqdata.dataCaches[mc].pkg_fn[taskfn]
      h = rqdata.runtaskentries[tid].unihash
-    return pn + ":" + "taskname" + h
+    return mc + ":" + pn + ":" + "taskname" + h
 
  class RunQueueStats:
      """
Signature · 1 line
2.17.1
Signature · 1 line
Kai Kang
#3 lore raw
Quoted 41 lines · Kang Kai
Quoted 34 lines · Richard Purdie
Quoted 25 lines · On Thu, 2019-11-28 at 17:27 +0800, [email protected] wrote:

From: Kai Kang <[email protected]>

When build multiconfig target by following steps in section "Enabling
Multiple Configuration Build Dependencies" in yocto dev-manual which
adds dependency between multiconfig targets:

do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs"

it shows a lot of warnings, such as:

Quoted 6 lines

WARNING: Deferring
mc:x86:virtual:native:/path/to/meta/recipes-
support/nss/nss_3.45.bb:do_populate_sysroot
after
mc:arm:virtual:native:/path/to/meta/recipes-
support/nss/nss_3.45.bb:do_populate_sysroot

In function pending_hash_index it only checks PN, taskname and hash.
But
they are same for multiconfig native and allarch packages. Add multiconfig
to pending hash index too to make the index unique.

Quoted 4 lines

Ref:
https://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#dev-enabling-multiple-configuration-build-dependencies

This patch completes defeats the point of this code, disabling it. Its
the wrong thing to do, we want this code to be active.

Quoted 5 lines

The bug is that this should not be a warning, probably just a note.

We need it to be a note so that we can tell the mechanism is in action
from build output logs.

Quoted 5 lines

I have sent a patch to change from warning to note. But without comment, I thought it should be fixed in this function. Would you like to review the patch "runqueue.py: not show warning for
deferred multiconfig task" on Oct 17, please?

I thought I'd seen a proper fix!

That patch is correct, not sure why it wasn't merged but I'd thought it should be. Its merged now, thanks.

Cheers,

Richard