Skip to contentSkip to conversation

bitbake-worker: Add DryRun event

PATCH In review Mark Hatle 3 participants 4 messages Oct 2, 2018 – Dec 4, 2021 1 message not in the archive

lore
Share & tools
lore

Mark Hatle

author
#0 lore raw

PATCH 1/1 bitbake-worker: Add DryRun event

There may be actions that a user wants to take, but the information is only available during task execution. Using the DryRun event you can prevent the actual task execution, but get access to the associated data stores.

Signed-off-by Mark Hatle <[email protected]>
---
 bin/bitbake-worker | 2 ++
 lib/bb/event.py    | 5 +++++
 2 files changed, 7 insertions(+)
bin/bitbake-worker +2 -0
index e925054b..e27fe7ac 100755
@@ -268,6 +268,8 @@ def fork_off_task(cfg, data, databuilder, workerdata, fn, task, taskname, append
                 os._exit(1)
             try:
                 if dry_run:
+                    localdata = bb.build._task_data(fn, taskname, the_data)
+                    bb.event.fire(bb.event.DryRun(), localdata)
                     return 0
                 return bb.build.exec_task(fn, taskname, the_data, cfg.profile)
             except:
lib/bb/event.py +5 -0
index 5b1b094a..c03541a1 100644
@@ -832,3 +832,8 @@ class FindSigInfoResult(Event):
     def __init__(self, result):
         Event.__init__(self)
         self.result = result
+
+class DryRun(Event):
+    """
+    Event to fire during a dry run
+    """
Signature · 1 line
2.16.0.rc2
#1 lore raw
Quoted 32 lines · Mark Hatle

There may be actions that a user wants to take, but the information is only available during task execution. Using the DryRun event you can prevent the actual task execution, but get access to the associated data stores.

Signed-off-by Mark Hatle <[email protected]>
---
 bin/bitbake-worker | 2 ++
 lib/bb/event.py    | 5 +++++
 2 files changed, 7 insertions(+)
bin/bitbake-worker +0 -0
index e925054b..e27fe7ac 100755
@@ -268,6 +268,8 @@ def fork_off_task(cfg, data, databuilder,

workerdata, fn, task, taskname, append

                 os._exit(1)
             try:
                 if dry_run:

+ localdata = bb.build._task_data(fn, taskname,
the_data)
+ bb.event.fire(bb.event.DryRun(), localdata)
return 0
return bb.build.exec_task(fn, taskname, the_data,
cfg.profile)
except:

lib/bb/event.py +0 -0
index 5b1b094a..c03541a1 100644

lib/bb/event.py class FindSigInfoResult(Event): · line 839

Quoted 8 lines
lib/bb/event.py +5 -0
@@ -832,3 +832,8 @@ class FindSigInfoResult(Event):
     def __init__(self, result):
         Event.__init__(self)
         self.result = result
+
+class DryRun(Event):
+    """
+    Event to fire during a dry run
+    """

I think we can add something like this but the DryRun event should really be in build.py and subclass TaskBase, called something like TaskDryRun?

I'm hoping that doesn't upset some of the other code that looks at TaskBase events but its certainly logically where the dryrun event belongs.

We may want to pass dry_run into exec_task as a parameter and put this code at the top of exec_task to make things cleaner across the files?

Cheers,

Richard

#2 lore raw
Quoted 50 lines · Richard Purdie
Quoted 32 lines · Mark Hatle

There may be actions that a user wants to take, but the information is only available during task execution. Using the DryRun event you can prevent the actual task execution, but get access to the associated data stores.

Signed-off-by Mark Hatle <[email protected]>
---
  bin/bitbake-worker | 2 ++
  lib/bb/event.py    | 5 +++++
  2 files changed, 7 insertions(+)
bin/bitbake-worker +0 -0
index e925054b..e27fe7ac 100755
@@ -268,6 +268,8 @@ def fork_off_task(cfg, data, databuilder,

workerdata, fn, task, taskname, append

                  os._exit(1)
              try:
                  if dry_run:

+ localdata = bb.build._task_data(fn, taskname,
the_data)
+ bb.event.fire(bb.event.DryRun(), localdata)
return 0
return bb.build.exec_task(fn, taskname, the_data,
cfg.profile)
except:

lib/bb/event.py +0 -0
index 5b1b094a..c03541a1 100644

lib/bb/event.py class FindSigInfoResult(Event): · line 839

Quoted 8 lines
lib/bb/event.py +5 -0
@@ -832,3 +832,8 @@ class FindSigInfoResult(Event):
      def __init__(self, result):
          Event.__init__(self)
          self.result = result
+
+class DryRun(Event):
+    """
+    Event to fire during a dry run
+    """

I think we can add something like this but the DryRun event should really be in build.py and subclass TaskBase, called something like TaskDryRun?

I'm hoping that doesn't upset some of the other code that looks at TaskBase events but its certainly logically where the dryrun event belongs.

Quoted 7 lines

We may want to pass dry_run into exec_task as a parameter and put this code at the top of exec_task to make things cleaner across the files?

Cheers,

Richard

Hey Mark,

I was looking at local WR patches that we carry and noticed this one.

Quoted 24 lines · you
Quoted 6 lines · Randy

Isn't this always firing?

It's mostly harmless but it's no longer really a completely dry-run

Quoted 7 lines

if you're logging PR info. Do you think anyone will object or

have you already discussed it with upstream folks?

Why not add a new flag?

If there is no event handler, then firing takes nearly no time at all.. I've

sent the change to bitbake.. We'll see if RP or others have comments.

The change itself though is designed to be generic, in that we want any possible

Quoted 9 lines

event handlers to work through this -- as it could be useful for collecting

statistics and such as well -- without running a build.

--Mark

so do you or Robert still care about this ?

I don't see it mentioned in the WR docs but
now that I think about it, it could be useful in a few use cases. The one that was mentioned in the internal email was:

Quoted 7 lines

Typically populating PR service takes a full build. This is because the

do_package task is what actually assigns the PR service values. So we want

a way that will allow us to generate the PR service values in significantly

less time then a full build.. (minutes vs hours)

Signature · 2 lines
# Randy MacLeod
# Wind River Linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw

Use Mark's current email address.
../Randy

Quoted 126 lines · Randy MacLeod
Quoted 50 lines · Richard Purdie
Quoted 35 lines · Mark Hatle

There may be actions that a user wants to take, but the information is only available during task execution.  Using the DryRun event you can prevent the actual task execution, but get access to the associated data stores.

Signed-off-by Mark Hatle <[email protected]>

---
  bin/bitbake-worker | 2 ++
  lib/bb/event.py    | 5 +++++

  2 files changed, 7 insertions(+)
bin/bitbake-worker +0 -0
index e925054b..e27fe7ac 100755
@@ -268,6 +268,8 @@ def fork_off_task(cfg, data, databuilder,

workerdata, fn, task, taskname, append
                  os._exit(1)
              try:
                  if dry_run:
+                    localdata = bb.build._task_data(fn, taskname, the_data)
+                    bb.event.fire(bb.event.DryRun(), localdata) return 0
                  return bb.build.exec_task(fn, taskname, the_data, cfg.profile)
              except:

lib/bb/event.py +0 -0
index 5b1b094a..c03541a1 100644
@@ -832,3 +832,8 @@ class FindSigInfoResult(Event):

      def __init__(self, result):
          Event.__init__(self)
          self.result = result

lib/bb/event.py class FindSigInfoResult(Event): · line 839

Quoted 5 lines
lib/bb/event.py +5 -0
+
+class DryRun(Event):
+    """
+    Event to fire during a dry run
+    """

I think we can add something like this but the DryRun event should really be in build.py and subclass TaskBase, called something like TaskDryRun?

I'm hoping that doesn't upset some of the other code that looks at TaskBase events but its certainly logically where the dryrun event belongs.

Quoted 7 lines

We may want to pass dry_run into exec_task as a parameter and put this code at the top of exec_task to make things cleaner across the files?

Cheers,

Richard

Hey Mark,

I was looking at local WR patches that we carry and noticed this one.

Quoted 24 lines · you
Quoted 6 lines · Randy

Isn't this always firing?

It's mostly harmless but it's no longer really a completely dry-run

Quoted 7 lines

if you're logging PR info. Do you think anyone will object or

have you already discussed it with upstream folks?

Why not add a new flag?

If there is no event handler, then firing takes nearly no time at all.. I've

sent the change to bitbake.. We'll see if RP or others have comments.

The change itself though is designed to be generic, in that we want any possible

Quoted 9 lines

event handlers to work through this -- as it could be useful for collecting

statistics and such as well -- without running a build.

--Mark

so do you or Robert still care about this ?

I don't see it mentioned in the WR docs but
now that I think about it, it could be useful in a few use cases. The one that was mentioned in the internal email was:

Quoted 7 lines

Typically populating PR service takes a full build.  This is because the

do_package task is what actually assigns the PR service values.  So we want

a way that will allow us to generate the PR service values in significantly

less time then a full build.. (minutes vs hours)

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#13130): https://lists.openembedded.org/g/bitbake-devel/message/13130
Mute This Topic: https://lists.openembedded.org/mt/72130803/3616765
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 2 lines
# Randy MacLeod
# Wind River Linux