Skip to contentSkip to conversation

Mark some tasks in extensible SDK as 'noexec'

PATCH In review Chen Qi 3 participants 5 messages May 14, 2015 – May 15, 2015

lore
Share & tools
lore

Chen Qi

author
#0 lore raw

v2 cover Mark some tasks in extensible SDK as 'noexec'

Changes since V1:
Use the 'for' loop to simplify codes.

The following changes since commit 2d09cb28f25080ed337d37d2d4381b846bbe6247:

gcc5: Add PR65779 patch to fix powerpc compile issues (2015-05-12 13:10:17 +0100)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib ChenQi/SDK-noexec
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=ChenQi/SDK-noexec

Chen Qi (1):
populate_sdk_ext: set noexec flag for some tasks

 meta/classes/populate_sdk_ext.bbclass | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
Signature · 1 line
1.9.1
author
#1 lore raw

PATCH v2 1/1 populate_sdk_ext: set noexec flag for some tasks

For the SDK, what we are interested in is the sysroot part. So the only useful task is do_populate_sysroot. Other tasks like do_package don't need to run. So mark them as 'noexec' for SDK.

This would result in two benefits:
1) The size of extensible SDK is larged reduced. For example, extensible SDK for core-image-minimal is reduced from 1.8G to 793M.
2) The installation time is reduced as a lot of tasks are not run.

[YOCTO #7590]

---
 meta/classes/populate_sdk_ext.bbclass | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
meta/classes/populate_sdk_ext.bbclass +11 -1
index dc2c58e..6c265b8 100644
@@ -141,8 +141,18 @@ python copy_buildsystem () {
     sigfile = d.getVar('WORKDIR', True) + '/locked-sigs.inc'
     oe.copy_buildsystem.generate_locked_sigs(sigfile, d)
 
+    # Mark tasks to be 'noexec' for the SDK as we only need the sysroot part
+    base_class_path = os.path.join(baseoutpath, core_meta_subdir, 'classes', 'base.bbclass')
+    if os.path.exists(base_class_path):
+        with open(base_class_path, 'a') as f:
+            noexec_tasks = ['do_package', 'do_packagedata', 'do_package_qa', 'do_package_write_rpm',
+                            'do_package_write_ipk', 'do_package_write_deb', 'do_deploy', 'do_populate_lic',
+                            'do_rootfs']
+            for t in noexec_tasks:
+                f.write('%s[noexec] = "1"\n' % t)
+
     # Filter the locked signatures file to just the sstate tasks we are interested in
-    allowed_tasks = ['do_populate_lic', 'do_populate_sysroot', 'do_packagedata', 'do_package_write_ipk', 'do_package_write_rpm', 'do_package_write_deb', 'do_package_qa', 'do_deploy']
+    allowed_tasks = ['do_populate_sysroot']
     excluded_targets = d.getVar('SDK_TARGETS', True)
     lockedsigs_pruned = baseoutpath + '/conf/locked-sigs.inc'
     oe.copy_buildsystem.prune_lockedsigs(allowed_tasks,
Signature · 1 line
1.9.1
#2 lore raw

Re: [PATCH V2 1/1] populate_sdk_ext: set noexec flag for some tasks

Quoted 21 lines · Chen Qi

For the SDK, what we are interested in is the sysroot part. So the only useful task is do_populate_sysroot. Other tasks like do_package don't need to run. So mark them as 'noexec' for SDK.

This would result in two benefits:
1) The size of extensible SDK is larged reduced. For example, extensible SDK for core-image-minimal is reduced from 1.8G to 793M.
2) The installation time is reduced as a lot of tasks are not run.

[YOCTO #7590]

---
  meta/classes/populate_sdk_ext.bbclass | 12 +++++++++++-
  1 file changed, 11 insertions(+), 1 deletion(-)
meta/classes/populate_sdk_ext.bbclass +0 -0
index dc2c58e..6c265b8 100644

meta/classes/populate_sdk_ext.bbclass python copy_buildsystem () { · line 145

Quoted 5 lines
meta/classes/populate_sdk_ext.bbclass +2 -0
@@ -141,8 +141,18 @@ python copy_buildsystem () {
      sigfile = d.getVar('WORKDIR', True) + '/locked-sigs.inc'
      oe.copy_buildsystem.generate_locked_sigs(sigfile, d)
 
+    # Mark tasks to be 'noexec' for the SDK as we only need the sysroot part
+    base_class_path = os.path.join(baseoutpath, core_meta_subdir, 'classes', 'base.bbclass')

Qi, I don't like the idea of modifying the source that is copied into the sdk. Also, this will continue to persist if the user ever wants to start generating images.

Ideally, we would only modify the behavior while setting up the sdk. And then the behavior afterward would remain as when using a regular bitbake workspace.

Quoted 15 lines
meta/classes/populate_sdk_ext.bbclass +9 -1
+    if os.path.exists(base_class_path):
+        with open(base_class_path, 'a') as f:
+            noexec_tasks = ['do_package', 'do_packagedata', 'do_package_qa', 'do_package_write_rpm',
+                            'do_package_write_ipk', 'do_package_write_deb', 'do_deploy', 'do_populate_lic',
+                            'do_rootfs']
+            for t in noexec_tasks:
+                f.write('%s[noexec] = "1"\n' % t)
+
      # Filter the locked signatures file to just the sstate tasks we are interested in
-    allowed_tasks = ['do_populate_lic', 'do_populate_sysroot', 'do_packagedata', 'do_package_write_ipk', 'do_package_write_rpm', 'do_package_write_deb', 'do_package_qa', 'do_deploy']
+    allowed_tasks = ['do_populate_sysroot']
      excluded_targets = d.getVar('SDK_TARGETS', True)
      lockedsigs_pruned = baseoutpath + '/conf/locked-sigs.inc'
      oe.copy_buildsystem.prune_lockedsigs(allowed_tasks,
#3 lore raw

Re: [PATCH V2 1/1] populate_sdk_ext: set noexec flag for some tasks

Quoted 37 lines · On Thursday 14 May 2015 09:49:50 Randy Witt wrote:
Quoted 24 lines · Chen Qi

For the SDK, what we are interested in is the sysroot part. So the only useful task is do_populate_sysroot. Other tasks like do_package don't need to run. So mark them as 'noexec' for SDK.

This would result in two benefits:
1) The size of extensible SDK is larged reduced. For example, extensible

SDK for core-image-minimal is reduced from 1.8G to 793M.

2) The installation time is reduced as a lot of tasks are not run.

[YOCTO #7590]

---

  meta/classes/populate_sdk_ext.bbclass | 12 +++++++++++-
  1 file changed, 11 insertions(+), 1 deletion(-)
(quoted diff) +0 -0

b/meta/classes/populate_sdk_ext.bbclass index dc2c58e..6c265b8 100644

meta/classes/populate_sdk_ext.bbclass python copy_buildsystem () { · line 147

Quoted 7 lines
meta/classes/populate_sdk_ext.bbclass +1 -0
@@ -141,8 +141,18 @@ python copy_buildsystem () {
 
      sigfile = d.getVar('WORKDIR', True) + '/locked-sigs.inc'
      oe.copy_buildsystem.generate_locked_sigs(sigfile, d)
 
+    # Mark tasks to be 'noexec' for the SDK as we only need the sysroot
 part +    base_class_path = os.path.join(baseoutpath, core_meta_subdir,
 'classes', 'base.bbclass')

Qi, I don't like the idea of modifying the source that is copied into the sdk. Also, this will continue to persist if the user ever wants to start generating images.

Quoted 3 lines

Ideally, we would only modify the behavior while setting up the sdk. And then the behavior afterward would remain as when using a regular bitbake workspace.

Right. In order to fix this bug we really ought to only affect what happens at SDK install time. If we need to inject things into the configuration just for that execution, one way to do that would be to create a pre/post configuration file and pass that into the command that gets run with -r/-R.

Re allowed_tasks, don't you also need to change the recrdeptask value?

I'd also like to see this configurable at SDK construction time, so you can set a variable to provide and install the artifacts for the rest of the tasks if you want to.

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
author
#4 lore raw

Re: [PATCH V2 1/1] populate_sdk_ext: set noexec flag for some tasks

Quoted 45 lines · Paul Eggleton
Quoted 36 lines · On Thursday 14 May 2015 09:49:50 Randy Witt wrote:
Quoted 24 lines · Chen Qi

For the SDK, what we are interested in is the sysroot part. So the only useful task is do_populate_sysroot. Other tasks like do_package don't need to run. So mark them as 'noexec' for SDK.

This would result in two benefits:
1) The size of extensible SDK is larged reduced. For example, extensible

SDK for core-image-minimal is reduced from 1.8G to 793M.

2) The installation time is reduced as a lot of tasks are not run.

[YOCTO #7590]

---

   meta/classes/populate_sdk_ext.bbclass | 12 +++++++++++-
   1 file changed, 11 insertions(+), 1 deletion(-)
(quoted diff) +0 -0

b/meta/classes/populate_sdk_ext.bbclass index dc2c58e..6c265b8 100644

meta/classes/populate_sdk_ext.bbclass python copy_buildsystem () { · line 147

Quoted 7 lines
meta/classes/populate_sdk_ext.bbclass +1 -0
@@ -141,8 +141,18 @@ python copy_buildsystem () {
 
       sigfile = d.getVar('WORKDIR', True) + '/locked-sigs.inc'
       oe.copy_buildsystem.generate_locked_sigs(sigfile, d)
 
+    # Mark tasks to be 'noexec' for the SDK as we only need the sysroot
 part +    base_class_path = os.path.join(baseoutpath, core_meta_subdir,
 'classes', 'base.bbclass')

Qi, I don't like the idea of modifying the source that is copied into the sdk. Also, this will continue to persist if the user ever wants to start generating images.

Quoted 3 lines

Ideally, we would only modify the behavior while setting up the sdk. And then the behavior afterward would remain as when using a regular bitbake workspace.

Right. In order to fix this bug we really ought to only affect what happens at SDK install time. If we need to inject things into the configuration just for that execution, one way to do that would be to create a pre/post configuration file and pass that into the command that gets run with -r/-R.

Quoted 9 lines

Re allowed_tasks, don't you also need to change the recrdeptask value?

I'd also like to see this configurable at SDK construction time, so you can set a variable to provide and install the artifacts for the rest of the tasks if you want to.

Cheers,
Paul

Hi Paul and Randy,

Thanks for your feedback.
I totally agree with you.

And I'm going to follow Paul's suggestions and make a new patch.

Best Regards,
Chen Qi