Some further build dependency timings

Just for reference, with a base configuration, sato image:

real 50m8.223s
user 298m41.450s
sys 52m42.200s

adding:

ASSUME_PROVIDED =+ "bison-native flex-native sqlite3-native git-native"

(and hacking the pseudo recipe to use a sqlite3-native):

real 42m6.740s
user 296m21.940s
sys 52m25.220s

We continue to have real dependency issues around gettext in both the native and target builds...

Quoted 10 lines · Richard Purdie

Just for reference, with a base configuration, sato image:

real 50m8.223s
user 298m41.450s
sys 52m42.200s

adding:

ASSUME_PROVIDED =+ "bison-native flex-native sqlite3-native git-native"

Quoted 8 lines

(and hacking the pseudo recipe to use a sqlite3-native):

real 42m6.740s
user 296m21.940s
sys 52m25.220s

We continue to have real dependency issues around gettext in both the native and target builds...

So to continue the story, adding:

DEBUG_FLAGS = ""
INHERIT_INSANE = ""
PACKAGE_CLASSES = "package_ipk"
USER_CLASSES = ""

Results in:

real 38m23.053s
user 237m7.430s
sys 39m3.720s

Then adding PSEUDO_RELOADED=yes to the environment of tasks not needing pseudo (hack patch below):

real 36m20.683s
user 236m19.580s
sys 37m54.450s

Finally, adding:

DISABLESTATIC = "--disable-static"
DISABLESTATIC_pn-qemu = ""
DISABLESTATIC_pn-qemu-native = ""
DISABLESTATIC_pn-openssl = ""
DISABLESTATIC_pn-openssl-native = ""
EXTRA_OECONF += "${DISABLESTATIC}"

real 34m53.877s
user 233m34.780s
sys 38m50.190s

Cheers,

Richard

bitbake/lib/bb/runqueue.py +0 -0

         sys.stdout.flush()
         sys.stderr.flush()
 

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Quoted 4 lines

bitbake/lib/bb/runqueue.py +0 -0

@@ -1096,6 +1096,10 @@ class RunQueueExecute:
 

bitbake/lib/bb/runqueue.py class RunQueueExecute: · line 1102

Quoted 6 lines

bitbake/lib/bb/runqueue.py +4 -0

             logger.debug(2, 'Running %s:%s under fakeroot, fakedirs: %s' %
                             (fn, taskname, ', '.join(fakedirs)))
+        else:
+            envbackup["PSEUDO_RELOADED"] = os.environ.get("PSEUDO_RELOADED")
+            os.environ["PSEUDO_RELOADED"] = "yes"
+            fakeenv["PSEUDO_RELOADED"] = "yes"

PSEUDO_RELOADED was intended to only be used internally by pseudo. Specifically to signify to fork/exec that we're about to run the pseudo server process and to remove PSEUDO from the LD_PRELOAD on the next fork/exec.

The system simply checks the environment on the fork-exec and doesn't check the value on pseudo-load. This is why it works.

That side effect means that any subsequent calls can NOT re-enable pseudo as it's no longer in memory at all.

So what you are seeing at best is the time difference from avoiding the LD_PRELOAD and the jump table setup. (Even when pseudo is disabled, we need to enable the jump table, we just pass the call to the original function.)

But more then likely because pseudo is no longer memory resident for the called processes and it's sub-processes.. there is likely some place in the system that uses pseudo functionality that is no longer functioning properly.

It would be interesting to determine first off, if there is anywhere pseudo should be running that it currently isn't. And second if this is simply due to LD_PRELOAD time, or if enabling the jump table is the culprit.

One potential optimization, when PSEUDO_DISABLE=1 is to only setup the jump table entries for fork/exec items.

Also if it is desired to unload pseudo, I'd suggest we add a new environment variable to pseudo that is specific about unloading pseudo, instead of using the existing PSEUDO_RELOADED. Not only is the name confusing, but since it's an internal item it's behavior may change or go away in the future.

(Below is from the pseudo man page:

PSEUDO_RELOADED
This purely internal variable is used to track state while try- ing to re-execute to get rid of the LD_PRELOAD value when spawning a server. (The pseudo server itself cannot function running in the pseudo environment.)
)

Quoted 4 lines · Mark Hatle

It would be interesting to determine first off, if there is anywhere pseudo should be running that it currently isn't. And second if this is simply due to LD_PRELOAD time, or if enabling the jump table is the culprit.

Quoted 2 lines

One potential optimization, when PSEUDO_DISABLE=1 is to only setup the jump table entries for fork/exec items.

Agreed, I don't know where the time is being spent exactly at this point. I do know we execute an absolute ton of exec/fork calls so removing any overhead from them will improve our speed though. This change means do_configure won't run with pseudo enabled for example which is a very exec heavy task.