cef3: use PARALLEL_MAKE to control build parallelism
PATCH 1/2 cef3: use PARALLEL_MAKE to control build parallelism
--- recipes-browser/chromium/cef3_280796.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
recipes-browser/chromium/cef3_280796.bb +1 -1
@@ -60,5 +60,5 @@ do_qa_configure() { } do_compile() { - ninja -C out/${CHROMIUM_BUILD_TYPE} -j${BB_NUMBER_THREADS} cefsimple + ninja -C out/${CHROMIUM_BUILD_TYPE} ${PARALLEL_MAKE} cefsimple }
Signature · 1 line
1.9.1
PATCH 2/2 chromium: use PARALLEL_MAKE to control build parallelism
--- recipes-browser/chromium/chromium_40.0.2214.91.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
recipes-browser/chromium/chromium_40.0.2214.91.bb +1 -1
@@ -196,7 +196,7 @@ do_configure_append() { do_compile() { # build with ninja - ninja -C ${S}/out/${CHROMIUM_BUILD_TYPE} -j${BB_NUMBER_THREADS} chrome chrome_sandbox + ninja -C ${S}/out/${CHROMIUM_BUILD_TYPE} ${PARALLEL_MAKE} chrome chrome_sandbox }
Signature · 1 line
1.9.1
Re: [meta-browser][PATCH 2/2] chromium: use PARALLEL_MAKE to control build parallelism
this is ok
Quoted 26 lines · Andre McCurdy
Signed-off-by Andre McCurdy <[email protected]>--- recipes-browser/chromium/chromium_40.0.2214.91.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)recipes-browser/chromium/chromium_40.0.2214.91.bb +1
-2@@ -196,7 +196,7 @@ do_configure_append() { do_compile() { # build with ninja - ninja -C ${S}/out/${CHROMIUM_BUILD_TYPE} -j${BB_NUMBER_THREADS} chrome chrome_sandbox + ninja -C ${S}/out/${CHROMIUM_BUILD_TYPE} ${PARALLEL_MAKE} chrome chrome_sandbox } --1.9.1
--
_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel
Re: [meta-browser][PATCH 2/2] chromium: use PARALLEL_MAKE to control build parallelism
In reply to Andre McCurdyHi Andre,
Quoted 10 lines · Andre McCurdy
Signed-off-by Andre McCurdy <[email protected]>--- recipes-browser/chromium/chromium_40.0.2214.91.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)recipes-browser/chromium/chromium_40.0.2214.91.bb +0
-0
recipes-browser/chromium/chromium_40.0.2214.91.bb do_configure_append() { · line 200
Quoted 6 lines
recipes-browser/chromium/chromium_40.0.2214.91.bb +1
-1@@ -196,7 +196,7 @@ do_configure_append() { do_compile() { # build with ninja - ninja -C ${S}/out/${CHROMIUM_BUILD_TYPE} -j${BB_NUMBER_THREADS} chrome chrome_sandbox + ninja -C ${S}/out/${CHROMIUM_BUILD_TYPE} ${PARALLEL_MAKE} chrome chrome_sandbox }
Can you please share the chromium build times on your build-box with and without this patch? I'm just curious how well this worked for you.
Thanks and regards,
Nikolay
Re: [meta-browser][PATCH 2/2] chromium: use PARALLEL_MAKE to control build parallelism
Hi Nikolay,
Quoted 16 lines · Nikolay Dimitrov
Hi Andre,
Quoted 13 lines · Andre McCurdy
Signed-off-by Andre McCurdy <[email protected]>--- recipes-browser/chromium/chromium_40.0.2214.91.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)recipes-browser/chromium/chromium_40.0.2214.91.bb +0
-0@@ -196,7 +196,7 @@ do_configure_append() {
Quoted 11 lines
Quoted 7 lines
recipes-browser/chromium/chromium_40.0.2214.91.bb +0
-1do_compile() { # build with ninja - ninja -C ${S}/out/${CHROMIUM_BUILD_TYPE} -j${BB_NUMBER_THREADS}chrome
chrome_sandbox
+ ninja -C ${S}/out/${CHROMIUM_BUILD_TYPE} ${PARALLEL_MAKE} chromechrome_sandbox
}Can you please share the chromium build times on your build-box with and without this patch? I'm just curious how well this worked for you.
By default BB_NUMBER_THREADS and PARALLEL_MAKE are both derived from the number of logical CPU cores (as detected by oe.utils.cpu_count()), so the patch should not have any effect on build times.
The patch is about correctness - it's not really right to use BB_NUMBER_THREADS for anything other than setting the maximum number of BitBake threads.
Quoted 2 lines
Thanks and regards,
Nikolay
Re: [meta-browser][PATCH 2/2] chromium: use PARALLEL_MAKE to control build parallelism
Hi Andre,
Quoted 31 lines · Andre McCurdy
Hi Nikolay,
Quoted 16 lines · Nikolay Dimitrov
Hi Andre,
Quoted 13 lines · Andre McCurdy
Signed-off-by Andre McCurdy <[email protected]>--- recipes-browser/chromium/chromium_40.0.2214.91.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)recipes-browser/chromium/chromium_40.0.2214.91.bb +0
-0@@ -196,7 +196,7 @@ do_configure_append() {Quoted 11 lines
Quoted 7 lines
recipes-browser/chromium/chromium_40.0.2214.91.bb +0
-1do_compile() { # build with ninja - ninja -C ${S}/out/${CHROMIUM_BUILD_TYPE} -j${BB_NUMBER_THREADS}chrome
chrome_sandbox
+ ninja -C ${S}/out/${CHROMIUM_BUILD_TYPE} ${PARALLEL_MAKE} chromechrome_sandbox
}Can you please share the chromium build times on your build-box with and without this patch? I'm just curious how well this worked for you.
Quoted 7 lines
By default
BB_NUMBER_THREADSandPARALLEL_MAKEare both derived from the number of logical CPU cores (as detected byoe.utils.cpu_count()), so the patch should not have any effect on build times.The patch is about correctness - it's not really right to use
BB_NUMBER_THREADSfor anything other than setting the maximum number of BitBake threads.
Ahh, I see. Thanks.
Regards,
Nikolay
Quoted 10 lines · Andre McCurdy
Signed-off-by Andre McCurdy <[email protected]>--- recipes-browser/chromium/cef3_280796.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)recipes-browser/chromium/cef3_280796.bb +0
-0
recipes-browser/chromium/cef3_280796.bb do_qa_configure() { · line 64
Quoted 6 lines
recipes-browser/chromium/cef3_280796.bb +1
-1@@ -60,5 +60,5 @@ do_qa_configure() { } do_compile() { - ninja -C out/${CHROMIUM_BUILD_TYPE} -j${BB_NUMBER_THREADS} cefsimple + ninja -C out/${CHROMIUM_BUILD_TYPE} ${PARALLEL_MAKE} cefsimple }
this is ok
Quoted 8 lines
--
1.9.1--
_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel
Hi,
isn't ninja picking up the number of cores separately? I'm not sure why this is needed.
Z
Quoted 26 lines · Andre McCurdy
Signed-off-by Andre McCurdy <[email protected]>--- recipes-browser/chromium/cef3_280796.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)recipes-browser/chromium/cef3_280796.bb +1
-2@@ -60,5 +60,5 @@ do_qa_configure() { } do_compile() { - ninja -C out/${CHROMIUM_BUILD_TYPE} -j${BB_NUMBER_THREADS} cefsimple + ninja -C out/${CHROMIUM_BUILD_TYPE} ${PARALLEL_MAKE} cefsimple } --1.9.1
--
_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel
Signature · 5 lines
Zoltan Kuscsik Senior Software Developer, Linaro Digital Home Group email: [email protected] IRC: zoli_ on irc.linaro.org
Quoted 5 lines · Zoltan Kuscsik
Hi,
isn't ninja picking up the number of cores separately? I'm not sure why this is needed.
thats another issue. This one atleast improves the current usage a nit.
Quoted 45 lines
Z
Quoted 20 lines · Andre McCurdy
Signed-off-by Andre McCurdy <[email protected]>--- recipes-browser/chromium/cef3_280796.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)recipes-browser/chromium/cef3_280796.bb +1
-2@@ -60,5 +60,5 @@ do_qa_configure() { } do_compile() { - ninja -C out/${CHROMIUM_BUILD_TYPE} -j${BB_NUMBER_THREADS} cefsimple + ninja -C out/${CHROMIUM_BUILD_TYPE} ${PARALLEL_MAKE} cefsimple } --1.9.1
Quoted 6 lines
--
_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel--
Zoltan KuscsikSenior Software Developer, Linaro Digital Home Group
email: [email protected]
IRC:zoli_on irc.linaro.org
--
_______________________________________________
Openembedded-devel mailing list [email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel
Quoted 4 lines · Zoltan Kuscsik
Hi,
isn't ninja picking up the number of cores separately? I'm not sure why this is needed.
Without any -j option, ninja will default to "-j (logical_cores + 2)".
The PARALLEL_MAKE value configured in OE isn't just about maximising parallelism though (although that's it's common usage), it's also useful to limit parallelism and reduce the load on the CPU. If PARALLEL_MAKE isn't passed to ninja builds then they will always fully load the CPU.
Quoted 44 lines
Z
Quoted 20 lines · Andre McCurdy
Signed-off-by Andre McCurdy <[email protected]>--- recipes-browser/chromium/cef3_280796.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)recipes-browser/chromium/cef3_280796.bb +1
-2@@ -60,5 +60,5 @@ do_qa_configure() { } do_compile() { - ninja -C out/${CHROMIUM_BUILD_TYPE} -j${BB_NUMBER_THREADS} cefsimple + ninja -C out/${CHROMIUM_BUILD_TYPE} ${PARALLEL_MAKE} cefsimple } --1.9.1
Quoted 6 lines
--
_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel--
Zoltan KuscsikSenior Software Developer, Linaro Digital Home Group
email: [email protected]
IRC:zoli_on irc.linaro.org
--
_______________________________________________
Openembedded-devel mailing list [email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel
Fair enough.
Z
Quoted 66 lines · Andre McCurdy
Quoted 4 lines · Zoltan Kuscsik
Hi,
isn't ninja picking up the number of cores separately? I'm not sure why this is needed.
Without any -j option, ninja will default to "-j (
logical_cores+ 2)".The
PARALLEL_MAKEvalue configured in OE isn't just about maximising parallelism though (although that's it's common usage), it's also useful to limit parallelism and reduce the load on the CPU. IfPARALLEL_MAKEisn't passed to ninja builds then they will always fully load the CPU.Quoted 17 lines
Z
Quoted 13 lines · Andre McCurdy
Signed-off-by Andre McCurdy <[email protected]>--- recipes-browser/chromium/cef3_280796.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)recipes-browser/chromium/cef3_280796.bb +0
-0@@ -60,5 +60,5 @@ do_qa_configure() { }Quoted 2 lines
recipes-browser/chromium/cef3_280796.bb do_qa_configure() { · line 63
Quoted 2 lines
recipes-browser/chromium/cef3_280796.bb +0
-1do_compile() { - ninja -C out/${CHROMIUM_BUILD_TYPE} -j${BB_NUMBER_THREADS}cefsimple
Quoted 17 lines
Quoted 11 lines
recipes-browser/chromium/cef3_280796.bb +1
-1+ ninja -C out/${CHROMIUM_BUILD_TYPE} ${PARALLEL_MAKE} cefsimple } --1.9.1
--
_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel--
Zoltan KuscsikQuoted 8 lines
Senior Software Developer, Linaro Digital Home Group
email: [email protected]
IRC:zoli_on irc.linaro.org
--
_______________________________________________
Openembedded-devel mailing list [email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel--
_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel
Signature · 5 lines
Zoltan Kuscsik Senior Software Developer, Linaro Digital Home Group email: [email protected] IRC: zoli_ on irc.linaro.org
I've added -j in http://patches.openembedded.org/patch/100133/, because chromium recipe wasn't fair to other recipes. For instance, if bitbake is processing 8 compile tasks (incl. chromium) in parallel on a 24 cores machine, 7 recipes will be compiled with -j${BB_NUMBER_THREADS} whereas chromium will use something like -j24
Just out of my curiosity, what "${PARALLEL_MAKE}" brings compared to "-j${BB_NUMBER_THREADS}" ?
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Zoltan Kuscsik
Sent: jeudi 12 novembre 2015 07:08
To: openembedded-devel <[email protected]>
Subject: Re: [oe] [meta-browser][PATCH 1/2] cef3: use PARALLEL_MAKE to control build parallelism
Fair enough.
Z
Quoted 66 lines · Andre McCurdy
Quoted 4 lines · Zoltan Kuscsik
Hi,
isn't ninja picking up the number of cores separately? I'm not sure why this is needed.
Without any -j option, ninja will default to "-j (
logical_cores+ 2)".The
PARALLEL_MAKEvalue configured in OE isn't just about maximising parallelism though (although that's it's common usage), it's also useful to limit parallelism and reduce the load on the CPU. IfPARALLEL_MAKEisn't passed to ninja builds then they will always fully load the CPU.Quoted 17 lines
Z
Quoted 13 lines · Andre McCurdy
Signed-off-by Andre McCurdy <[email protected]>--- recipes-browser/chromium/cef3_280796.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)recipes-browser/chromium/cef3_280796.bb +0
-0@@ -60,5 +60,5 @@ do_qa_configure() { }Quoted 2 lines
recipes-browser/chromium/cef3_280796.bb do_qa_configure() { · line 63
Quoted 2 lines
recipes-browser/chromium/cef3_280796.bb +0
-1do_compile() { - ninja -C out/${CHROMIUM_BUILD_TYPE} -j${BB_NUMBER_THREADS}cefsimple
Quoted 17 lines
Quoted 11 lines
recipes-browser/chromium/cef3_280796.bb +1
-1+ ninja -C out/${CHROMIUM_BUILD_TYPE} ${PARALLEL_MAKE} cefsimple } --1.9.1
--
_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel--
Zoltan KuscsikQuoted 8 lines
Senior Software Developer, Linaro Digital Home Group
email: [email protected]
IRC:zoli_on irc.linaro.org
--
_______________________________________________
Openembedded-devel mailing list [email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel--
_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel
Signature · 10 lines
Zoltan Kuscsik Senior Software Developer, Linaro Digital Home Group email: [email protected] IRC: zoli_ on irc.linaro.org -- _______________________________________________ Openembedded-devel mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-devel
Quoted 5 lines · Julien Brianceau (jbriance)
I've added -j in http://patches.openembedded.org/patch/100133/, because chromium recipe wasn't fair to other recipes. For instance, if bitbake is processing 8 compile tasks (incl. chromium) in parallel on a 24 cores machine, 7 recipes will be compiled with -j${BB_NUMBER_THREADS} whereas chromium will use something like -j24
Just out of my curiosity, what "${PARALLEL_MAKE}" brings compared to "-j${BB_NUMBER_THREADS}" ?
BB_NUMBER_THREADS -> max bitbake tasks run at the same time PARALLEL_MAKE -> make tasks (gcc) at the same time
Andrease
That makes sense, thanks for the clarification.
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Andreas Müller
Sent: jeudi 12 novembre 2015 10:34
To: [email protected]
Subject: Re: [oe] [meta-browser][PATCH 1/2] cef3: use PARALLEL_MAKE to control build parallelism
Quoted 5 lines · Julien Brianceau (jbriance)
I've added -j in http://patches.openembedded.org/patch/100133/, because chromium recipe wasn't fair to other recipes. For instance, if bitbake is processing 8 compile tasks (incl. chromium) in parallel on a 24 cores machine, 7 recipes will be compiled with -j${BB_NUMBER_THREADS} whereas chromium will use something like -j24
Just out of my curiosity, what "${PARALLEL_MAKE}" brings compared to "-j${BB_NUMBER_THREADS}" ?
BB_NUMBER_THREADS -> max bitbake tasks run at the same time PARALLEL_MAKE -> make tasks (gcc) at the same time
Andrease
Signature · 4 lines
_______________________________________________ Openembedded-devel mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-devel