I hava an idea to Make OpenEmbedded Run Faster
EveryOne:
By two week of using OE , I hava notice that task was topology
sort to a queue .and Run Task One By One.
draw on the experience of Design of CPU . we can make a multiply thread run of it !
I notice that the biggest bottleneck of OE is do_fetch . then we
can use a mutiply thread downloader instead of wget
And about task schedule ,use method like this:
bb_No1 do_fetch do_patch do_configure do_build do_install
bb_No2 do_fetch do_patch do_configuredo_build do_install
bb_No3 do_fetch do_patchdo_configure do_build do_install
bb_No4
do_fetch .......
Idea 1 : focus on download one file one time ,and make it more
fast use mutiply thread downloader
Idea 2 : make 0ther task of bb files to run asynchronously
Idea 3 : deal with task topology more carefully to make sure
that the bb file that was dependend was finish before
this one start to do_patch (Notice : not beforedo_fetch )
Idea 4 : If topology can't make sure Idea 3 ,we must tell OE how
and when to wait
If the Idea can come true ,The speed can go 3 times faster then
now
Tongfeng Yang
2007-8-27
Quoted 28 lines · 杨同峰
EveryOne:
By two week of using OE , I hava notice that task was topology
sort to a queue .and Run Task One By One.
draw on the experience of Design of CPU . we can make a multiply thread run of it !I notice that the biggest bottleneck of OE is
do_fetch. then we
can use a mutiply thread downloader instead of wget
And about task schedule ,use method like this:
bb_No1do_fetchdo_patchdo_configuredo_builddo_install
bb_No2do_fetchdo_patchdo_configuredo_builddo_install
bb_No3do_fetchdo_patchdo_configuredo_builddo_installbb_No4
do_fetch.......Idea 1 : focus on download one file one time ,and make it more
fast use mutiply thread downloader
Idea 2 : make 0ther task of bb files to run asynchronously
Idea 3 : deal with task topology more carefully to make sure
that the bb file that was dependend was finish before
this one start todo_patch(Notice : not beforedo_fetch)
Idea 4 : If topology can't make sure Idea 3 ,we must tell OE how
and when to wait
Quoted 3 lines
If the Idea can come true ,The speed can go 3 times faster then
now
bitbake already does this.
Signature · 1 line
Justin Patrin
really?? Bitbake same to be single thread . My Main Idear is like this when some task is compiling ,Some task is fetching ,and other maybe do_installing .
but bitbake don't look like this .
Tongfeng.Yang
Quoted 35 lines · 在 2007-08-26日的 21:04 -0700,Justin Patrin写道:
Quoted 28 lines · 杨同峰
EveryOne:
By two week of using OE , I hava notice that task was topology
sort to a queue .and Run Task One By One.
draw on the experience of Design of CPU . we can make a multiply thread run of it !I notice that the biggest bottleneck of OE is
do_fetch. then we
can use a mutiply thread downloader instead of wget
And about task schedule ,use method like this:
bb_No1do_fetchdo_patchdo_configuredo_builddo_install
bb_No2do_fetchdo_patchdo_configuredo_builddo_install
bb_No3do_fetchdo_patchdo_configuredo_builddo_installbb_No4
do_fetch.......Idea 1 : focus on download one file one time ,and make it more
fast use mutiply thread downloader
Idea 2 : make 0ther task of bb files to run asynchronously
Idea 3 : deal with task topology more carefully to make sure
that the bb file that was dependend was finish before
this one start todo_patch(Notice : not beforedo_fetch)
Idea 4 : If topology can't make sure Idea 3 ,we must tell OE how
and when to waitQuoted 3 lines
If the Idea can come true ,The speed can go 3 times faster then
nowbitbake already does this.
KEEP DOWNLOADING UNTIL ALL WAS DOWNLOAD,DON'T LET NET REST !
-------------------------------
We See that , all the times when in compiling . No Task is downloading .
This is a big waste of time .
My Idear is KEEP DOWNLOADING UNTIL ALL WAS DOWNLOAD,DON'T LET NET REST !
If the whole compile is not finish ,Keep Downling !!!!!
The one who has a big width of net ,can't understand my feeling who has only 5k/s width of net .
R
Quoted 5 lines · 杨同峰
really?? Bitbake same to be single thread . My Main Idear is like this when some task is compiling ,Some task is fetching ,and other maybe
do_installing.
but bitbake don't look like this .
What version of bitbake are you using? I know that 1.8 supports multithreading.
Quoted 43 lines · 在 2007-08-26日的 21:04 -0700,Justin Patrin写道:
Quoted 30 lines · 杨同峰
Quoted 28 lines
EveryOne:
By two week of using OE , I hava notice that task was topology
sort to a queue .and Run Task One By One.
draw on the experience of Design of CPU . we can make a multiply thread run of it !I notice that the biggest bottleneck of OE is
do_fetch. then we
can use a mutiply thread downloader instead of wget
And about task schedule ,use method like this:
bb_No1do_fetchdo_patchdo_configuredo_builddo_install
bb_No2do_fetchdo_patchdo_configuredo_builddo_install
bb_No3do_fetchdo_patchdo_configuredo_builddo_installbb_No4
do_fetch.......Idea 1 : focus on download one file one time ,and make it more
fast use mutiply thread downloader
Idea 2 : make 0ther task of bb files to run asynchronously
Idea 3 : deal with task topology more carefully to make sure
that the bb file that was dependend was finish before
this one start todo_patch(Notice : not beforedo_fetch)
Idea 4 : If topology can't make sure Idea 3 ,we must tell OE how
and when to waitQuoted 6 lines
Quoted 3 lines
If the Idea can come true ,The speed can go 3 times faster then
nowbitbake already does this.
_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
Signature · 1 line
Justin Patrin
Oh , My bitbake is 1.6.6
I'll try 1.8 thank you !
Thanks to Justin Patrin !
I update the bitbake to 1.8.6 and export BB_NUMBER_THREADS=10, Now it's Run fast !!