Skip to contentSkip to conversation

I hava an idea to Make OpenEmbedded Run Faster

DISCUSSION 杨同峰 2 participants 7 messages Aug 27, 2007

lore
Share & tools
lore

杨同峰

author
#0 lore raw

EveryOne:
By two week of using OE , I hava notice that task was topology
sort to a queue .and Run Task One By One.
draw on the experience of Design of CPU . we can make a multiply thread run of it !

I notice that the biggest bottleneck of OE is do_fetch . then we
can use a mutiply thread downloader instead of wget
And about task schedule ,use method like this:

bb_No1 do_fetch do_patch do_configure do_build do_install
bb_No2 do_fetch do_patch do_configure
do_build do_install
bb_No3 do_fetch do_patch
do_configure do_build do_install

     bb_No4

do_fetch .......

Idea 1 : focus on download one file one time ,and make it more
fast use mutiply thread downloader
Idea 2 : make 0ther task of bb files to run asynchronously
Idea 3 : deal with task topology more carefully to make sure
that the bb file that was dependend was finish before
this one start to do_patch (Notice : not before
do_fetch )
Idea 4 : If topology can't make sure Idea 3 ,we must tell OE how
and when to wait

If the Idea can come true ,The speed can go 3 times faster then
now

Tongfeng Yang
2007-8-27

#1 lore raw
Quoted 28 lines · 杨同峰

EveryOne:
By two week of using OE , I hava notice that task was topology
sort to a queue .and Run Task One By One.
draw on the experience of Design of CPU . we can make a multiply thread run of it !

I notice that the biggest bottleneck of OE is do_fetch . then we
can use a mutiply thread downloader instead of wget
And about task schedule ,use method like this:

bb_No1 do_fetch do_patch do_configure do_build do_install
bb_No2 do_fetch do_patch do_configure
do_build do_install
bb_No3 do_fetch do_patch
do_configure do_build do_install

     bb_No4

do_fetch .......

Idea 1 : focus on download one file one time ,and make it more
fast use mutiply thread downloader
Idea 2 : make 0ther task of bb files to run asynchronously
Idea 3 : deal with task topology more carefully to make sure
that the bb file that was dependend was finish before
this one start to do_patch (Notice : not before
do_fetch )
Idea 4 : If topology can't make sure Idea 3 ,we must tell OE how
and when to wait

Quoted 3 lines

If the Idea can come true ,The speed can go 3 times faster then
now

bitbake already does this.

Signature · 1 line
Justin Patrin
author
#2 lore raw

really?? Bitbake same to be single thread . My Main Idear is like this when some task is compiling ,Some task is fetching ,and other maybe do_installing .
but bitbake don't look like this .

Tongfeng.Yang

Quoted 35 lines · 在 2007-08-26日的 21:04 -0700,Justin Patrin写道:
Quoted 28 lines · 杨同峰

EveryOne:
By two week of using OE , I hava notice that task was topology
sort to a queue .and Run Task One By One.
draw on the experience of Design of CPU . we can make a multiply thread run of it !

I notice that the biggest bottleneck of OE is do_fetch . then we
can use a mutiply thread downloader instead of wget
And about task schedule ,use method like this:

bb_No1 do_fetch do_patch do_configure do_build do_install
bb_No2 do_fetch do_patch do_configure
do_build do_install
bb_No3 do_fetch do_patch
do_configure do_build do_install

     bb_No4

do_fetch .......

Idea 1 : focus on download one file one time ,and make it more
fast use mutiply thread downloader
Idea 2 : make 0ther task of bb files to run asynchronously
Idea 3 : deal with task topology more carefully to make sure
that the bb file that was dependend was finish before
this one start to do_patch (Notice : not before
do_fetch )
Idea 4 : If topology can't make sure Idea 3 ,we must tell OE how
and when to wait

Quoted 3 lines

If the Idea can come true ,The speed can go 3 times faster then
now

bitbake already does this.

author
#3 lore raw

KEEP DOWNLOADING UNTIL ALL WAS DOWNLOAD,DON'T LET NET REST !
-------------------------------

We See that , all the times when in compiling . No Task is downloading .
This is a big waste of time .

My Idear is KEEP DOWNLOADING UNTIL ALL WAS DOWNLOAD,DON'T LET NET REST !

If the whole compile is not finish ,Keep Downling !!!!!

The one who has a big width of net ,can't understand my feeling who has only 5k/s width of net .

R

#4 lore raw
In reply to 杨同峰
Quoted 5 lines · 杨同峰

really?? Bitbake same to be single thread . My Main Idear is like this when some task is compiling ,Some task is fetching ,and other maybe do_installing .
but bitbake don't look like this .

What version of bitbake are you using? I know that 1.8 supports multithreading.

Quoted 43 lines · 在 2007-08-26日的 21:04 -0700,Justin Patrin写道:
Quoted 30 lines · 杨同峰
Quoted 28 lines

EveryOne:
By two week of using OE , I hava notice that task was topology
sort to a queue .and Run Task One By One.
draw on the experience of Design of CPU . we can make a multiply thread run of it !

I notice that the biggest bottleneck of OE is do_fetch . then we
can use a mutiply thread downloader instead of wget
And about task schedule ,use method like this:

bb_No1 do_fetch do_patch do_configure do_build do_install
bb_No2 do_fetch do_patch do_configure
do_build do_install
bb_No3 do_fetch do_patch
do_configure do_build do_install

     bb_No4

do_fetch .......

Idea 1 : focus on download one file one time ,and make it more
fast use mutiply thread downloader
Idea 2 : make 0ther task of bb files to run asynchronously
Idea 3 : deal with task topology more carefully to make sure
that the bb file that was dependend was finish before
this one start to do_patch (Notice : not before
do_fetch )
Idea 4 : If topology can't make sure Idea 3 ,we must tell OE how
and when to wait

Quoted 6 lines
Quoted 3 lines

If the Idea can come true ,The speed can go 3 times faster then
now

bitbake already does this.

_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel

Signature · 1 line
Justin Patrin
author
#5 lore raw

Oh , My bitbake is 1.6.6
I'll try 1.8 thank you !