Definition of Yocto tasks
Is there a document that provides a detailed definition of what each these tasks do? Yocto, OE, and bitbake manuals tell us how to command a specific task to be run, but not what they do.
The task name provides a good hint sometimes, but often that is not enough. Users shouldn't have to read the python code for this information. These descriptions should be a part of the yocto manual.
Sections 5.3.4 to 5.3.11 of the Yocto mega-manual provide a good overview of some of these tasks in the context of a workflow, but doesn't mention most of these tasks and doesn't go into enough detail on the tasks it discusses.
It wouldn't be appropriate to add more detail at that point in the manual, but it would be appropriate to include details regarding all of the tasks in an appendix.
Examples...
The manual states that do_fetch fetches source, but doesn't state that when git is used whether it performs a pull, fetch, or clone. What are the common failure conditions of do_fetch? And what should the user do to fix the problem?
do_build is mentioned a few times, but there is no reference to it being the default task or what tasks are invoked by do_build and which are omitted.
do_rm_work and do_wm_work_all are not mentioned at all.
Result of listtasks:do_fetchalldo_builddo_devshelldo_package_write_ipkdo_cleansstatedo_savedefconfigdo_uboot_mkimagedo_sizecheckdo_stripdo_packagedata_setscenedo_configuredo_cleando_deploy_setscenedo_cleanalldo_populate_licdo_populate_sysrootdo_devicetree_imagedo_deploydo_menuconfigdo_patchdo_bundle_initramfsdo_packagedatado_listtasksdo_compiledo_package_setscenedo_populate_lic_setscenedo_fetchdo_checkurido_compile_kernelmodulesdo_package_write_ipk_setscenedo_package_writedo_rm_workdo_packagedo_unpackdo_installdo_checkurialldo_populate_sysroot_setscenedo_rm_work_all
I don't have the knowledge to create such an appendix, but I volunteer to be a proofreader.
Regards,
Bob
An appendix for a reference of these tasks seems like a good idea.
Scott
Quoted 87 lines
-----Original Message-----
From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Bob Feretich
Sent: Monday, May 05, 2014 4:52 PM
To: [email protected]
Subject: [yocto] Definition of Yocto tasksIs there a document that provides a detailed definition of what each these tasks do?
Yocto, OE, and bitbake manuals tell us how to command a specific task to be run, but not what they do.The task name provides a good hint sometimes, but often that is not enough. Users shouldn't have to read the python code for this information. These descriptions should be a part of the yocto manual.
Sections 5.3.4 to 5.3.11 of the Yocto mega-manual provide a good overview of some of these tasks in the context of a workflow, but doesn't mention most of these tasks and doesn't go into enough detail on the tasks it discusses.
It wouldn't be appropriate to add more detail at that point in the manual, but it would be appropriate to include details regarding all of the tasks in an appendix.
Examples...
The manual states thatdo_fetchfetches source, but doesn't state that when git is used whether it performs a pull, fetch, or clone. What are the common failure conditions ofdo_fetch? And what should the user do to fix the problem?
do_buildis mentioned a few times, but there is no reference to it being the default task or what tasks are invoked bydo_buildand which are omitted.
do_rm_workanddo_wm_work_allare not mentioned at all.Result of listtasks:
do_fetchalldo_builddo_devshelldo_package_write_ipkdo_cleansstatedo_savedefconfigdo_uboot_mkimagedo_sizecheckdo_stripdo_packagedata_setscenedo_configuredo_cleando_deploy_setscenedo_cleanalldo_populate_licdo_populate_sysrootdo_devicetree_imagedo_deploydo_menuconfigdo_patchdo_bundle_initramfsdo_packagedatado_listtasksdo_compiledo_package_setscenedo_populate_lic_setscenedo_fetchdo_checkurido_compile_kernelmodulesdo_package_write_ipk_setscenedo_package_writedo_rm_workdo_packagedo_unpackdo_installdo_checkurialldo_populate_sysroot_setscenedo_rm_work_allI don't have the knowledge to create such an appendix, but I volunteer to be a proofreader.
Regards,
Bob--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto
Quoted 77 lines · Scott M
Quoted 77 lines
-----Original Message-----
From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Bob Feretich
Sent: Monday, May 05, 2014 4:52 PM
To: [email protected]
Subject: [yocto] Definition of Yocto tasksIs there a document that provides a detailed definition of what each these tasks do?
Yocto, OE, and bitbake manuals tell us how to command a specific task to be run, but not what they do.The task name provides a good hint sometimes, but often that is not enough. Users shouldn't have to read the python code for this information. These descriptions should be a part of the yocto manual.
Sections 5.3.4 to 5.3.11 of the Yocto mega-manual provide a good overview of some of these tasks in the context of a workflow, but doesn't mention most of these tasks and doesn't go into enough detail on the tasks it discusses.
It wouldn't be appropriate to add more detail at that point in the manual, but it would be appropriate to include details regarding all of the tasks in an appendix.
Examples...
The manual states thatdo_fetchfetches source, but doesn't state that when git is used whether it performs a pull, fetch, or clone. What are the common failure conditions ofdo_fetch? And what should the user do to fix the problem?
do_buildis mentioned a few times, but there is no reference to it being the default task or what tasks are invoked bydo_buildand which are omitted.
do_rm_workanddo_wm_work_allare not mentioned at all.Result of listtasks:
do_fetchalldo_builddo_devshelldo_package_write_ipkdo_cleansstatedo_savedefconfigdo_uboot_mkimagedo_sizecheckdo_stripdo_packagedata_setscenedo_configuredo_cleando_deploy_setscenedo_cleanalldo_populate_licdo_populate_sysrootdo_devicetree_imagedo_deploydo_menuconfigdo_patchdo_bundle_initramfsdo_packagedatado_listtasksdo_compiledo_package_setscenedo_populate_lic_setscenedo_fetchdo_checkurido_compile_kernelmodulesdo_package_write_ipk_setscenedo_package_writedo_rm_workdo_packagedo_unpackdo_installdo_checkurialldo_populate_sysroot_setscenedo_rm_work_all
Quoted 4 lines
Quoted 2 lines
I don't have the knowledge to create such an appendix, but I volunteer to be a proofreader.
An appendix for a reference of these tasks seems like a good idea.
FYI, you may already have seen it but we have a bit of coverage for the common tasks in the following section of the manual:
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#closer-look
If we wanted to add an appendix to list them all (and it might be worth us doing so) a good starting point would be the task descriptions in documentation.conf:
http://cgit.openembedded.org/openembedded-core/tree/meta/conf/documentation.conf
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Quoted 78 lines · Paul Eggleton
Quoted 77 lines · Scott M
Quoted 77 lines
-----Original Message-----
From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Bob Feretich
Sent: Monday, May 05, 2014 4:52 PM
To: [email protected]
Subject: [yocto] Definition of Yocto tasksIs there a document that provides a detailed definition of what each these tasks do?
Yocto, OE, and bitbake manuals tell us how to command a specific task to be run, but not what they do.The task name provides a good hint sometimes, but often that is not enough. Users shouldn't have to read the python code for this information. These descriptions should be a part of the yocto manual.
Sections 5.3.4 to 5.3.11 of the Yocto mega-manual provide a good overview of some of these tasks in the context of a workflow, but doesn't mention most of these tasks and doesn't go into enough detail on the tasks it discusses.
It wouldn't be appropriate to add more detail at that point in the manual, but it would be appropriate to include details regarding all of the tasks in an appendix.
Examples...
The manual states thatdo_fetchfetches source, but doesn't state that when git is used whether it performs a pull, fetch, or clone. What are the common failure conditions ofdo_fetch? And what should the user do to fix the problem?
do_buildis mentioned a few times, but there is no reference to it being the default task or what tasks are invoked bydo_buildand which are omitted.
do_rm_workanddo_wm_work_allare not mentioned at all.Result of listtasks:
do_fetchalldo_builddo_devshelldo_package_write_ipkdo_cleansstatedo_savedefconfigdo_uboot_mkimagedo_sizecheckdo_stripdo_packagedata_setscenedo_configuredo_cleando_deploy_setscenedo_cleanalldo_populate_licdo_populate_sysrootdo_devicetree_imagedo_deploydo_menuconfigdo_patchdo_bundle_initramfsdo_packagedatado_listtasksdo_compiledo_package_setscenedo_populate_lic_setscenedo_fetchdo_checkurido_compile_kernelmodulesdo_package_write_ipk_setscenedo_package_writedo_rm_workdo_packagedo_unpackdo_installdo_checkurialldo_populate_sysroot_setscenedo_rm_work_all
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
I don't have the knowledge to create such an appendix, but I volunteer to be a proofreader.
An appendix for a reference of these tasks seems like a good idea.
FYI, you may already have seen it but we have a bit of coverage for the common tasks in the following section of the manual:
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#closer-look
This section seems to be a more polished version of the mega manual Sections 5.3.4 to 5.3.11. It provides an overview to the build process, but its still at a very high level (few details).
Quoted 5 lines
If we wanted to add an appendix to list them all (and it might be worth us doing so) a good starting point would be the task descriptions in documentation.conf:
http://cgit.openembedded.org/openembedded-core/tree/meta/conf/documentation.conf
This file at least provides one sentence on most tasks. (do_setscene is missing, maybe more). Regards,
Bob
Quoted 4 lines
Cheers,
Paul
Quoted 92 lines · On Tuesday 06 May 2014 15:23:45 Bob Feretich wrote:
Quoted 91 lines · Paul Eggleton
Quoted 89 lines · Scott M
Quoted 85 lines
-----Original Message-----
From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Bob Feretich
Sent: Monday, May 05, 2014 4:52 PM
To: [email protected]
Subject: [yocto] Definition of Yocto tasksIs there a document that provides a detailed definition of what each these
tasks do?
Yocto, OE, and bitbake manuals tell us how to command a specific task to be
run, but not what they do.The task name provides a good hint sometimes, but often that is not enough.
Users shouldn't have to read the python code for this information. These descriptions should be a part of the yocto manual.Sections 5.3.4 to 5.3.11 of the Yocto mega-manual provide a good overview
of some of these tasks in the context of a workflow, but doesn't mention most of these tasks and doesn't go into enough detail on the tasks it discusses.It wouldn't be appropriate to add more detail at that point in the manual,
but it would be appropriate to include details regarding all of the tasks
in an appendix.Examples...
The manual states thatdo_fetchfetches source, but doesn't state that when
git is used whether it performs a pull, fetch, or clone. What are the common failure conditions ofdo_fetch? And what should the user do to fix
the problem?
do_buildis mentioned a few times, but there is no reference to it being the default task or what tasks are invoked bydo_buildand which are omitted.
do_rm_workanddo_wm_work_allare not mentioned at all.Result of listtasks:
do_fetchalldo_builddo_devshelldo_package_write_ipkdo_cleansstatedo_savedefconfigdo_uboot_mkimagedo_sizecheckdo_stripdo_packagedata_setscenedo_configuredo_cleando_deploy_setscenedo_cleanalldo_populate_licdo_populate_sysrootdo_devicetree_imagedo_deploydo_menuconfigdo_patchdo_bundle_initramfsdo_packagedatado_listtasksdo_compiledo_package_setscenedo_populate_lic_setscenedo_fetchdo_checkurido_compile_kernelmodulesdo_package_write_ipk_setscenedo_package_writedo_rm_workdo_packagedo_unpackdo_installdo_checkurialldo_populate_sysroot_setscenedo_rm_work_allQuoted 2 lines
I don't have the knowledge to create such an appendix, but I volunteer to be a proofreader.
An appendix for a reference of these tasks seems like a good idea.
Quoted 8 lines
Quoted 5 lines
FYI, you may already have seen it but we have a bit of coverage for the common>
tasks in the following section of the manual:
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#clo
ser-lookThis section seems to be a more polished version of the mega manual Sections 5.3.4 to 5.3.11.
They should be identical. The mega manual is simply the other manuals combined together...
Quoted 2 lines
It provides an overview to the build process, but its still at a very high level (few details).
Could you expand on the details are you looking for that you're not finding there?
Quoted 9 lines
Quoted 6 lines
If we wanted to add an appendix to list them all (and it might be worth us doing so) a good starting point would be the task descriptions in
documentation.conf:
http://cgit.openembedded.org/openembedded-core/tree/meta/conf/documenta
tion.confThis file at least provides one sentence on most tasks. (
do_setsceneis missing, maybe more).
do_setscene itself isn't a task that we have. Setscene equivalents exist for all of the sstate-enabled tasks that we have i.e. do_populate_sysroot_setscene is the setscene equivalent of do_populate_sysroot. We should touch on it elsewhere as well, but FYI we do have an explanation of the setscene process in the BitBake manual:
http://www.yoctoproject.org/docs/1.6/bitbake-user-manual/bitbake-user-manual.html#setscene
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Quoted 63 lines · Paul Eggleton
Quoted 54 lines · On Tuesday 06 May 2014 15:23:45 Bob Feretich wrote:
Quoted 53 lines · Paul Eggleton
Quoted 52 lines · Scott M
Quoted 52 lines
-----Original Message-----
From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Bob Feretich
Sent: Monday, May 05, 2014 4:52 PM
To: [email protected]
Subject: [yocto] Definition of Yocto tasksIs there a document that provides a detailed definition of what each these
tasks do?
Yocto, OE, and bitbake manuals tell us how to command a specific task to be
run, but not what they do.The task name provides a good hint sometimes, but often that is not enough.
Users shouldn't have to read the python code for this information. These descriptions should be a part of the yocto manual.Sections 5.3.4 to 5.3.11 of the Yocto mega-manual provide a good overview
of some of these tasks in the context of a workflow, but doesn't mention most of these tasks and doesn't go into enough detail on the tasks it discusses.It wouldn't be appropriate to add more detail at that point in the manual,
but it would be appropriate to include details regarding all of the tasks
in an appendix.Examples...
The manual states thatdo_fetchfetches source, but doesn't state that when
git is used whether it performs a pull, fetch, or clone. What are the common failure conditions ofdo_fetch? And what should the user do to fix
the problem?
do_buildis mentioned a few times, but there is no reference to it being the default task or what tasks are invoked bydo_buildand which are omitted.
do_rm_workanddo_wm_work_allare not mentioned at all.Result of listtasks:
do_fetchall
... snipped ...do_rm_work_allI don't have the knowledge to create such an appendix, but I volunteer to be a proofreader.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
An appendix for a reference of these tasks seems like a good idea.
FYI, you may already have seen it but we have a bit of coverage for the common>
tasks in the following section of the manual:
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#clo
ser-lookThis section seems to be a more polished version of the mega manual Sections 5.3.4 to 5.3.11.
Quoted 7 lines
They should be identical. The mega manual is simply the other manuals combined together...
Quoted 2 lines
It provides an overview to the build process, but its still at a very high level (few details).
Could you expand on the details are you looking for that you're not finding there?
I was looking for more detailed information because I was trying to trouble shoot failures that occurred during the "do_fetch" of some source.
The most common failure was the inability to access any of the servers that contained the source. This was probably due to those servers being temporarily down for maintenance. Simply restarting the bitbake fixed those problems. (A common failure that users should be told not to be concerned about. I did read that somewhere, but I don't remember where.)
The next problem was a recipe error effecting "do_install" of some recipe. I troubleshot this and got a fix from the Angstrom people. while troubleshooting (doing lots of double Control-Cs), I believe that I hung the "do_fetch" mechanism. (some "do_fetch" tasks were being executed in parallel with the broken "do_install"). After the "do_install" problem was fixed, "do_fetch" seemed to run, but didn't transfer any data (no network traffic), and it would eventually time out.
That raised the question regarding the use of lock files. (Did my Control-Cs leave a Yocto lock file intact that needed to be cleaned up? Did "do_fetch" use any locking mechanism? How could I do a partial "clean" that would fix the problem without setting my progress back further than necessary?)
During my search for info, I found e-mail discussions of clean, cleanstate, and cleanall. My "do_fetch" failure was occurring at about step 6900 of 8100 build steps. With my internet connection, restarting would have set me back 48 hrs, with no guarantee that the restart would result in a fix. I was looking for ways to "clean" the condition without having to completely start over. Eventually, I gave up and did the cleanall... it did not fix the problem. Even though none of cleanxxx tasks worked for me in fixing the "do_fetch" hang, info about them should have been easier to find.
I fixed the problem with a shotgun approach.
I erased the entire Yocto build directory, rebooted my host build system, and power cycled my network router and DSL modem. This worked, but was probably more than I needed to do. (I now think that my "do_fetch" hang was do to not properly reinitializing a firewall port.)
My searches also led me to the "do_fetchall" task, as I has just visited with a friend whose network connection was 10x better than mine. Had I know about fetchall, I would have used it during the visit.
In general, the appendix should contain a description of the program logic of the task, the task's intended use, and a discussion of common errors (and fixes) that can occur during execution of the task.
The most probable readers of the appendix would be:
* someone trying to troubleshoot a problem that is occuring during the task.
* someone who saw an e-mail reference for a use for the task and wants to understand it better.
Regards,
Bob
Quoted 20 lines
Quoted 9 lines
Quoted 7 lines
If we wanted to add an appendix to list them all (and it might be worth us doing so) a good starting point would be the task descriptions in
documentation.conf:
http://cgit.openembedded.org/openembedded-core/tree/meta/conf/documenta
tion.confThis file at least provides one sentence on most tasks. (
do_setsceneis missing, maybe more).
do_setsceneitself isn't a task that we have. Setscene equivalents exist for all of the sstate-enabled tasks that we have i.e.do_populate_sysroot_setsceneis the setscene equivalent ofdo_populate_sysroot. We should touch on it elsewhere as well, but FYI we do have an explanation of the setscene process in the BitBake manual:http://www.yoctoproject.org/docs/1.6/bitbake-user-manual/bitbake-user-manual.html#setscene
Cheers,
Paul
Bob,
This is good information regarding the process you had to go through and what would have helped along the way. I am currently working on a new chapter that documents the do_<task> tasks in the YP Reference Manual.
Scott
Quoted 156 lines
-----Original Message-----
From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Bob Feretich
Sent: Thursday, May 08, 2014 4:25 PM
To: [email protected]
Subject: Re: [yocto] Definition of Yocto tasksQuoted 46 lines · Paul Eggleton
Quoted 45 lines · On Tuesday 06 May 2014 15:23:45 Bob Feretich wrote:
Quoted 44 lines · Paul Eggleton
Quoted 43 lines · Scott M
Quoted 43 lines
-----Original Message-----
From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Bob Feretich
Sent: Monday, May 05, 2014 4:52 PM
To: [email protected]
Subject: [yocto] Definition of Yocto tasksIs there a document that provides a detailed definition of what each these tasks do?
Yocto, OE, and bitbake manuals tell us how to command a specific task to be run, but not what they do.The task name provides a good hint sometimes, but often that is not enough.
Users shouldn't have to read the python code for this information. These descriptions should be a part of the yocto manual.Sections 5.3.4 to 5.3.11 of the Yocto mega-manual provide a good overview of some of these tasks in the context of a workflow, but doesn't mention most of these tasks and doesn't go into enough detail on the tasks it discusses.
It wouldn't be appropriate to add more detail at that point in the manual, but it would be appropriate to include details regarding all of the tasks in an appendix.
Examples...
The manual states thatdo_fetchfetches source, but doesn't state that when git is used whether it performs a pull, fetch, or clone. What are the common failure conditions ofdo_fetch? And what should the user do to fix the problem?
do_buildis mentioned a few times, but there is no reference to it being the default task or what tasks are invoked bydo_buildand which are omitted.
do_rm_workanddo_wm_work_allare not mentioned at all.Result of listtasks:
do_fetchall
... snipped ...do_rm_work_allQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 2 lines
I don't have the knowledge to create such an appendix, but I volunteer to be a proofreader.
An appendix for a reference of these tasks seems like a good idea.
FYI, you may already have seen it but we have a bit of coverage for the
common>
tasks in the following section of the manual:
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#clo
Quoted 10 lines
Quoted 3 lines
Quoted 1 line
ser-look
This section seems to be a more polished version of the mega manual Sections 5.3.4 to 5.3.11.
They should be identical. The mega manual is simply the other manuals combined together...
Quoted 2 lines
It provides an overview to the build process, but its still at a very high level (few details).
Could you expand on the details are you looking for that you're not finding there?
I was looking for more detailed information because I was trying to trouble shoot failures that occurred during the "
do_fetch" of some source. The most common failure was the inability to access any of the servers that contained the source. This was probably due to those servers being temporarily down for maintenance. Simply restarting the bitbake fixed those problems.
(A common failure that users should be told not to be concerned about. I did read that somewhere, but I don't remember where.)The next problem was a recipe error effecting "
do_install" of some recipe. I troubleshot this and got a fix from the Angstrom people. while troubleshooting (doing lots of double Control-Cs), I believe that I hung the "do_fetch" mechanism. (some "do_fetch" tasks were being executed in parallel with the broken "do_install"). After the "do_install" problem was fixed, "do_fetch" seemed to run, but didn't transfer any data (no network traffic), and it would eventually time out.That raised the question regarding the use of lock files. (Did my Control-Cs leave a Yocto lock file intact that needed to be cleaned up?
Did "do_fetch" use any locking mechanism? How could I do a partial "clean" that would fix the problem without setting my progress back further than necessary?)During my search for info, I found e-mail discussions of clean, cleanstate, and cleanall. My "
do_fetch" failure was occurring at about step 6900 of 8100 build steps. With my internet connection, restarting would have set me back 48 hrs, with no guarantee that the restart would result in a fix. I was looking for ways to "clean" the condition without having to completely start over. Eventually, I gave up and did the cleanall... it did not fix the problem.
Even though none of cleanxxx tasks worked for me in fixing the "do_fetch" hang, info about them should have been easier to find.I fixed the problem with a shotgun approach.
I erased the entire Yocto build directory, rebooted my host build system, and power cycled my network router and DSL modem. This worked, but was probably more than I needed to do. (I now think that my "do_fetch" hang was do to not properly reinitializing a firewall port.)My searches also led me to the "
do_fetchall" task, as I has just visited with a friend whose network connection was 10x better than mine. Had I know about fetchall, I would have used it during the visit.In general, the appendix should contain a description of the program logic of the task, the task's intended use, and a discussion of common errors (and fixes) that can occur during execution of the task.
The most probable readers of the appendix would be:
* someone trying to troubleshoot a problem that is occuring during the task.
* someone who saw an e-mail reference for a use for the task and wants to understand it better.Regards,
BobQuoted 7 lines
Quoted 7 lines
Quoted 7 lines
If we wanted to add an appendix to list them all (and it might be worth us doing so) a good starting point would be the task descriptions in
documentation.conf:
http://cgit.openembedded.org/openembedded-core/tree/meta/conf/documenta
Quoted 9 lines
Quoted 3 lines
Quoted 1 line
tion.conf
This file at least provides one sentence on most tasks. (
do_setsceneis missing, maybe more).
do_setsceneitself isn't a task that we have. Setscene equivalents exist for all of the sstate-enabled tasks that we have i.e.do_populate_sysroot_setsceneis the setscene equivalent ofdo_populate_sysroot. We should touch on it elsewhere as well, but FYI we do have an explanation of the setscene process in the BitBake manual:Quoted 6 lines
http://www.yoctoproject.org/docs/1.6/bitbake-user-manual/bitbake-user-
manual.html#setsceneCheers,
Paul--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto