Yocto Technical Team Minutes, Engineering Sync, for July 28 2020
Yocto Technical Team Minutes, Engineering Sync, for July 28 2020
archive: https://docs.google.com/document/d/1ly8nyhO14kDNnFcW2QskANXW3ZT7QwKC5wWVDg9dDH4/edit
== disclaimer ==
Best efforts are made to ensure the below is accurate and valid. However, errors sometimes happen. If any errors or omissions are found, please feel free to reply to this email with any corrections.
== attendees ==
Trevor Woerner, Stephen Jolly, Armin Kuster, Josef Holzmayr, Richard Purdie, Trevor Gamblin, Joshua Watt, Mark Morton, Ross Burton, Bruce Ashfield, Steve Sakoman, (phone-in ??), Jon Mason, Randy MacLeod, Scott Murray, Denys Dmytriyenko
== notes ==
- fixed a race on AB wrt perl
- 3.1.2 into QA
- 3.2-m2 to QA once they’re done with 3.1.2
== general ==
RP: load of perl issues over the weekend, believe to have found issue
RP: lots of qemumips issues, not sure why, perhaps a timeouts issue? (no ping
within 30 seconds)
RP: some infrastructure issues (one worker has become corrupted)
RP: curious to know what people think about 3.1.2: there is a bitbake issue
with toaster, should we release then fix or wait for fix before release?
SS: maybe we should get feedback from people using toaster
RP: the fix is simple, maybe release 3.1.2 (with release note) then fix
RP: merged re-arrangement of package manager code, this puts more of our code
into python libraries instead of bbclass files, which is the direction i want to take going forward
RP: i need to start a conversation on oe-architecture for future directions
RP: e.g. binary package feeds, we want to capture these use-cases into the
wiki
[NB: done! see https://wiki.yoctoproject.org/wiki/Future_Directions]
RP: we also posted the current agreement on “inclusive language” as agreed
on by various groups (oe board, oe-tsc, yp-tsc)
TW: what’s the thinking behind moving code to libraries vs bbclass?
RP: pro: parsing speed con: variable dependency tracking is lost
RP: lots of python code gets written to logs etc, which slows it down
JPEW: does bitbake automatically detect variable dependencies in library code?
RP: no. maybe we just need to do the scan once?
TW: do we lose flexibility? (e.g. it’s easy to copy+paste+modify a bbclass,
will that be possible?)
RP: there will still be all the same entry points, so it should be possible
RP: i’m worried about the size of the datastore in memory and it is a
bottleneck
RP: i wonder why qemumips is so slow?
Randy: i think Cisco are the only ones caring about mips
RP: yes, and Comcast too i think
MM: have been wrapped up in $WORK stuff, but will be working on docs
conversion soon
RP: MH has been putting infrastructure in place and Nico is looking forward to
getting this going
MM: do we want to try to reproduce our current colours/formatting or try
something new
RP: i like what we have, the project does have an existing scheme, but we can
change if something else makes sense
TW: i do lots of builds, never seen issues, why are there so many failures on
AB?
RP: do you do oe-selftest or testsdk builds?
TW: no
RP: do you do mips builds?
TW: no
RP: so these are the areas where we see these failures, “regular” builds
are okay
SS: i don’t see issues on my builds but that’s probably because of working
these issues out on the AB
RP: infrastructure issues are low, it’s mostly race issues that come out of
the extreme load of the AB or intermittent architecture issues that are rarely tested
Quoted 58 lines · Trevor Woerner
Yocto Technical Team Minutes, Engineering Sync, for July 28 2020
archive: https://docs.google.com/document/d/1ly8nyhO14kDNnFcW2QskANXW3ZT7QwKC5wWVDg9dDH4/edit== disclaimer ==
Best efforts are made to ensure the below is accurate and valid. However, errors sometimes happen. If any errors or omissions are found, please feel free to reply to this email with any corrections.== attendees ==
Trevor Woerner, Stephen Jolly, Armin Kuster, Josef Holzmayr, Richard Purdie, Trevor Gamblin, Joshua Watt, Mark Morton, Ross Burton, Bruce Ashfield, Steve Sakoman, (phone-in ??), Jon Mason, Randy MacLeod, Scott Murray, Denys Dmytriyenko== notes ==
- fixed a race on AB wrt perl
- 3.1.2 into QA
- 3.2-m2 to QA once they’re done with 3.1.2== general ==
RP: load of perl issues over the weekend, believe to have found issue
RP: lots of qemumips issues, not sure why, perhaps a timeouts issue? (no ping
within 30 seconds)
RP: some infrastructure issues (one worker has become corrupted)RP: curious to know what people think about 3.1.2: there is a bitbake issue
with toaster, should we release then fix or wait for fix before release?
SS: maybe we should get feedback from people using toaster
RP: the fix is simple, maybe release 3.1.2 (with release note) then fixRP: merged re-arrangement of package manager code, this puts more of our code
into python libraries instead of bbclass files, which is the direction i want to take going forward
RP: i need to start a conversation on oe-architecture for future directions
RP: e.g. binary package feeds, we want to capture these use-cases into the
wiki[NB: done! see https://wiki.yoctoproject.org/wiki/Future_Directions]
RP: we also posted the current agreement on “inclusive language” as agreed
on by various groups (oe board, oe-tsc, yp-tsc)TW: what’s the thinking behind moving code to libraries vs bbclass?
RP: pro: parsing speed con: variable dependency tracking is lost
RP: lots of python code gets written to logs etc, which slows it down
JPEW: does bitbake automatically detect variable dependencies in library code?
RP: no. maybe we just need to do the scan once?
TW: do we lose flexibility? (e.g. it’s easy to copy+paste+modify a bbclass,
will that be possible?)
RP: there will still be all the same entry points, so it should be possible
RP: i’m worried about the size of the datastore in memory and it is a
bottleneckRP: i wonder why qemumips is so slow?
Randy: i think Cisco are the only ones caring about mips
RP: yes, and Comcast too i think
Quoted 8 lines
MM: have been wrapped up in $WORK stuff, but will be working on docs
conversion soon
RP: MH has been putting infrastructure in place and Nico is looking forward to
getting this going
MM: do we want to try to reproduce our current colours/formatting or try
something new
RP: i like what we have, the project does have an existing scheme, but we can
change if something else makes sense
I have some updates on Sphinx..
I recall that the work in progress can be found here:
https://git.yoctoproject.org/cgit/cgit.cgi/yocto-docs/log/?h=sphinx
The automated conversion from DocBook to Sphinx is managing 'internal' links quite badly, so it results in a significant amount of work to fixup links. Also since I wanted to use the Sphinx builtin 'glossary' for terms and variables in the reference manual, that also required me to update how links work for terms and variables. That's why it's taking more time than I would like. I manage to use some automation/regexp, and do a bit of manual fix up as well.
I also started to experiment with customization in the CSS theme, to change some color/formatting. It's not done yet, but it has got better:
https://people.linaro.org/~nicolas.dechesne/yp-docs/html/ref-manual/ref-manual.html
For now we generate a single large documentation 'website' that includes all existing docs, instead of publishing each document separately. I think it's an important change compared to how our documentation was published so far. I personally like that quite a bit, and think it's 'better' this way, but I would really want to hear from you and that we agree this is what we want for the project. We still need to look at how we want to generate PDF/offline documentation though. And we need to decide how to manage the bitbake manual which sits in the bitbake git tree, so right now it's published separately.
Side note: the VSCode extension for reST is quite good
(https://docs.restructuredtext.net/index.html)
Quoted 15 lines
TW: i do lots of builds, never seen issues, why are there so many failures on
AB?RP: do you do oe-selftest or testsdk builds?
TW: no
RP: do you do mips builds?
TW: no
RP: so these are the areas where we see these failures, “regular” builds
are okay
SS: i don’t see issues on my builds but that’s probably because of working
these issues out on the AB
RP: infrastructure issues are low, it’s mostly race issues that come out of
the extreme load of the AB or intermittent architecture issues that are rarely tested
Quoted 105 lines
-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Nicolas Dechesne
Sent: Wednesday, July 29, 2020 12:37 AM
To: Trevor Woerner <[email protected]>
Cc: [email protected]; Morton, Mark <[email protected]>
Subject: Re: [docs] [yocto] Yocto Technical Team Minutes, Engineering Sync, for July 28 2020Quoted 4 lines · Trevor Woerner
Yocto Technical Team Minutes, Engineering Sync, for July 28 2020 archive:
https://docs.google.com/document/d/1ly8nyhO14kDNnFcW2QskANXW3ZT7
QwKC5wQuoted 13 lines
WVDg9dDH4/edit
== disclaimer ==
Best efforts are made to ensure the below is accurate and valid. However, errors sometimes happen. If any errors or omissions are found, please feel free to reply to this email with any corrections.== attendees ==
Trevor Woerner, Stephen Jolly, Armin Kuster, Josef Holzmayr, Richard Purdie, Trevor Gamblin, Joshua Watt, Mark Morton, Ross Burton, Bruce Ashfield, Steve Sakoman, (phone-in ??), Jon Mason, Randy MacLeod, Scott Murray, Denys DmytriyenkoQuoted 8 lines
== notes ==
- fixed a race on AB wrt perl
- 3.1.2 into QA
- 3.2-m2 to QA once they’re done with 3.1.2== general ==
RP: load of perl issues over the weekend, believe to have found issue
RP: lots of qemumips issues, not sure why, perhaps a timeouts issue? (noping
Quoted 10 lines
within 30 seconds)
RP: some infrastructure issues (one worker has become corrupted)RP: curious to know what people think about 3.1.2: there is a bitbake issue
with toaster, should we release then fix or wait for fix before release?
SS: maybe we should get feedback from people using toaster
RP: the fix is simple, maybe release 3.1.2 (with release note) then fixRP: merged re-arrangement of package manager code, this puts more of
our code
Quoted 9 lines
into python libraries instead of bbclass files, which is the direction i want to take going forward
RP: i need to start a conversation on oe-architecture for future directions
RP: e.g. binary package feeds, we want to capture these use-cases into the
wiki[NB: done! see https://wiki.yoctoproject.org/wiki/Future_Directions]
Quoted 7 lines
RP: we also posted the current agreement on “inclusive language” as agreed
on by various groups (oe board, oe-tsc, yp-tsc)TW: what’s the thinking behind moving code to libraries vs bbclass?
RP: pro: parsing speed con: variable dependency tracking is lost
RP: lots of python code gets written to logs etc, which slows it down
JPEW: does bitbake automatically detect variable dependencies in librarycode?
Quoted 8 lines
RP: no. maybe we just need to do the scan once?
TW: do we lose flexibility? (e.g. it’s easy to copy+paste+modify a bbclass,
will that be possible?)
RP: there will still be all the same entry points, so it should be possible
RP: i’m worried about the size of the datastore in memory and it is a
bottleneckQuoted 7 lines
RP: i wonder why qemumips is so slow?
Randy: i think Cisco are the only ones caring about mips
RP: yes, and Comcast too i thinkMM: have been wrapped up in $WORK stuff, but will be working on docs
conversion soon
RP: MH has been putting infrastructure in place and Nico is looking forwardto
Quoted 4 lines
getting this going
MM: do we want to try to reproduce our current colours/formatting or try
something new
RP: i like what we have, the project does have an existing scheme, but wecan
Quoted 1 line
change if something else makes sense
I have some updates on Sphinx..
I recall that the work in progress can be found here:
https://git.yoctoproject.org/cgit/cgit.cgi/yocto-docs/log/?h=sphinxThe automated conversion from DocBook to Sphinx is managing 'internal' links quite badly, so it results in a significant amount of work to fixup links. Also since I wanted to use the Sphinx builtin 'glossary'
for terms and variables in the reference manual, that also required me to update how links work for terms and variables. That's why it's taking more time than I would like. I manage to use some automation/regexp, and do a bit of manual fix up as well.
Quoted 5 lines
I also started to experiment with customization in the CSS theme, to change some color/formatting. It's not done yet, but it has got better:
https://people.linaro.org/~nicolas.dechesne/yp-docs/html/ref-manual/ref-
manual.html
I see you got rid of some of the more obvius "funny colors" - the Notes look much better, for example.
I cloned the new sphinx branch and started looking at the implementation some. To make these changes, are you updating the repoDir/documentation/sphinx-static/theme_overrides.css file, or is there somewhere else you're making the changes?
Thanks,
-Mark M
Quoted 28 lines
For now we generate a single large documentation 'website' that includes all existing docs, instead of publishing each document separately. I think it's an important change compared to how our documentation was published so far. I personally like that quite a bit, and think it's 'better' this way, but I would really want to hear from you and that we agree this is what we want for the project. We still need to look at how we want to generate PDF/offline documentation though. And we need to decide how to manage the bitbake manual which sits in the bitbake git tree, so right now it's published separately.
Side note: the VSCode extension for reST is quite good
(https://docs.restructuredtext.net/index.html)Quoted 15 lines
TW: i do lots of builds, never seen issues, why are there so many failures on
AB?RP: do you do oe-selftest or testsdk builds?
TW: no
RP: do you do mips builds?
TW: no
RP: so these are the areas where we see these failures, “regular” builds
are okay
SS: i don’t see issues on my builds but that’s probably because of working
these issues out on the AB
RP: infrastructure issues are low, it’s mostly race issues that come out of
the extreme load of the AB or intermittent architecture issues that are rarely tested
Le ven. 31 juil. 2020 à 22:06, Morton, Mark <[email protected]> a écrit :
Quoted 128 lines
Quoted 4 lines
-----Original Message-----
From: [email protected] <[email protected]> OnBehalf
Quoted 6 lines
Of Nicolas Dechesne
Sent: Wednesday, July 29, 2020 12:37 AM
To: Trevor Woerner <[email protected]>
Cc: [email protected]; Morton, Mark <[email protected]>
Subject: Re: [docs] [yocto] Yocto Technical Team Minutes, EngineeringSync,
Quoted 31 lines
for July 28 2020
Quoted 4 lines · Trevor Woerner
Yocto Technical Team Minutes, Engineering Sync, for July 28 2020 archive:
https://docs.google.com/document/d/1ly8nyhO14kDNnFcW2QskANXW3ZT7
QwKC5wQuoted 21 lines
WVDg9dDH4/edit
== disclaimer ==
Best efforts are made to ensure the below is accurate and valid. However, errors sometimes happen. If any errors or omissions are found, please feel free to reply to this email with any corrections.== attendees ==
Trevor Woerner, Stephen Jolly, Armin Kuster, Josef Holzmayr, Richard Purdie, Trevor Gamblin, Joshua Watt, Mark Morton, Ross Burton, Bruce Ashfield, Steve Sakoman, (phone-in ??), Jon Mason, Randy MacLeod, Scott Murray, Denys Dmytriyenko== notes ==
- fixed a race on AB wrt perl
- 3.1.2 into QA
- 3.2-m2 to QA once they’re done with 3.1.2== general ==
RP: load of perl issues over the weekend, believe to have found issue
RP: lots of qemumips issues, not sure why, perhaps a timeouts issue?(no
Quoted 5 lines
ping
Quoted 4 lines
within 30 seconds)
RP: some infrastructure issues (one worker has become corrupted)RP: curious to know what people think about 3.1.2: there is a bitbake
issue
Quoted 1 line
Quoted 1 line
with toaster, should we release then fix or wait for fix before
release?
Quoted 7 lines
Quoted 5 lines
SS: maybe we should get feedback from people using toaster
RP: the fix is simple, maybe release 3.1.2 (with release note) then fixRP: merged re-arrangement of package manager code, this puts more of
our code
Quoted 1 line
into python libraries instead of bbclass files, which is the
direction i
Quoted 4 lines
Quoted 4 lines
want to take going forward
RP: i need to start a conversation on oe-architecture for future directions
RP: e.g. binary package feeds, we want to capture these use-cases intothe
Quoted 5 lines
Quoted 5 lines
wiki
[NB: done! see https://wiki.yoctoproject.org/wiki/Future_Directions]
RP: we also posted the current agreement on “inclusive language” as
agreed
Quoted 6 lines
Quoted 6 lines
on by various groups (oe board, oe-tsc, yp-tsc)
TW: what’s the thinking behind moving code to libraries vs bbclass?
RP: pro: parsing speed con: variable dependency tracking is lost
RP: lots of python code gets written to logs etc, which slows it down
JPEW: does bitbake automatically detect variable dependencies inlibrary
Quoted 3 lines
code?
Quoted 2 lines
RP: no. maybe we just need to do the scan once?
TW: do we lose flexibility? (e.g. it’s easy to copy+paste+modify abbclass,
Quoted 6 lines
Quoted 6 lines
will that be possible?)
RP: there will still be all the same entry points, so it should be possible
RP: i’m worried about the size of the datastore in memory and it is a
bottleneckQuoted 7 lines
Quoted 7 lines
RP: i wonder why qemumips is so slow?
Randy: i think Cisco are the only ones caring about mips
RP: yes, and Comcast too i thinkMM: have been wrapped up in $WORK stuff, but will be working on docs
conversion soon
RP: MH has been putting infrastructure in place and Nico is lookingforward
Quoted 3 lines
to
Quoted 2 lines
getting this going
MM: do we want to try to reproduce our current colours/formatting ortry
Quoted 2 lines
Quoted 2 lines
something new
RP: i like what we have, the project does have an existing scheme, butwe
Quoted 9 lines
can
Quoted 1 line
change if something else makes sense
I have some updates on Sphinx..
I recall that the work in progress can be found here:
https://git.yoctoproject.org/cgit/cgit.cgi/yocto-docs/log/?h=sphinxThe automated conversion from DocBook to Sphinx is managing 'internal' links quite badly, so it results in a significant amount of work to
fixup links. Also
Quoted 3 lines
since I wanted to use the Sphinx builtin 'glossary'
for terms and variables in the reference manual, that also required me to update how links work for terms and variables. That's why it's takingmore
Quoted 1 line
time than I would like. I manage to use some automation/regexp, and do a
bit
Quoted 3 lines
of manual fix up as well.
I also started to experiment with customization in the CSS theme, to
change
Quoted 4 lines
some color/formatting. It's not done yet, but it has got better:
https://people.linaro.org/~nicolas.dechesne/yp-docs/html/ref-manual/ref-
manual.html
Quoted 6 lines
I see you got rid of some of the more obvius "funny colors" - the Notes look much better, for example.
I cloned the new sphinx branch and started looking at the implementation some. To make these changes, are you updating the repoDir/documentation/sphinx-static/theme_overrides.css file, or is there somewhere else you're making the changes?
Yes. All CSS customization is done in this file. It overrides the css coming from the base theme template that we use.
Thanks for your help! I definitely need some help!
Quoted 44 lines
Thanks,
-Mark MQuoted 2 lines
For now we generate a single large documentation 'website' that includes
all
Quoted 1 line
existing docs, instead of publishing each document separately. I think
it's an
Quoted 1 line
important change compared to how our documentation was published so far.
I
Quoted 1 line
personally like that quite a bit, and think it's 'better' this way, but
I would really
Quoted 1 line
want to hear from you and that we agree this is what we want for the
project.
Quoted 1 line
We still need to look at how we want to generate PDF/offline
documentation
Quoted 8 lines
though. And we need to decide how to manage the bitbake manual which sits in the bitbake git tree, so right now it's published separately.
Side note: the VSCode extension for reST is quite good
(https://docs.restructuredtext.net/index.html)Quoted 2 lines
TW: i do lots of builds, never seen issues, why are there so many
failures on
Quoted 6 lines
Quoted 6 lines
AB?RP: do you do oe-selftest or testsdk builds?
TW: no
RP: do you do mips builds?
TW: no
RP: so these are the areas where we see these failures, “regular”builds
Quoted 2 lines
Quoted 2 lines
are okay
SS: i don’t see issues on my builds but that’s probably because ofworking
Quoted 2 lines
Quoted 2 lines
these issues out on the AB
RP: infrastructure issues are low, it’s mostly race issues that comeout of
Quoted 1 line
Quoted 1 line
the extreme load of the AB or intermittent architecture issues
that are
Quoted 2 lines
Quoted 2 lines
rarely tested