Skip to contentSkip to conversation

Yocto Technical Team Minutes, Engineering Sync, for July 28 2020

DISCUSSION Trevor Woerner 3 participants 4 messages Jul 28, 2020 – Jul 31, 2020

lore
Share & tools
lore
#0 lore raw

Yocto Technical Team Minutes, Engineering Sync, for July 28 2020
archive: https://docs.google.com/document/d/1ly8nyhO14kDNnFcW2QskANXW3ZT7QwKC5wWVDg9dDH4/edit

== disclaimer ==
Best efforts are made to ensure the below is accurate and valid. However, errors sometimes happen. If any errors or omissions are found, please feel free to reply to this email with any corrections.

== attendees ==
Trevor Woerner, Stephen Jolly, Armin Kuster, Josef Holzmayr, Richard Purdie, Trevor Gamblin, Joshua Watt, Mark Morton, Ross Burton, Bruce Ashfield, Steve Sakoman, (phone-in ??), Jon Mason, Randy MacLeod, Scott Murray, Denys Dmytriyenko

== notes ==
- fixed a race on AB wrt perl
- 3.1.2 into QA
- 3.2-m2 to QA once they’re done with 3.1.2

== general ==
RP: load of perl issues over the weekend, believe to have found issue
RP: lots of qemumips issues, not sure why, perhaps a timeouts issue? (no ping
within 30 seconds)
RP: some infrastructure issues (one worker has become corrupted)

RP: curious to know what people think about 3.1.2: there is a bitbake issue
with toaster, should we release then fix or wait for fix before release?
SS: maybe we should get feedback from people using toaster
RP: the fix is simple, maybe release 3.1.2 (with release note) then fix

RP: merged re-arrangement of package manager code, this puts more of our code
into python libraries instead of bbclass files, which is the direction i want to take going forward
RP: i need to start a conversation on oe-architecture for future directions
RP: e.g. binary package feeds, we want to capture these use-cases into the
wiki

[NB: done! see https://wiki.yoctoproject.org/wiki/Future_Directions]

RP: we also posted the current agreement on “inclusive language” as agreed
on by various groups (oe board, oe-tsc, yp-tsc)

TW: what’s the thinking behind moving code to libraries vs bbclass?
RP: pro: parsing speed con: variable dependency tracking is lost
RP: lots of python code gets written to logs etc, which slows it down
JPEW: does bitbake automatically detect variable dependencies in library code?
RP: no. maybe we just need to do the scan once?
TW: do we lose flexibility? (e.g. it’s easy to copy+paste+modify a bbclass,
will that be possible?)
RP: there will still be all the same entry points, so it should be possible
RP: i’m worried about the size of the datastore in memory and it is a
bottleneck

RP: i wonder why qemumips is so slow?
Randy: i think Cisco are the only ones caring about mips
RP: yes, and Comcast too i think

MM: have been wrapped up in $WORK stuff, but will be working on docs
conversion soon
RP: MH has been putting infrastructure in place and Nico is looking forward to
getting this going
MM: do we want to try to reproduce our current colours/formatting or try
something new
RP: i like what we have, the project does have an existing scheme, but we can
change if something else makes sense

TW: i do lots of builds, never seen issues, why are there so many failures on

    AB?

RP: do you do oe-selftest or testsdk builds?
TW: no
RP: do you do mips builds?
TW: no
RP: so these are the areas where we see these failures, “regular” builds
are okay
SS: i don’t see issues on my builds but that’s probably because of working
these issues out on the AB
RP: infrastructure issues are low, it’s mostly race issues that come out of
the extreme load of the AB or intermittent architecture issues that are rarely tested

#1 lore raw
Quoted 58 lines · Trevor Woerner

Yocto Technical Team Minutes, Engineering Sync, for July 28 2020
archive: https://docs.google.com/document/d/1ly8nyhO14kDNnFcW2QskANXW3ZT7QwKC5wWVDg9dDH4/edit

== disclaimer ==
Best efforts are made to ensure the below is accurate and valid. However, errors sometimes happen. If any errors or omissions are found, please feel free to reply to this email with any corrections.

== attendees ==
Trevor Woerner, Stephen Jolly, Armin Kuster, Josef Holzmayr, Richard Purdie, Trevor Gamblin, Joshua Watt, Mark Morton, Ross Burton, Bruce Ashfield, Steve Sakoman, (phone-in ??), Jon Mason, Randy MacLeod, Scott Murray, Denys Dmytriyenko

== notes ==
- fixed a race on AB wrt perl
- 3.1.2 into QA
- 3.2-m2 to QA once they’re done with 3.1.2

== general ==
RP: load of perl issues over the weekend, believe to have found issue
RP: lots of qemumips issues, not sure why, perhaps a timeouts issue? (no ping
within 30 seconds)
RP: some infrastructure issues (one worker has become corrupted)

RP: curious to know what people think about 3.1.2: there is a bitbake issue
with toaster, should we release then fix or wait for fix before release?
SS: maybe we should get feedback from people using toaster
RP: the fix is simple, maybe release 3.1.2 (with release note) then fix

RP: merged re-arrangement of package manager code, this puts more of our code
into python libraries instead of bbclass files, which is the direction i want to take going forward
RP: i need to start a conversation on oe-architecture for future directions
RP: e.g. binary package feeds, we want to capture these use-cases into the
wiki

[NB: done! see https://wiki.yoctoproject.org/wiki/Future_Directions]

RP: we also posted the current agreement on “inclusive language” as agreed
on by various groups (oe board, oe-tsc, yp-tsc)

TW: what’s the thinking behind moving code to libraries vs bbclass?
RP: pro: parsing speed con: variable dependency tracking is lost
RP: lots of python code gets written to logs etc, which slows it down
JPEW: does bitbake automatically detect variable dependencies in library code?
RP: no. maybe we just need to do the scan once?
TW: do we lose flexibility? (e.g. it’s easy to copy+paste+modify a bbclass,
will that be possible?)
RP: there will still be all the same entry points, so it should be possible
RP: i’m worried about the size of the datastore in memory and it is a
bottleneck

RP: i wonder why qemumips is so slow?
Randy: i think Cisco are the only ones caring about mips
RP: yes, and Comcast too i think

Quoted 8 lines

MM: have been wrapped up in $WORK stuff, but will be working on docs
conversion soon
RP: MH has been putting infrastructure in place and Nico is looking forward to
getting this going
MM: do we want to try to reproduce our current colours/formatting or try
something new
RP: i like what we have, the project does have an existing scheme, but we can
change if something else makes sense

I have some updates on Sphinx..
I recall that the work in progress can be found here:
https://git.yoctoproject.org/cgit/cgit.cgi/yocto-docs/log/?h=sphinx

The automated conversion from DocBook to Sphinx is managing 'internal' links quite badly, so it results in a significant amount of work to fixup links. Also since I wanted to use the Sphinx builtin 'glossary' for terms and variables in the reference manual, that also required me to update how links work for terms and variables. That's why it's taking more time than I would like. I manage to use some automation/regexp, and do a bit of manual fix up as well.

I also started to experiment with customization in the CSS theme, to change some color/formatting. It's not done yet, but it has got better:
https://people.linaro.org/~nicolas.dechesne/yp-docs/html/ref-manual/ref-manual.html

For now we generate a single large documentation 'website' that includes all existing docs, instead of publishing each document separately. I think it's an important change compared to how our documentation was published so far. I personally like that quite a bit, and think it's 'better' this way, but I would really want to hear from you and that we agree this is what we want for the project. We still need to look at how we want to generate PDF/offline documentation though. And we need to decide how to manage the bitbake manual which sits in the bitbake git tree, so right now it's published separately.

Side note: the VSCode extension for reST is quite good
(https://docs.restructuredtext.net/index.html)

Quoted 15 lines

TW: i do lots of builds, never seen issues, why are there so many failures on

    AB?

RP: do you do oe-selftest or testsdk builds?
TW: no
RP: do you do mips builds?
TW: no
RP: so these are the areas where we see these failures, “regular” builds
are okay
SS: i don’t see issues on my builds but that’s probably because of working
these issues out on the AB
RP: infrastructure issues are low, it’s mostly race issues that come out of
the extreme load of the AB or intermittent architecture issues that are rarely tested

#2 lore raw
Quoted 105 lines

-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Nicolas Dechesne
Sent: Wednesday, July 29, 2020 12:37 AM
To: Trevor Woerner <[email protected]>
Cc: [email protected]; Morton, Mark <[email protected]>
Subject: Re: [docs] [yocto] Yocto Technical Team Minutes, Engineering Sync, for July 28 2020

Quoted 4 lines · Trevor Woerner

Yocto Technical Team Minutes, Engineering Sync, for July 28 2020 archive:

https://docs.google.com/document/d/1ly8nyhO14kDNnFcW2QskANXW3ZT7
QwKC5w

Quoted 13 lines

WVDg9dDH4/edit

== disclaimer ==
Best efforts are made to ensure the below is accurate and valid. However, errors sometimes happen. If any errors or omissions are found, please feel free to reply to this email with any corrections.

== attendees ==
Trevor Woerner, Stephen Jolly, Armin Kuster, Josef Holzmayr, Richard Purdie, Trevor Gamblin, Joshua Watt, Mark Morton, Ross Burton, Bruce Ashfield, Steve Sakoman, (phone-in ??), Jon Mason, Randy MacLeod, Scott Murray, Denys Dmytriyenko

Quoted 8 lines

== notes ==
- fixed a race on AB wrt perl
- 3.1.2 into QA
- 3.2-m2 to QA once they’re done with 3.1.2

== general ==
RP: load of perl issues over the weekend, believe to have found issue
RP: lots of qemumips issues, not sure why, perhaps a timeouts issue? (no

ping

Quoted 10 lines

within 30 seconds)
RP: some infrastructure issues (one worker has become corrupted)

RP: curious to know what people think about 3.1.2: there is a bitbake issue
with toaster, should we release then fix or wait for fix before release?
SS: maybe we should get feedback from people using toaster
RP: the fix is simple, maybe release 3.1.2 (with release note) then fix

RP: merged re-arrangement of package manager code, this puts more of

our code

Quoted 9 lines

into python libraries instead of bbclass files, which is the direction i want to take going forward
RP: i need to start a conversation on oe-architecture for future directions
RP: e.g. binary package feeds, we want to capture these use-cases into the
wiki

[NB: done! see https://wiki.yoctoproject.org/wiki/Future_Directions]

Quoted 7 lines

RP: we also posted the current agreement on “inclusive language” as agreed
on by various groups (oe board, oe-tsc, yp-tsc)

TW: what’s the thinking behind moving code to libraries vs bbclass?
RP: pro: parsing speed con: variable dependency tracking is lost
RP: lots of python code gets written to logs etc, which slows it down
JPEW: does bitbake automatically detect variable dependencies in library

code?

Quoted 8 lines

RP: no. maybe we just need to do the scan once?
TW: do we lose flexibility? (e.g. it’s easy to copy+paste+modify a bbclass,
will that be possible?)
RP: there will still be all the same entry points, so it should be possible
RP: i’m worried about the size of the datastore in memory and it is a
bottleneck

Quoted 7 lines

RP: i wonder why qemumips is so slow?
Randy: i think Cisco are the only ones caring about mips
RP: yes, and Comcast too i think

MM: have been wrapped up in $WORK stuff, but will be working on docs
conversion soon
RP: MH has been putting infrastructure in place and Nico is looking forward

to

Quoted 4 lines

getting this going
MM: do we want to try to reproduce our current colours/formatting or try
something new
RP: i like what we have, the project does have an existing scheme, but we

can

Quoted 1 line

change if something else makes sense

I have some updates on Sphinx..
I recall that the work in progress can be found here:
https://git.yoctoproject.org/cgit/cgit.cgi/yocto-docs/log/?h=sphinx

The automated conversion from DocBook to Sphinx is managing 'internal' links quite badly, so it results in a significant amount of work to fixup links. Also since I wanted to use the Sphinx builtin 'glossary'
for terms and variables in the reference manual, that also required me to update how links work for terms and variables. That's why it's taking more time than I would like. I manage to use some automation/regexp, and do a bit of manual fix up as well.

Quoted 5 lines

I also started to experiment with customization in the CSS theme, to change some color/formatting. It's not done yet, but it has got better:
https://people.linaro.org/~nicolas.dechesne/yp-docs/html/ref-manual/ref-
manual.html

I see you got rid of some of the more obvius "funny colors" - the Notes look much better, for example.
I cloned the new sphinx branch and started looking at the implementation some. To make these changes, are you updating the repoDir/documentation/sphinx-static/theme_overrides.css file, or is there somewhere else you're making the changes?

Thanks,
-Mark M

Quoted 28 lines

For now we generate a single large documentation 'website' that includes all existing docs, instead of publishing each document separately. I think it's an important change compared to how our documentation was published so far. I personally like that quite a bit, and think it's 'better' this way, but I would really want to hear from you and that we agree this is what we want for the project. We still need to look at how we want to generate PDF/offline documentation though. And we need to decide how to manage the bitbake manual which sits in the bitbake git tree, so right now it's published separately.

Side note: the VSCode extension for reST is quite good
(https://docs.restructuredtext.net/index.html)

Quoted 15 lines

TW: i do lots of builds, never seen issues, why are there so many failures on

    AB?

RP: do you do oe-selftest or testsdk builds?
TW: no
RP: do you do mips builds?
TW: no
RP: so these are the areas where we see these failures, “regular” builds
are okay
SS: i don’t see issues on my builds but that’s probably because of working
these issues out on the AB
RP: infrastructure issues are low, it’s mostly race issues that come out of
the extreme load of the AB or intermittent architecture issues that are rarely tested

#3 lore raw

Le ven. 31 juil. 2020 à 22:06, Morton, Mark <[email protected]> a écrit :

Quoted 128 lines
Quoted 4 lines

-----Original Message-----
From: [email protected] <[email protected]> On

Behalf

Quoted 6 lines

Of Nicolas Dechesne
Sent: Wednesday, July 29, 2020 12:37 AM
To: Trevor Woerner <[email protected]>
Cc: [email protected]; Morton, Mark <[email protected]>
Subject: Re: [docs] [yocto] Yocto Technical Team Minutes, Engineering

Sync,

Quoted 31 lines

for July 28 2020

Quoted 4 lines · Trevor Woerner

Yocto Technical Team Minutes, Engineering Sync, for July 28 2020 archive:

https://docs.google.com/document/d/1ly8nyhO14kDNnFcW2QskANXW3ZT7
QwKC5w

Quoted 21 lines

WVDg9dDH4/edit

== disclaimer ==
Best efforts are made to ensure the below is accurate and valid. However, errors sometimes happen. If any errors or omissions are found, please feel free to reply to this email with any corrections.

== attendees ==
Trevor Woerner, Stephen Jolly, Armin Kuster, Josef Holzmayr, Richard Purdie, Trevor Gamblin, Joshua Watt, Mark Morton, Ross Burton, Bruce Ashfield, Steve Sakoman, (phone-in ??), Jon Mason, Randy MacLeod, Scott Murray, Denys Dmytriyenko

== notes ==
- fixed a race on AB wrt perl
- 3.1.2 into QA
- 3.2-m2 to QA once they’re done with 3.1.2

== general ==
RP: load of perl issues over the weekend, believe to have found issue
RP: lots of qemumips issues, not sure why, perhaps a timeouts issue?

(no

Quoted 5 lines

ping

Quoted 4 lines

within 30 seconds)
RP: some infrastructure issues (one worker has become corrupted)

RP: curious to know what people think about 3.1.2: there is a bitbake

issue

Quoted 1 line
Quoted 1 line

with toaster, should we release then fix or wait for fix before

release?

Quoted 7 lines
Quoted 5 lines

SS: maybe we should get feedback from people using toaster
RP: the fix is simple, maybe release 3.1.2 (with release note) then fix

RP: merged re-arrangement of package manager code, this puts more of

our code

Quoted 1 line

into python libraries instead of bbclass files, which is the

direction i

Quoted 4 lines
Quoted 4 lines

want to take going forward
RP: i need to start a conversation on oe-architecture for future directions
RP: e.g. binary package feeds, we want to capture these use-cases into

the

Quoted 5 lines
Quoted 5 lines

wiki

[NB: done! see https://wiki.yoctoproject.org/wiki/Future_Directions]

RP: we also posted the current agreement on “inclusive language” as

agreed

Quoted 6 lines
Quoted 6 lines

on by various groups (oe board, oe-tsc, yp-tsc)

TW: what’s the thinking behind moving code to libraries vs bbclass?
RP: pro: parsing speed con: variable dependency tracking is lost
RP: lots of python code gets written to logs etc, which slows it down
JPEW: does bitbake automatically detect variable dependencies in

library

Quoted 3 lines

code?

Quoted 2 lines

RP: no. maybe we just need to do the scan once?
TW: do we lose flexibility? (e.g. it’s easy to copy+paste+modify a

bbclass,

Quoted 6 lines
Quoted 6 lines

will that be possible?)
RP: there will still be all the same entry points, so it should be possible
RP: i’m worried about the size of the datastore in memory and it is a
bottleneck

Quoted 7 lines
Quoted 7 lines

RP: i wonder why qemumips is so slow?
Randy: i think Cisco are the only ones caring about mips
RP: yes, and Comcast too i think

MM: have been wrapped up in $WORK stuff, but will be working on docs
conversion soon
RP: MH has been putting infrastructure in place and Nico is looking

forward

Quoted 3 lines

to

Quoted 2 lines

getting this going
MM: do we want to try to reproduce our current colours/formatting or

try

Quoted 2 lines
Quoted 2 lines

something new
RP: i like what we have, the project does have an existing scheme, but

we

Quoted 9 lines

can

Quoted 1 line

change if something else makes sense

I have some updates on Sphinx..
I recall that the work in progress can be found here:
https://git.yoctoproject.org/cgit/cgit.cgi/yocto-docs/log/?h=sphinx

The automated conversion from DocBook to Sphinx is managing 'internal' links quite badly, so it results in a significant amount of work to

fixup links. Also

Quoted 3 lines

since I wanted to use the Sphinx builtin 'glossary'
for terms and variables in the reference manual, that also required me to update how links work for terms and variables. That's why it's taking

more

Quoted 1 line

time than I would like. I manage to use some automation/regexp, and do a

bit

Quoted 3 lines

of manual fix up as well.

I also started to experiment with customization in the CSS theme, to

change

Quoted 4 lines

some color/formatting. It's not done yet, but it has got better:
https://people.linaro.org/~nicolas.dechesne/yp-docs/html/ref-manual/ref-
manual.html

Quoted 6 lines

I see you got rid of some of the more obvius "funny colors" - the Notes look much better, for example.
I cloned the new sphinx branch and started looking at the implementation some. To make these changes, are you updating the repoDir/documentation/sphinx-static/theme_overrides.css file, or is there somewhere else you're making the changes?

Yes. All CSS customization is done in this file. It overrides the css coming from the base theme template that we use.

Thanks for your help! I definitely need some help!

Quoted 44 lines

Thanks,
-Mark M

Quoted 2 lines

For now we generate a single large documentation 'website' that includes

all

Quoted 1 line

existing docs, instead of publishing each document separately. I think

it's an

Quoted 1 line

important change compared to how our documentation was published so far.

I

Quoted 1 line

personally like that quite a bit, and think it's 'better' this way, but

I would really

Quoted 1 line

want to hear from you and that we agree this is what we want for the

project.

Quoted 1 line

We still need to look at how we want to generate PDF/offline

documentation

Quoted 8 lines

though. And we need to decide how to manage the bitbake manual which sits in the bitbake git tree, so right now it's published separately.

Side note: the VSCode extension for reST is quite good
(https://docs.restructuredtext.net/index.html)

Quoted 2 lines

TW: i do lots of builds, never seen issues, why are there so many

failures on

Quoted 6 lines
Quoted 6 lines
    AB?

RP: do you do oe-selftest or testsdk builds?
TW: no
RP: do you do mips builds?
TW: no
RP: so these are the areas where we see these failures, “regular”

builds

Quoted 2 lines
Quoted 2 lines

are okay
SS: i don’t see issues on my builds but that’s probably because of

working

Quoted 2 lines
Quoted 2 lines

these issues out on the AB
RP: infrastructure issues are low, it’s mostly race issues that come

out of

Quoted 1 line
Quoted 1 line

the extreme load of the AB or intermittent architecture issues

that are

Quoted 2 lines
Quoted 2 lines

rarely tested