Skip to contentSkip to conversation

meta-arm: Remove meta-python dependency

PATCH In review Joshua Watt 7 participants 14 messages May 29, 2020 – Jun 8, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 meta-arm: Remove meta-python dependency

Removes the dependency on meta-arm, now that the required python modules recipes are in oe-core. Ensure that a sufficiently new enough version of oe-core is present for the recipes to be provided.

---
 meta-arm/conf/layer.conf | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
meta-arm/conf/layer.conf +1 -2
index d96e9f1..1c241d4 100644
@@ -11,6 +11,5 @@ BBFILE_PRIORITY_meta-arm = "6"
 
 LAYERDEPENDS_meta-arm = " \
     core \
-    meta-python \
 "
-LAYERSERIES_COMPAT_meta-arm = "warrior zeus dunfell"
+LAYERSERIES_COMPAT_meta-arm = "gatesgarth"
Signature · 1 line
2.17.1
#1 lore raw
Quoted 19 lines · Joshua Watt via lists.yoctoproject.org

---
meta-arm/conf/layer.conf | 3 +--

1 file changed, 1 insertion(+), 2 deletions(-)
meta-arm/conf/layer.conf +0 -1
index d96e9f1..1c241d4 100644
@@ -11,6 +11,5 @@ BBFILE_PRIORITY_meta-arm = "6"
 
 LAYERDEPENDS_meta-arm = " \
    core \
-    meta-python \

"

meta-arm/conf/layer.conf BBFILE_PRIORITY_meta-arm = "6" · line 15

Quoted 2 lines
meta-arm/conf/layer.conf +1 -1
-LAYERSERIES_COMPAT_meta-arm = "warrior zeus dunfell"
+LAYERSERIES_COMPAT_meta-arm = "gatesgarth"

Could you explain why you want to remove dunfell compatibility ?

Bertrand

Quoted 4 lines
Signature · 1 line
2.17.1
#2 lore raw
Quoted 14 lines · Bertrand Marquis
Quoted 14 lines · Joshua Watt via lists.yoctoproject.org

---
meta-arm/conf/layer.conf | 3 +--

1 file changed, 1 insertion(+), 2 deletions(-)
meta-arm/conf/layer.conf +0 -0
index d96e9f1..1c241d4 100644
@@ -11,6 +11,5 @@ BBFILE_PRIORITY_meta-arm = "6"
 
Quoted 7 lines

meta-arm/conf/layer.conf BBFILE_PRIORITY_meta-arm = "6" · line 15

Quoted 6 lines
meta-arm/conf/layer.conf +0 -1
 LAYERDEPENDS_meta-arm = " \
     core \
-    meta-python \

"

meta-arm/conf/layer.conf +1 -1
-LAYERSERIES_COMPAT_meta-arm = "warrior zeus dunfell"
+LAYERSERIES_COMPAT_meta-arm = "gatesgarth"

Could you explain why you want to remove dunfell compatibility ?

dunfell doesn't have the required python recipes in oe-core, so it isn't compatible with it anymore.

Quoted 7 lines

Bertrand

Quoted 4 lines
Signature · 1 line
2.17.1
#3 lore raw
Quoted 28 lines · Joshua Watt
Quoted 14 lines · Bertrand Marquis
Quoted 14 lines · Joshua Watt via lists.yoctoproject.org

---
meta-arm/conf/layer.conf | 3 +--

1 file changed, 1 insertion(+), 2 deletions(-)
meta-arm/conf/layer.conf +0 -0
index d96e9f1..1c241d4 100644
@@ -11,6 +11,5 @@ BBFILE_PRIORITY_meta-arm = "6"
 
Quoted 7 lines

meta-arm/conf/layer.conf BBFILE_PRIORITY_meta-arm = "6" · line 15

Quoted 6 lines
meta-arm/conf/layer.conf +0 -1
 LAYERDEPENDS_meta-arm = " \
    core \
-    meta-python \

"

meta-arm/conf/layer.conf +1 -1
-LAYERSERIES_COMPAT_meta-arm = "warrior zeus dunfell"
+LAYERSERIES_COMPAT_meta-arm = "gatesgarth"

Could you explain why you want to remove dunfell compatibility ?

Quoted 1 line

dunfell doesn't have the required python recipes in oe-core, so it isn't compatible with it anymore.

Nothing has been tested or is compatible with gatesgarth right now so this change will really have a breaking impact.

Bertrand

Quoted 9 lines
Quoted 9 lines

Bertrand

Quoted 4 lines
Signature · 1 line
2.17.1
#4 lore raw
Quoted 29 lines · Bertrand Marquis
Quoted 8 lines · Joshua Watt
Quoted 2 lines · Bertrand Marquis
Quoted 2 lines

On 29 May 2020, at 16:14, Joshua Watt via lists.yoctoproject.org

Quoted 20 lines · <[email protected]> wrote:
Quoted 19 lines
Quoted 16 lines

---
meta-arm/conf/layer.conf | 3 +--

1 file changed, 1 insertion(+), 2 deletions(-)
meta-arm/conf/layer.conf +0 -1
index d96e9f1..1c241d4 100644
@@ -11,6 +11,5 @@ BBFILE_PRIORITY_meta-arm = "6"
 
 LAYERDEPENDS_meta-arm = " \
    core \
-    meta-python \

"

meta-arm/conf/layer.conf BBFILE_PRIORITY_meta-arm = "6" · line 15

Quoted 2 lines
meta-arm/conf/layer.conf +1 -1
-LAYERSERIES_COMPAT_meta-arm = "warrior zeus dunfell"
+LAYERSERIES_COMPAT_meta-arm = "gatesgarth"

Could you explain why you want to remove dunfell compatibility ?

Quoted 6 lines
Quoted 1 line

dunfell doesn't have the required python recipes in oe-core, so it isn't

compatible with it anymore.

Nothing has been tested or is compatible with gatesgarth right now so this change will really have a breaking impact.

Right, you certainly wouldn't want to backport it to the dunfell branch; this is just for master where the next release would be gatesgarth anyway.

Also, with the qemu support I added, I think we have reasonable testing coverage on meta-arm, and in particular enough to have high confidence in this change.

Finally, FWIW, upstream oe-core and meta-python have already made these changes on their master branches, so it seems like we wouldn't want a useless dependency on meta-python in meta-arm, *especially* if the idea is to get other bsp layers to depend on meta-arm to consolidate things like TF-A and optee.

Quoted 13 lines

Bertrand

Quoted 9 lines
Quoted 9 lines

Bertrand

Quoted 4 lines

--
2.17.1

#5 lore raw
Quoted 46 lines · Joshua Watt
Quoted 29 lines · Bertrand Marquis
Quoted 8 lines · Joshua Watt
Quoted 2 lines · Bertrand Marquis
Quoted 2 lines

On 29 May 2020, at 16:14, Joshua Watt via lists.yoctoproject.org

Quoted 20 lines · <[email protected]> wrote:
Quoted 19 lines
Quoted 16 lines

---
meta-arm/conf/layer.conf | 3 +--

1 file changed, 1 insertion(+), 2 deletions(-)
meta-arm/conf/layer.conf +0 -1
index d96e9f1..1c241d4 100644
@@ -11,6 +11,5 @@ BBFILE_PRIORITY_meta-arm = "6"
 
 LAYERDEPENDS_meta-arm = " \
    core \
-    meta-python \

"

meta-arm/conf/layer.conf BBFILE_PRIORITY_meta-arm = "6" · line 15

Quoted 2 lines
meta-arm/conf/layer.conf +1 -1
-LAYERSERIES_COMPAT_meta-arm = "warrior zeus dunfell"
+LAYERSERIES_COMPAT_meta-arm = "gatesgarth"

Could you explain why you want to remove dunfell compatibility ?

Quoted 6 lines
Quoted 1 line

dunfell doesn't have the required python recipes in oe-core, so it isn't

compatible with it anymore.

Nothing has been tested or is compatible with gatesgarth right now so this change will really have a breaking impact.

Right, you certainly wouldn't want to backport it to the dunfell branch; this is just for master where the next release would be gatesgarth anyway.

Also, with the qemu support I added, I think we have reasonable testing coverage on meta-arm, and in particular enough to have high confidence in this change.

Quoted 5 lines

Finally, FWIW, upstream oe-core and meta-python have already made these changes on their master branches, so it seems like we wouldn't want a useless dependency on meta-python in meta-arm, *especially* if the idea is to get other bsp layers to depend on meta-arm to consolidate things like TF-A and optee.

We currently have a requirement for dunfell compatability on the meta-arm master branch for Arm's BSP software releases (which will be YP based going forward). To make a stable release for our customers, we need to draw a line between stablity of our dependent layers and the latest code in the meta-arm layer. This especially relates to the BSPs Arm will be adding to meta-arm-bsp in this release.

We do not have to keep dunfell support in master after the release of gatesgarth, but we will need it until then. So, please stash this patch until October (which will be here sooner than you think).

Thanks,
Jon

Quoted 17 lines
Quoted 15 lines

Bertrand

Quoted 9 lines
Quoted 9 lines

Bertrand

Quoted 4 lines

--
2.17.1

#6 lore raw
Quoted 55 lines · Jon Mason
Quoted 41 lines · Joshua Watt
Quoted 25 lines · Bertrand Marquis
Quoted 6 lines · Joshua Watt
Quoted 1 line · Bertrand Marquis
Quoted 1 line

On 29 May 2020, at 16:14, Joshua Watt via lists.yoctoproject.org

Quoted 18 lines · <[email protected]> wrote:
Quoted 18 lines
Quoted 15 lines

---
meta-arm/conf/layer.conf | 3 +--

1 file changed, 1 insertion(+), 2 deletions(-)
meta-arm/conf/layer.conf +0 -1
index d96e9f1..1c241d4 100644
@@ -11,6 +11,5 @@ BBFILE_PRIORITY_meta-arm = "6"
 
 LAYERDEPENDS_meta-arm = " \
     core \
-    meta-python \

"

meta-arm/conf/layer.conf BBFILE_PRIORITY_meta-arm = "6" · line 15

Quoted 2 lines
meta-arm/conf/layer.conf +1 -1
-LAYERSERIES_COMPAT_meta-arm = "warrior zeus dunfell"
+LAYERSERIES_COMPAT_meta-arm = "gatesgarth"

Could you explain why you want to remove dunfell compatibility ?

Quoted 6 lines
Quoted 1 line

dunfell doesn't have the required python recipes in oe-core, so it isn't

compatible with it anymore.

Nothing has been tested or is compatible with gatesgarth right now so this change will really have a breaking impact.

Right, you certainly wouldn't want to backport it to the dunfell branch; this is just for master where the next release would be gatesgarth anyway.

Also, with the qemu support I added, I think we have reasonable testing coverage on meta-arm, and in particular enough to have high confidence in this change.

Quoted 5 lines

Finally, FWIW, upstream oe-core and meta-python have already made these changes on their master branches, so it seems like we wouldn't want a useless dependency on meta-python in meta-arm, *especially* if the idea is to get other bsp layers to depend on meta-arm to consolidate things like TF-A and optee.

We currently have a requirement for dunfell compatability on the meta-arm master branch for Arm's BSP software releases (which will be YP based going forward). To make a stable release for our customers, we need to draw a line between stablity of our dependent layers and the latest code in the meta-arm layer. This especially relates to the BSPs Arm will be adding to meta-arm-bsp in this release.

Quoted 3 lines

We do not have to keep dunfell support in master after the release of gatesgarth, but we will need it until then. So, please stash this patch until October (which will be here sooner than you think).

Hmm, that's really unfortunate. That makes me think this layer is not actually intended to be a centralized location for ARM related things, and I'm probably not going to try and consolidate things from other layers here anymore. Not having a master branch that can be used with the master branch of OE-core really eliminates the possibility for me to use a BSP layer like e.g. meta-rockchip for upstream development and testing if that BSP layer depends on meta-arm and meta-arm is perpetually stuck on the last release.

Quoted 17 lines

Thanks,
Jon

Quoted 12 lines
Quoted 11 lines

Bertrand

Quoted 7 lines
Quoted 7 lines

Bertrand

Quoted 4 lines

--
2.17.1

#7 lore raw
Quoted 61 lines · Joshua Watt
Quoted 55 lines · Jon Mason
Quoted 41 lines · Joshua Watt
Quoted 25 lines · Bertrand Marquis
Quoted 6 lines · Joshua Watt
Quoted 1 line · Bertrand Marquis
Quoted 1 line

On 29 May 2020, at 16:14, Joshua Watt via lists.yoctoproject.org

Quoted 18 lines · <[email protected]> wrote:
Quoted 18 lines
Quoted 15 lines

---
meta-arm/conf/layer.conf | 3 +--

1 file changed, 1 insertion(+), 2 deletions(-)
meta-arm/conf/layer.conf +0 -1
index d96e9f1..1c241d4 100644
@@ -11,6 +11,5 @@ BBFILE_PRIORITY_meta-arm = "6"
 
 LAYERDEPENDS_meta-arm = " \
     core \
-    meta-python \

"

meta-arm/conf/layer.conf

Quoted 2 lines
meta-arm/conf/layer.conf +1 -1
-LAYERSERIES_COMPAT_meta-arm = "warrior zeus dunfell"
+LAYERSERIES_COMPAT_meta-arm = "gatesgarth"

Could you explain why you want to remove dunfell compatibility ?

Quoted 6 lines
Quoted 1 line

dunfell doesn't have the required python recipes in oe-core, so it isn't

compatible with it anymore.

Nothing has been tested or is compatible with gatesgarth right now so this change will really have a breaking impact.

Right, you certainly wouldn't want to backport it to the dunfell branch; this is just for master where the next release would be gatesgarth anyway.

Also, with the qemu support I added, I think we have reasonable testing coverage on meta-arm, and in particular enough to have high confidence in this change.

Quoted 5 lines

Finally, FWIW, upstream oe-core and meta-python have already made these changes on their master branches, so it seems like we wouldn't want a useless dependency on meta-python in meta-arm, *especially* if the idea is to get other bsp layers to depend on meta-arm to consolidate things like TF-A and optee.

We currently have a requirement for dunfell compatability on the meta-arm master branch for Arm's BSP software releases (which will be YP based going forward). To make a stable release for our customers, we need to draw a line between stablity of our dependent layers and the latest code in the meta-arm layer. This especially relates to the BSPs Arm will be adding to meta-arm-bsp in this release.

Quoted 3 lines

We do not have to keep dunfell support in master after the release of gatesgarth, but we will need it until then. So, please stash this patch until October (which will be here sooner than you think).

Quoted 7 lines

Hmm, that's really unfortunate. That makes me think this layer is not actually intended to be a centralized location for ARM related things, and I'm probably not going to try and consolidate things from other layers here anymore. Not having a master branch that can be used with the master branch of OE-core really eliminates the possibility for me to use a BSP layer like e.g. meta-rockchip for upstream development and testing if that BSP layer depends on meta-arm and meta-arm is perpetually stuck on the last release.

I'm in no way saying that meta-arm master should not work with upstream master. I'm saying that we want to keep compatability with the previous release so that we can cut stable releases for customers.

We have a requirement to release BSP SW based on YP. Just like I would expect other vendors/people who want to use meta-arm. So, there are a few ways we can approach this issue.

1.  Merge patches onto dunfell branch
2.  A development branch based on dunfell
3.  Use master meta-arm-bsp and dunfell branches for meta-arm and

meta-arm-toolchain
4. Keep master compatible with dunfell

One is simple, we actually do our development on the dunfell branch. However, this breaks it from being a stable branch. So, this is a non-starter

Two is still simple, we do all of our internal development on a second dunfell branch ("dunfell-dev") and make any releases we have off of that. The problem with this is the integration and testing. If there are multiple branches to test, can we expect developers to test every combination? Also, there might be a need for different versions of the patches depending on how much master and dunfell-dev diverge.

Three is more complex. Since only meta-arm-bsp needs releases, we can simply let that one float and fix the others on the stable branches. The problem with this is that we are currently needing to add recipes for things into meta-arm that others might need (like SCP, EDK2, etc). So, this will work once we have more stability in meta-arm, but right now we are adding too much stuff to make it viable for the next release.

Four is simple, we keep compatability with dunfell and make the releases off of that. The potential hazard with this is if something in YP master requires that compatability to be broken.

I presonally would like #4, because it is simple. However, if this will prevent others from using meta-arm, then let's have a discussion on it. Perhaps I am not understanding something fundamental.

Thanks,
Jon

Quoted 19 lines
Quoted 19 lines

Thanks,
Jon

Quoted 12 lines
Quoted 11 lines

Bertrand

Quoted 7 lines
Quoted 7 lines

Bertrand

Quoted 4 lines

--
2.17.1

#8 lore raw

hey Jon,

Quoted 84 lines · Jon Mason
Quoted 4 lines · Joshua Watt
Quoted 2 lines · Jon Mason
Quoted 1 line · Joshua Watt

On Fri, May 29, 2020, 10:37 AM Bertrand Marquis <

[email protected]>

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines · wrote:
Quoted 2 lines

On 29 May 2020, at 16:32, Joshua Watt <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines · Bertrand Marquis
Quoted 1 line
Quoted 1 line

On 29 May 2020, at 16:14, Joshua Watt via

lists.yoctoproject.org

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines · <[email protected]> wrote:
Quoted 6 lines

Review

Quoted 6 lines

---
meta-arm/conf/layer.conf | 3 +--

1 file changed, 1 insertion(+), 2 deletions(-)
(quoted diff) +0 -0

b/meta-arm/conf/layer.conf

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

index d96e9f1..1c241d4 100644

meta-arm/conf/layer.conf +0 -0
@@ -11,6 +11,5 @@ BBFILE_PRIORITY_meta-arm = "6"
 
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

meta-arm/conf/layer.conf

Quoted 6 lines
meta-arm/conf/layer.conf +0 -1
 LAYERDEPENDS_meta-arm = " \
     core \
-    meta-python \

"

meta-arm/conf/layer.conf +1 -1
-LAYERSERIES_COMPAT_meta-arm = "warrior zeus dunfell"
+LAYERSERIES_COMPAT_meta-arm = "gatesgarth"

Could you explain why you want to remove dunfell compatibility

?

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

dunfell doesn't have the required python recipes in oe-core, so

it isn't

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

compatible with it anymore.

Nothing has been tested or is compatible with gatesgarth right now

so this

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines

change will really have a breaking impact.

Right, you certainly wouldn't want to backport it to the dunfell

branch;

Quoted 1 line
Quoted 1 line
Quoted 1 line

this is just for master where the next release would be gatesgarth

anyway.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Also, with the qemu support I added, I think we have reasonable

testing

Quoted 1 line
Quoted 1 line
Quoted 1 line

coverage on meta-arm, and in particular enough to have high

confidence in

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

this change.

Finally, FWIW, upstream oe-core and meta-python have already made

these

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

changes on their master branches, so it seems like we wouldn't want a useless dependency on meta-python in meta-arm, *especially* if the

idea is

Quoted 1 line
Quoted 1 line
Quoted 1 line

to get other bsp layers to depend on meta-arm to consolidate things

like

Quoted 9 lines
Quoted 9 lines
Quoted 1 line

TF-A and optee.

We currently have a requirement for dunfell compatability on the meta-arm master branch for Arm's BSP software releases (which will be YP based going forward). To make a stable release for our customers, we need to draw a line between stablity of our dependent layers and the latest code in the meta-arm layer. This especially relates to the BSPs Arm will be adding to meta-arm-bsp in this release.

Quoted 6 lines
Quoted 3 lines

We do not have to keep dunfell support in master after the release of gatesgarth, but we will need it until then. So, please stash this patch until October (which will be here sooner than you think).

Hmm, that's really unfortunate. That makes me think this layer is not actually intended to be a centralized location for ARM related things,

and

Quoted 1 line

I'm probably not going to try and consolidate things from other layers

here

Quoted 1 line

anymore. Not having a master branch that can be used with the master

branch

Quoted 1 line

of OE-core really eliminates the possibility for me to use a BSP layer

Quoted 9 lines

like

Quoted 2 lines

e.g. meta-rockchip for upstream development and testing if that BSP layer depends on meta-arm and meta-arm is perpetually stuck on the last

release.

I'm in no way saying that meta-arm master should not work with upstream master. I'm saying that we want to keep compatability with the previous release so that we can cut stable releases for customers.

I have been very supportive about meta-arm as the central location for ARM pieces, and still believe it's the right thing to do. But like Joshua, I find this discussion a bit disappointing. A layer is a "social contract" between the community behind this layer, and the rest of the OE/YP community. Refusing proper and meaningful community patches in master because it doesn't some 'internal' needs doesn't feel right.

Quoted 10 lines

We have a requirement to release BSP SW based on YP. Just like I would expect other vendors/people who want to use meta-arm. So, there are a few ways we can approach this issue.

1.  Merge patches onto dunfell branch
2.  A development branch based on dunfell
3.  Use master meta-arm-bsp and dunfell branches for meta-arm and

meta-arm-toolchain
4. Keep master compatible with dunfell

I think the mistake we made (I am saying we, since I was involved in that discussion) is to have a single tree that includes both the 'community' content that the ARM ecosystem wants to depend on and the specific ARM "products" that are needed by very few specific people (e.g. your BSP product).

I think #3 is the most sensible choice, e.g. you need to split meta-arm into 'community' pieces (I think that would correspond to meta-arm and meta-arm-toolchain layers) and your product pieces. When we build your product with OE, you would reply on the 'community' meta-arm, and if any hack or custom integration is needed, then you can do that in your product tree. You might even overlay things for some time there, it's fine for a product layer. Of course you will need to worry about doing the 'proper' fix into the community meta-arm at some point. But if you don't commit to maintaining a layer for the ARM community, the risk is that everyone will start forking meta-arm recipes, which is not what we want.

Quoted 5 lines

One is simple, we actually do our development on the dunfell branch. However, this breaks it from being a stable branch. So, this is a non-starter

Quoted 7 lines

Two is still simple, we do all of our internal development on a second dunfell branch ("dunfell-dev") and make any releases we have off of that. The problem with this is the integration and testing. If there are multiple branches to test, can we expect developers to test every combination? Also, there might be a need for different versions of the patches depending on how much master and dunfell-dev diverge.

That is an alternate variant of splitting the trees.. doing your 'product' in a product branch, and keep the expected branches (dunfell, master, ...) as community branches. It really depends if meta-arm-bsp is really expected to be a community thing ever, or if it's your product only. In which case it might be more appropriate to split it apart.

Quoted 9 lines

Three is more complex. Since only meta-arm-bsp needs releases, we can simply let that one float and fix the others on the stable branches. The problem with this is that we are currently needing to add recipes for things into meta-arm that others might need (like SCP, EDK2, etc). So, this will work once we have more stability in meta-arm, but right now we are adding too much stuff to make it viable for the next release.

Quoted 9 lines

Four is simple, we keep compatability with dunfell and make the releases off of that. The potential hazard with this is if something in YP master requires that compatability to be broken.

I presonally would like #4, because it is simple. However, if this will prevent others from using meta-arm, then let's have a discussion on it. Perhaps I am not understanding something fundamental.

But the whole discussion started because you said you needed to keep master compatible with dunfell. Even though as you said, it might fail eventually.

Quoted 24 lines

Thanks,
Jon

Quoted 19 lines
Quoted 19 lines

Thanks,
Jon

Quoted 12 lines
Quoted 11 lines

Bertrand

Quoted 7 lines
Quoted 7 lines

Bertrand

Quoted 4 lines

--
2.17.1

#11 lore raw
In reply to Jon Mason
Quoted 39 lines · Jon Mason

I'm in no way saying that meta-arm master should not work with upstream master. I'm saying that we want to keep compatability with the previous release so that we can cut stable releases for customers.

We have a requirement to release BSP SW based on YP. Just like I would expect other vendors/people who want to use meta-arm. So, there
are a few ways we can approach this issue.

1.  Merge patches onto dunfell branch
2.  A development branch based on dunfell
3.  Use master meta-arm-bsp and dunfell branches for meta-arm and

meta-arm-toolchain
4. Keep master compatible with dunfell

One is simple, we actually do our development on the dunfell branch. However, this breaks it from being a stable branch. So, this is a non-starter

Two is still simple, we do all of our internal development on a second
dunfell branch ("dunfell-dev") and make any releases we have off of that. The problem with this is the integration and testing. If there
are multiple branches to test, can we expect developers to test every combination? Also, there might be a need for different versions of the patches depending on how much master and dunfell-dev diverge.

Three is more complex. Since only meta-arm-bsp needs releases, we can
simply let that one float and fix the others on the stable branches. The problem with this is that we are currently needing to add recipes for things into meta-arm that others might need (like SCP, EDK2, etc).
So, this will work once we have more stability in meta-arm, but right now we are adding too much stuff to make it viable for the next release.

Quoted 7 lines

Four is simple, we keep compatability with dunfell and make the releases off of that. The potential hazard with this is if something in YP master requires that compatability to be broken.

I presonally would like #4, because it is simple. However, if this will prevent others from using meta-arm, then let's have a discussion on it. Perhaps I am not understanding something fundamental.

I can see why you want to do this but it isn't how people expect to work with layers so it may need some further thought...

Cheers,

Richard

#13 lore raw
In reply to Joshua Watt
Quoted 57 lines · Joshua Watt
Quoted 55 lines · Jon Mason
Quoted 41 lines · Joshua Watt
Quoted 25 lines · Bertrand Marquis
Quoted 6 lines · Joshua Watt
Quoted 1 line · Bertrand Marquis
Quoted 1 line

On 29 May 2020, at 16:14, Joshua Watt via lists.yoctoproject.org

Quoted 18 lines · <[email protected]> wrote:
Quoted 18 lines
Quoted 15 lines

---
meta-arm/conf/layer.conf | 3 +--

1 file changed, 1 insertion(+), 2 deletions(-)
meta-arm/conf/layer.conf +0 -1
index d96e9f1..1c241d4 100644
@@ -11,6 +11,5 @@ BBFILE_PRIORITY_meta-arm = "6"
 
 LAYERDEPENDS_meta-arm = " \
      core \
-    meta-python \

"

meta-arm/conf/layer.conf

Quoted 2 lines
meta-arm/conf/layer.conf +1 -1
-LAYERSERIES_COMPAT_meta-arm = "warrior zeus dunfell"
+LAYERSERIES_COMPAT_meta-arm = "gatesgarth"

Could you explain why you want to remove dunfell compatibility ?

Quoted 6 lines
Quoted 1 line

dunfell doesn't have the required python recipes in oe-core, so it isn't

compatible with it anymore.

Nothing has been tested or is compatible with gatesgarth right now so this change will really have a breaking impact.

Right, you certainly wouldn't want to backport it to the dunfell branch; this is just for master where the next release would be gatesgarth anyway.

Also, with the qemu support I added, I think we have reasonable testing coverage on meta-arm, and in particular enough to have high confidence in this change.

Quoted 5 lines

Finally, FWIW, upstream oe-core and meta-python have already made these changes on their master branches, so it seems like we wouldn't want a useless dependency on meta-python in meta-arm, *especially* if the idea is to get other bsp layers to depend on meta-arm to consolidate things like TF-A and optee.

We currently have a requirement for dunfell compatability on the meta-arm master branch for Arm's BSP software releases (which will be YP based going forward).  To make a stable release for our customers, we need to draw a line between stablity of our dependent layers and the latest code in the meta-arm layer.  This especially relates to the BSPs Arm will be adding to meta-arm-bsp in this release.

Quoted 5 lines
Quoted 3 lines

We do not have to keep dunfell support in master after the release of gatesgarth, but we will need it until then.  So, please stash this patch until October (which will be here sooner than you think).

Hmm, that's really unfortunate. That makes me think this layer is not actually intended to be a centralized location for ARM related things, and I'm probably not going to try and consolidate things from other layers here anymore. Not having a master branch that can be used with the master branch of OE-core really eliminates the possibility for me to use a BSP layer like e.g. meta-rockchip for upstream development and testing if that BSP layer depends on meta-arm and meta-arm is perpetually stuck on the last release.

Sorry for going a little overboard here, I was a bit upset because one of my main goals had been to remove the meta-python dependency and I overreacted. I do still plan on contributing to meta-arm :)

One thing that might help is to define what the testing criteria for the community focused layers is. As an example: We have pretty good coverage with the recipes in meta-arm on qemu (we could even tie the op-tee tests into ptest). Is testing meta-arm changes on qemu sufficient to get them accepted to master?

Quoted 19 lines
Quoted 19 lines

Thanks,
Jon

Quoted 12 lines
Quoted 11 lines

Bertrand

Quoted 7 lines
Quoted 7 lines

Bertrand

Quoted 4 lines
Signature · 1 line
2.17.1
#12 lore raw
In reply to Jon Mason
Quoted 7 lines · Jon Mason

We have a requirement to release BSP SW based on YP. Just like I would expect other vendors/people who want to use meta-arm. So, there are a few ways we can approach this issue.

1.  Merge patches onto dunfell branch
2.  A development branch based on dunfell
3.  Use master meta-arm-bsp and dunfell branches for meta-arm and
Quoted 6 lines

meta-arm-toolchain
4. Keep master compatible with dunfell

One is simple, we actually do our development on the dunfell branch. However, this breaks it from being a stable branch. So, this is a non-starter

Especially as meta-arm is a new BSP that is rapidly growing, I don't think we should rule out #1.

To expand: meta-arm master is compatible with oe-core master. meta-arm dunfell branch is compatible with dunfell. Because the BSP is in active development, and because we want to support the users as much as possible, new versions of some recipes can be backported from master into dunfell. Probably not every change, and the impact to testing isn't negligible, but in the greater scheme customers being able to track Dunfell but also get the latest software is probably the right balance.

Prior art includes other BSPs such as meta-intel. The master branch is tested against master, but release branches also get selected upgrades such as new kernels or other components (media components, etc).

Testing, now, that's another discussion entirely...

Ross

#9 lore raw
In reply to Nicolas Dechesne
Quoted 45 lines · Nicolas Dechesne

hey Jon,

Quoted 8 lines
Quoted 2 lines

Hmm, that's really unfortunate. That makes me think this layer is not actually intended to be a centralized location for ARM related things,

and

Quoted 1 line

I'm probably not going to try and consolidate things from other layers

here

Quoted 1 line

anymore. Not having a master branch that can be used with the master

branch

Quoted 1 line

of OE-core really eliminates the possibility for me to use a BSP layer

Quoted 9 lines

like

Quoted 2 lines

e.g. meta-rockchip for upstream development and testing if that BSP layer depends on meta-arm and meta-arm is perpetually stuck on the last

release.

I'm in no way saying that meta-arm master should not work with upstream master. I'm saying that we want to keep compatability with the previous release so that we can cut stable releases for customers.

I have been very supportive about meta-arm as the central location for ARM pieces, and still believe it's the right thing to do. But like Joshua, I find this discussion a bit disappointing. A layer is a "social contract" between the community behind this layer, and the rest of the OE/YP community. Refusing proper and meaningful community patches in master because it doesn't some 'internal' needs doesn't feel right.

Quoted 10 lines

We have a requirement to release BSP SW based on YP. Just like I would expect other vendors/people who want to use meta-arm. So, there are a few ways we can approach this issue.

1.  Merge patches onto dunfell branch
2.  A development branch based on dunfell
3.  Use master meta-arm-bsp and dunfell branches for meta-arm and

meta-arm-toolchain
4. Keep master compatible with dunfell

I think the mistake we made (I am saying we, since I was involved in that discussion) is to have a single tree that includes both the 'community' content that the ARM ecosystem wants to depend on and the specific ARM "products" that are needed by very few specific people (e.g. your BSP product).

Quoted 10 lines

I think #3 is the most sensible choice, e.g. you need to split meta-arm into 'community' pieces (I think that would correspond to meta-arm and meta-arm-toolchain layers) and your product pieces. When we build your product with OE, you would reply on the 'community' meta-arm, and if any hack or custom integration is needed, then you can do that in your product tree. You might even overlay things for some time there, it's fine for a product layer. Of course you will need to worry about doing the 'proper' fix into the community meta-arm at some point. But if you don't commit to maintaining a layer for the ARM community, the risk is that everyone will start forking meta-arm recipes, which is not what we want.

After giving it some thought for couple days, I tend to agree with Nico that it may be better off to split out meta-arm-bsp away from the rest of meta-arm repository to allow it to make releases against any YP versions and branches.

Quoted 41 lines
Quoted 5 lines

One is simple, we actually do our development on the dunfell branch. However, this breaks it from being a stable branch. So, this is a non-starter

Quoted 7 lines

Two is still simple, we do all of our internal development on a second dunfell branch ("dunfell-dev") and make any releases we have off of that. The problem with this is the integration and testing. If there are multiple branches to test, can we expect developers to test every combination? Also, there might be a need for different versions of the patches depending on how much master and dunfell-dev diverge.

That is an alternate variant of splitting the trees.. doing your 'product' in a product branch, and keep the expected branches (dunfell, master, ...) as community branches. It really depends if meta-arm-bsp is really expected to be a community thing ever, or if it's your product only. In which case it might be more appropriate to split it apart.

Quoted 9 lines

Three is more complex. Since only meta-arm-bsp needs releases, we can simply let that one float and fix the others on the stable branches. The problem with this is that we are currently needing to add recipes for things into meta-arm that others might need (like SCP, EDK2, etc). So, this will work once we have more stability in meta-arm, but right now we are adding too much stuff to make it viable for the next release.

Quoted 9 lines

Four is simple, we keep compatability with dunfell and make the releases off of that. The potential hazard with this is if something in YP master requires that compatability to be broken.

I presonally would like #4, because it is simple. However, if this will prevent others from using meta-arm, then let's have a discussion on it. Perhaps I am not understanding something fundamental.

But the whole discussion started because you said you needed to keep master compatible with dunfell. Even though as you said, it might fail eventually.

#10 lore raw
Quoted 57 lines · Denys Dmytriyenko
Quoted 45 lines · Nicolas Dechesne

hey Jon,

Quoted 8 lines
Quoted 2 lines

Hmm, that's really unfortunate. That makes me think this layer is not actually intended to be a centralized location for ARM related things,

and

Quoted 1 line

I'm probably not going to try and consolidate things from other layers

here

Quoted 1 line

anymore. Not having a master branch that can be used with the master

branch

Quoted 1 line

of OE-core really eliminates the possibility for me to use a BSP layer

Quoted 9 lines

like

Quoted 2 lines

e.g. meta-rockchip for upstream development and testing if that BSP layer depends on meta-arm and meta-arm is perpetually stuck on the last

release.

I'm in no way saying that meta-arm master should not work with upstream master. I'm saying that we want to keep compatability with the previous release so that we can cut stable releases for customers.

I have been very supportive about meta-arm as the central location for ARM pieces, and still believe it's the right thing to do. But like Joshua, I find this discussion a bit disappointing. A layer is a "social contract" between the community behind this layer, and the rest of the OE/YP community. Refusing proper and meaningful community patches in master because it doesn't some 'internal' needs doesn't feel right.

Quoted 10 lines

We have a requirement to release BSP SW based on YP. Just like I would expect other vendors/people who want to use meta-arm. So, there are a few ways we can approach this issue.

1.  Merge patches onto dunfell branch
2.  A development branch based on dunfell
3.  Use master meta-arm-bsp and dunfell branches for meta-arm and

meta-arm-toolchain
4. Keep master compatible with dunfell

I think the mistake we made (I am saying we, since I was involved in that discussion) is to have a single tree that includes both the 'community' content that the ARM ecosystem wants to depend on and the specific ARM "products" that are needed by very few specific people (e.g. your BSP product).

Quoted 10 lines

I think #3 is the most sensible choice, e.g. you need to split meta-arm into 'community' pieces (I think that would correspond to meta-arm and meta-arm-toolchain layers) and your product pieces. When we build your product with OE, you would reply on the 'community' meta-arm, and if any hack or custom integration is needed, then you can do that in your product tree. You might even overlay things for some time there, it's fine for a product layer. Of course you will need to worry about doing the 'proper' fix into the community meta-arm at some point. But if you don't commit to maintaining a layer for the ARM community, the risk is that everyone will start forking meta-arm recipes, which is not what we want.

Quoted 46 lines

After giving it some thought for couple days, I tend to agree with Nico that it may be better off to split out meta-arm-bsp away from the rest of meta-arm repository to allow it to make releases against any YP versions and branches.

Quoted 41 lines
Quoted 5 lines

One is simple, we actually do our development on the dunfell branch. However, this breaks it from being a stable branch. So, this is a non-starter

Quoted 7 lines

Two is still simple, we do all of our internal development on a second dunfell branch ("dunfell-dev") and make any releases we have off of that. The problem with this is the integration and testing. If there are multiple branches to test, can we expect developers to test every combination? Also, there might be a need for different versions of the patches depending on how much master and dunfell-dev diverge.

That is an alternate variant of splitting the trees.. doing your 'product' in a product branch, and keep the expected branches (dunfell, master, ...) as community branches. It really depends if meta-arm-bsp is really expected to be a community thing ever, or if it's your product only. In which case it might be more appropriate to split it apart.

Quoted 9 lines

Three is more complex. Since only meta-arm-bsp needs releases, we can simply let that one float and fix the others on the stable branches. The problem with this is that we are currently needing to add recipes for things into meta-arm that others might need (like SCP, EDK2, etc). So, this will work once we have more stability in meta-arm, but right now we are adding too much stuff to make it viable for the next release.

Quoted 9 lines

Four is simple, we keep compatability with dunfell and make the releases off of that. The potential hazard with this is if something in YP master requires that compatability to be broken.

I presonally would like #4, because it is simple. However, if this will prevent others from using meta-arm, then let's have a discussion on it. Perhaps I am not understanding something fundamental.

But the whole discussion started because you said you needed to keep master compatible with dunfell. Even though as you said, it might fail eventually.

I would appreciate it if everyone could take a look at the email I sent out on Friday, "Proposal for the branching policy of meta-arm".
https://lists.yoctoproject.org/g/meta-arm/message/557

Creating and maintaining an active community around meta-arm is our #1 priority. We had extensive discussions internally, and came up with this proposal. I believe it addresses the issues everyone had, and should prevent this kind of thing from happening in the future. This is a proposal and not a dictate. So, please post comments and questions to that thread.

Thanks,
Jon