How do I change the "Architecture" of a package?

What if the architecture of a package was accidentally left at its default, but it should have been "all" for example?

Just putting "inherit allarch" or simply PACKAGE_ARCH="all" into the recipe is not enough. You get stuck with a "more specific" older version, so that no device wants to upgrade to the newer version that's "all" architecture compatible.

Only way to fix this was to manually remove the old version from the feeds and also manually remove it from the device, and then reinstall it. This works for me personally, but what if I want to apply this to about a million devices that are already in the field?

What can I do to get them to upgrade from the (in this case) "mipsel32" to "all" package?

(and I have a few dozen of these mis-architectured packages...)

Signature · 2 lines

Mike Looijmans
-- 

Quoted 8 lines · Mike Looijmans

What if the architecture of a package was accidentally left at its default, but it should have been "all" for example?

Just putting "inherit allarch" or simply PACKAGE_ARCH="all" into the recipe is not enough. You get stuck with a "more specific" older version, so that no device wants to upgrade to the newer version that's "all" architecture compatible.

What package manager are you using on the device? If you're using opkg it should prioritise by version not arch unless the command line option '--prefer-arch-to-version' is passed. If you're using opkg and it's not doing that, let me know and I'll look into it when I get chance to.

Quoted 7 lines

Only way to fix this was to manually remove the old version from the feeds and also manually remove it from the device, and then reinstall it. This works for me personally, but what if I want to apply this to about a million devices that are already in the field?

What can I do to get them to upgrade from the (in this case) "mipsel32" to "all" package?

As an absolute hack, you could rename the package and set it to replace/conflict/provide the old package name. I think there should be better solutions though.

Quoted 10 lines · Paul Barker

Quoted 8 lines · Mike Looijmans

What if the architecture of a package was accidentally left at its default, but it should have been "all" for example?

Just putting "inherit allarch" or simply PACKAGE_ARCH="all" into the recipe is not enough. You get stuck with a "more specific" older version, so that no device wants to upgrade to the newer version that's "all" architecture compatible.

Quoted 6 lines

What package manager are you using on the device? If you're using opkg it should
prioritise by version not arch unless the command line option '--prefer-arch-to-version' is passed. If you're using opkg and it's not doing
that, let me know and I'll look into it when I get chance to.

It's opkg.

But on closer inspection I noticed that the "git" version is also mysteriously reset to 0, so that the package also gets a lower version number instead of a higher one. Seems to be the PR server borking things again or so, I'll have to investigate that next year...

Quoted 18 lines · Mike Looijmans

Quoted 10 lines · Paul Barker

Quoted 8 lines · Mike Looijmans

What if the architecture of a package was accidentally left at its default, but it should have been "all" for example?

Just putting "inherit allarch" or simply PACKAGE_ARCH="all" into the recipe is not enough. You get stuck with a "more specific" older version, so that no device wants to upgrade to the newer version that's "all" architecture compatible.

Quoted 6 lines

What package manager are you using on the device? If you're using opkg it should
prioritise by version not arch unless the command line option '--prefer-arch-to-version' is passed. If you're using opkg and it's not doing
that, let me know and I'll look into it when I get chance to.

Quoted 6 lines

It's opkg.

But on closer inspection I noticed that the "git" version is also mysteriously reset to 0, so that the package also gets a lower version number instead of a higher one. Seems to be the PR server borking things again or so, I'll have to investigate that next year...

Weird, something in OE killed "gitpkgv".

in the recipe, I have this:

inherit gitpkgv
PV = "2.0+git${SRCPV}"
PKGV = "2.0+git${GITPKGV}"

$ bitbake enigma2-plugin-extensions-autobackup -e | grep PKGV

delivers correct information:

PKGV="2.0+git68+2e7a1db"
GITPKGVTAG="0.0-68-g2e7a1db"
GITPKGV="68+2e7a1db"

But after building and deploying the package, the version number will eventually end up being this one:

2.0+git5+2e7a1db509-r0.2

What in OE is replacing a perfectly good PKGV tag with something completely different bearing no relation whatsoever? Even the number of digits in the git tag differs from the one I put in the recipe!

Even if I put some random text into PKGV, it gets replaced.

Quoted 50 lines · Mike Looijmans

Quoted 18 lines · Mike Looijmans

Quoted 10 lines · Paul Barker

Quoted 8 lines · Mike Looijmans

What if the architecture of a package was accidentally left at its default, but it should have been "all" for example?

Just putting "inherit allarch" or simply PACKAGE_ARCH="all" into the recipe is not enough. You get stuck with a "more specific" older version, so that no device wants to upgrade to the newer version that's "all" architecture compatible.

Quoted 6 lines

What package manager are you using on the device? If you're using opkg it should
prioritise by version not arch unless the command line option '--prefer-arch-to-version' is passed. If you're using opkg and it's not doing
that, let me know and I'll look into it when I get chance to.

Quoted 6 lines

It's opkg.

But on closer inspection I noticed that the "git" version is also mysteriously reset to 0, so that the package also gets a lower version number instead of a higher one. Seems to be the PR server borking things again or so, I'll have to investigate that next year...

Weird, something in OE killed "gitpkgv".

in the recipe, I have this:

inherit gitpkgv
PV = "2.0+git${SRCPV}"
PKGV = "2.0+git${GITPKGV}"

$ bitbake enigma2-plugin-extensions-autobackup -e | grep PKGV

delivers correct information:

PKGV="2.0+git68+2e7a1db"
GITPKGVTAG="0.0-68-g2e7a1db"
GITPKGV="68+2e7a1db"

But after building and deploying the package, the version number will eventually end up being this one:

2.0+git5+2e7a1db509-r0.2

Quoted 5 lines

What in OE is replacing a perfectly good PKGV tag with something completely different bearing no relation whatsoever? Even the number of digits in the git tag differs from the one I put in the recipe!

Even if I put some random text into PKGV, it gets replaced.

Did something come from sstate?

Quoted 52 lines · Richard Purdie

Quoted 51 lines · Mike Looijmans

Quoted 18 lines · Mike Looijmans

Quoted 10 lines · Paul Barker

Quoted 8 lines · Mike Looijmans

What if the architecture of a package was accidentally left at its default, but it should have been "all" for example?

Just putting "inherit allarch" or simply PACKAGE_ARCH="all" into the recipe is not enough. You get stuck with a "more specific" older version, so that no device wants to upgrade to the newer version that's "all" architecture compatible.

Quoted 6 lines

What package manager are you using on the device? If you're using opkg it should
prioritise by version not arch unless the command line option '--prefer-arch-to-version' is passed. If you're using opkg and it's not doing
that, let me know and I'll look into it when I get chance to.

Quoted 7 lines

It's opkg.

But on closer inspection I noticed that the "git" version is also mysteriously reset to 0, so that the package also gets a lower version number instead of a higher one. Seems to be the PR server borking things
again or so, I'll have to investigate that next year...

Weird, something in OE killed "gitpkgv".

in the recipe, I have this:

inherit gitpkgv
PV = "2.0+git${SRCPV}"
PKGV = "2.0+git${GITPKGV}"

$ bitbake enigma2-plugin-extensions-autobackup -e | grep PKGV

delivers correct information:

PKGV="2.0+git68+2e7a1db"
GITPKGVTAG="0.0-68-g2e7a1db"
GITPKGV="68+2e7a1db"

But after building and deploying the package, the version number will eventually end up being this one:

2.0+git5+2e7a1db509-r0.2

Quoted 7 lines

Quoted 5 lines

What in OE is replacing a perfectly good PKGV tag with something completely different bearing no relation whatsoever? Even the number of digits in the git tag differs from the one I put in the recipe!

Even if I put some random text into PKGV, it gets replaced.

Did something come from sstate?

Nope, I do a "bitbake -c cleansstate [package]" first.

i've been experimenting for a while now. It appears that whatever I put into PKGV gets ignored and replaced with what was in PV.

Here's a recipe where I stubbed the actual building and install phases to demonstrate the issue:

(cut here)

DESCRIPTION = "Version numbers are borked"

inherit gitpkgv
PV = "z-pv+${SRCPV}"
PKGV = "z-pkgv+${GITPKGV}"

# Completely random public GIT repo
SRCREV = "${AUTOREV}"
MODULE = "AutoBackup"
SRC_URI = "git://github.com/E2OpenPlugins/e2openplugin-${MODULE}.git"

LICENSE = "GPLv2"
LIC_FILES_CHKSUM = "file://../LICENSE.GPLv2;md5=eb723b61539feef013de476e68b5c50a" SRC_URI_append = " file://LICENSE.GPLv2"

S="${WORKDIR}/git"

do_compile() {
     touch ${B}/helloworld
}

do_install() {
     install -d ${D}${sysconfdir}
     install -m 644 ${B}/helloworld ${D}${sysconfdir}/
}

(EOF)

If I build this package, it ends up getting version "z-pv+0+2e7a1db509-r0.1" instead of "z-pkgv+68+2e7a1db509-r0.1" which I'd expect.

Quoted 10 lines · Paul Barker

Quoted 8 lines · Mike Looijmans

What if the architecture of a package was accidentally left at its default, but it should have been "all" for example?

Just putting "inherit allarch" or simply PACKAGE_ARCH="all" into the recipe is not enough. You get stuck with a "more specific" older version, so that no device wants to upgrade to the newer version that's "all" architecture compatible.

Quoted 6 lines

What package manager are you using on the device? If you're using opkg it should
prioritise by version not arch unless the command line option '--prefer-arch-to-version' is passed. If you're using opkg and it's not doing
that, let me know and I'll look into it when I get chance to.

It's opkg.

But on closer inspection I noticed that the "git" version is also mysteriously reset to 0, so that the package also gets a lower version number instead of a higher one. Seems to be the PR server borking things again or so, I'll have to investigate that next year...

Quoted 50 lines · Mike Looijmans

Quoted 18 lines · Mike Looijmans

Quoted 10 lines · Paul Barker

Quoted 8 lines · Mike Looijmans

What if the architecture of a package was accidentally left at its default, but it should have been "all" for example?

Just putting "inherit allarch" or simply PACKAGE_ARCH="all" into the recipe is not enough. You get stuck with a "more specific" older version, so that no device wants to upgrade to the newer version that's "all" architecture compatible.

Quoted 6 lines

What package manager are you using on the device? If you're using opkg it should
prioritise by version not arch unless the command line option '--prefer-arch-to-version' is passed. If you're using opkg and it's not doing
that, let me know and I'll look into it when I get chance to.

Quoted 6 lines

It's opkg.

But on closer inspection I noticed that the "git" version is also mysteriously reset to 0, so that the package also gets a lower version number instead of a higher one. Seems to be the PR server borking things again or so, I'll have to investigate that next year...

Weird, something in OE killed "gitpkgv".

in the recipe, I have this:

inherit gitpkgv
PV = "2.0+git${SRCPV}"
PKGV = "2.0+git${GITPKGV}"

$ bitbake enigma2-plugin-extensions-autobackup -e | grep PKGV

delivers correct information:

PKGV="2.0+git68+2e7a1db"
GITPKGVTAG="0.0-68-g2e7a1db"
GITPKGV="68+2e7a1db"

But after building and deploying the package, the version number will eventually end up being this one:

2.0+git5+2e7a1db509-r0.2

Quoted 5 lines

What in OE is replacing a perfectly good PKGV tag with something completely different bearing no relation whatsoever? Even the number of digits in the git tag differs from the one I put in the recipe!

Even if I put some random text into PKGV, it gets replaced.

Did something come from sstate?

Quoted 51 lines · Richard Purdie

Quoted 50 lines · Mike Looijmans

Quoted 18 lines · Mike Looijmans

Quoted 10 lines · Paul Barker

Quoted 8 lines · Mike Looijmans

What if the architecture of a package was accidentally left at its default, but it should have been "all" for example?

Just putting "inherit allarch" or simply PACKAGE_ARCH="all" into the recipe is not enough. You get stuck with a "more specific" older version, so that no device wants to upgrade to the newer version that's "all" architecture compatible.

Quoted 6 lines

What package manager are you using on the device? If you're using opkg it should
prioritise by version not arch unless the command line option '--prefer-arch-to-version' is passed. If you're using opkg and it's not doing
that, let me know and I'll look into it when I get chance to.

Quoted 6 lines

It's opkg.

But on closer inspection I noticed that the "git" version is also mysteriously reset to 0, so that the package also gets a lower version number instead of a higher one. Seems to be the PR server borking things again or so, I'll have to investigate that next year...

Weird, something in OE killed "gitpkgv".

in the recipe, I have this:

inherit gitpkgv
PV = "2.0+git${SRCPV}"
PKGV = "2.0+git${GITPKGV}"

$ bitbake enigma2-plugin-extensions-autobackup -e | grep PKGV

delivers correct information:

PKGV="2.0+git68+2e7a1db"
GITPKGVTAG="0.0-68-g2e7a1db"
GITPKGV="68+2e7a1db"

But after building and deploying the package, the version number will eventually end up being this one:

2.0+git5+2e7a1db509-r0.2

Quoted 7 lines

Quoted 5 lines

What in OE is replacing a perfectly good PKGV tag with something completely different bearing no relation whatsoever? Even the number of digits in the git tag differs from the one I put in the recipe!

Even if I put some random text into PKGV, it gets replaced.

Did something come from sstate?

I got todays master from openembedded-core and meta-openembedded, and the recipe above (with a fix for the license). I leave everything as the script "oe-init-build-env" does, I only add meta-openembedded/meta-oe to the bblayers.conf list to get access to the gitpkgv class.

I named the recipe above "weirdversion.bb", and when I build it "as is", I get the correct version number "z-pkgv+68+2e7a1db-r0" (with "pkgv" and "68" in it).

However, when I add the following line to local.conf to activate the PR-SERVER, things go wrong:

PRSERV_HOST = "localhost:0"

After this change, the package suddenly gets "z-pv+0+2e7a1db509-r0.0" as version string, so apparently the PR server kills the PKGV variable and replaces it with something derived from PV alone.

Since this is a completely clean situation with nothing but core code, it must be a bug in the PR server (it seems to act as PV server instead...).

Quoted 72 lines · Mike Looijmans

Quoted 51 lines · Richard Purdie

Quoted 50 lines · Mike Looijmans

Quoted 18 lines · Mike Looijmans

Quoted 10 lines · Paul Barker

Quoted 8 lines · Mike Looijmans

What if the architecture of a package was accidentally left at its default, but it should have been "all" for example?

Just putting "inherit allarch" or simply PACKAGE_ARCH="all" into the recipe is not enough. You get stuck with a "more specific" older version, so that no device wants to upgrade to the newer version that's "all" architecture compatible.

Quoted 6 lines

What package manager are you using on the device? If you're using opkg it should
prioritise by version not arch unless the command line option '--prefer-arch-to-version' is passed. If you're using opkg and it's not doing
that, let me know and I'll look into it when I get chance to.

Quoted 6 lines

It's opkg.

But on closer inspection I noticed that the "git" version is also mysteriously reset to 0, so that the package also gets a lower version number instead of a higher one. Seems to be the PR server borking things again or so, I'll have to investigate that next year...

Weird, something in OE killed "gitpkgv".

in the recipe, I have this:

inherit gitpkgv
PV = "2.0+git${SRCPV}"
PKGV = "2.0+git${GITPKGV}"

$ bitbake enigma2-plugin-extensions-autobackup -e | grep PKGV

delivers correct information:

PKGV="2.0+git68+2e7a1db"
GITPKGVTAG="0.0-68-g2e7a1db"
GITPKGV="68+2e7a1db"

But after building and deploying the package, the version number will eventually end up being this one:

2.0+git5+2e7a1db509-r0.2

Quoted 7 lines

Quoted 5 lines

What in OE is replacing a perfectly good PKGV tag with something completely different bearing no relation whatsoever? Even the number of digits in the git tag differs from the one I put in the recipe!

Even if I put some random text into PKGV, it gets replaced.

Did something come from sstate?

I got todays master from openembedded-core and meta-openembedded, and the recipe above (with a fix for the license). I leave everything as the script "oe-init-build-env" does, I only add meta-openembedded/meta-oe to the bblayers.conf list to get access to the gitpkgv class.

I named the recipe above "weirdversion.bb", and when I build it "as is", I get the correct version number "z-pkgv+68+2e7a1db-r0" (with "pkgv" and "68" in it).

However, when I add the following line to local.conf to activate the PR-SERVER, things go wrong:

Quoted 8 lines

PRSERV_HOST = "localhost:0"

After this change, the package suddenly gets "z-pv+0+2e7a1db509-r0.0" as version string, so apparently the PR server kills the PKGV variable and replaces it with something derived from PV alone.

Since this is a completely clean situation with nothing but core code, it must be a bug in the PR server (it seems to act as PV server instead...).

Imagine you're not using gitpkgv. You set:

PV = "x.y+${SRCPV}"

Since SRCPV contains a revision hash, you can end up in a situation where the version changes and you cannot upgrade the package since the hash didn't 'increase'.

The PR server therefore combines with the git fetcher to add an incremental number at the start of the SRCPV string and yes, in that scenario, it acts as a PV server. This is actually working as designed.

When you add in gitpkgv, something is obviously going wrong. gitpkgv is in meta-oe since I've refused to add it to core on the several occasions its been requested. I've said this before but I will say is again, it *needs* become part of the standard fetcher API rather than a hacked in afterthought which doesn't integrate well.

Quoted 5 lines · Richard Purdie

On Sun, 2015-01-04 at 16:20 +0100, Mike Looijmans wrote:
Imagine you're not using gitpkgv. You set:

PV = "x.y+${SRCPV}"

Quoted 7 lines

Since SRCPV contains a revision hash, you can end up in a situation where the version changes and you cannot upgrade the package since the hash didn't 'increase'.

The PR server therefore combines with the git fetcher to add an incremental number at the start of the SRCPV string and yes, in that scenario, it acts as a PV server. This is actually working as designed.

Then the design is wrong. If a package chose to override PKGV manually, then the rest of the system should leave that value as is, and not touch it. Apparently the recipe author knows better, so please let him use that wisdom.

Also, if you change the architecture of the package, the PR server will reset the version counter to 0 and break the upgrade path too. That was the problem that caused me to discover this problem, the PR server is making it hard to fix arch errors in recipes.

Quoted 5 lines

When you add in gitpkgv, something is obviously going wrong. gitpkgv is in meta-oe since I've refused to add it to core on the several occasions its been requested. I've said this before but I will say is again, it *needs* become part of the standard fetcher API rather than a hacked in afterthought which doesn't integrate well.

I already volunteered and tried to do that, but got stuck in lack of understanding how the fetcher works, and did not get any help so abandoned it in favor of keep using gitpkgv "as is". I'm still volunteering, but without help I can't do it.

Quoted 14 lines · Mike Looijmans

Quoted 5 lines · Richard Purdie

On Sun, 2015-01-04 at 16:20 +0100, Mike Looijmans wrote:
Imagine you're not using gitpkgv. You set:

PV = "x.y+${SRCPV}"

Quoted 7 lines

Since SRCPV contains a revision hash, you can end up in a situation where the version changes and you cannot upgrade the package since the hash didn't 'increase'.

The PR server therefore combines with the git fetcher to add an incremental number at the start of the SRCPV string and yes, in that scenario, it acts as a PV server. This is actually working as designed.

Quoted 8 lines

Then the design is wrong. If a package chose to override PKGV manually, then the rest of the system should leave that value as is, and not touch it. Apparently the recipe author knows better, so please let him use that wisdom.

Also, if you change the architecture of the package, the PR server will reset the version counter to 0 and break the upgrade path too. That was the problem that caused me to discover this problem, the PR server is making it hard to fix arch errors in recipes.

You realise why it does that though? Imagine multiple MACHINE values being built against a PR server. Each MACHINE will have different package architecture values and different hashes. The 'PR' values should therefore be seen as different. If the system didn't do this, it would increment PR for every MACHINE change.

Ok, so you say, lets make it work against MACHINE. That fails in the case where multiple MACHINES share a package architecture :/. Its not a simple problem to try and deal with :(

I make no claim the system is perfect or free from bugs but these behaviours you're referring to are like this for various reasons. I'm open to changing them but we need to address the underlying problems like changing MACHINE above.

There are several other issues with changing package architecture such as the sstate files in the sysroot. There is an open bug for this and right now, I simply don't know how to solve it properly :(.

Quoted 10 lines

Quoted 5 lines

When you add in gitpkgv, something is obviously going wrong. gitpkgv is in meta-oe since I've refused to add it to core on the several occasions its been requested. I've said this before but I will say is again, it *needs* become part of the standard fetcher API rather than a hacked in afterthought which doesn't integrate well.

I already volunteered and tried to do that, but got stuck in lack of understanding how the fetcher works, and did not get any help so abandoned it in favor of keep using gitpkgv "as is". I'm still volunteering, but without help I can't do it.

I'm struggling since to provide the help you need, I'd nearly have to solve the problem myself. I've helped several different people understand the fetcher in the past, only to have most of them move onto other things :(. Add in the 101 other distractions I have and its frustrating for everyone including me. Can you remind me where you were at (links to the right emails would help)?

Quoted 34 lines · Richard Purdie

Quoted 14 lines · Mike Looijmans

Quoted 5 lines · Richard Purdie

On Sun, 2015-01-04 at 16:20 +0100, Mike Looijmans wrote:
Imagine you're not using gitpkgv. You set:

PV = "x.y+${SRCPV}"

Quoted 7 lines

Since SRCPV contains a revision hash, you can end up in a situation where the version changes and you cannot upgrade the package since the hash didn't 'increase'.

The PR server therefore combines with the git fetcher to add an incremental number at the start of the SRCPV string and yes, in that scenario, it acts as a PV server. This is actually working as designed.

Quoted 8 lines

Then the design is wrong. If a package chose to override PKGV manually, then the rest of the system should leave that value as is, and not touch it. Apparently the recipe author knows better, so please let him use that wisdom.

Also, if you change the architecture of the package, the PR server will reset the version counter to 0 and break the upgrade path too. That was the problem that caused me to discover this problem, the PR server is making it hard to fix arch errors in recipes.

You realise why it does that though? Imagine multiple MACHINE values being built against a PR server. Each MACHINE will have different package architecture values and different hashes. The 'PR' values should therefore be seen as different. If the system didn't do this, it would increment PR for every MACHINE change.

Ok, so you say, lets make it work against MACHINE. That fails in the case where multiple MACHINES share a package architecture :/. Its not a simple problem to try and deal with :(

Quoted 8 lines

I make no claim the system is perfect or free from bugs but these behaviours you're referring to are like this for various reasons. I'm open to changing them but we need to address the underlying problems like changing MACHINE above.

There are several other issues with changing package architecture such as the sstate files in the sysroot. There is an open bug for this and right now, I simply don't know how to solve it properly :(.

Yeah, I stand corrected, this is not something easily fixed.

I'd still like to have a way to override the PR server. Is there a way to make the PR server ignore a package, or at least, have it NOT modify the PKGV variable for a package?
And if not, would you accept a patch to add that functionality to the prserver system?

Quoted 11 lines

Quoted 10 lines

Quoted 5 lines

When you add in gitpkgv, something is obviously going wrong. gitpkgv is in meta-oe since I've refused to add it to core on the several occasions its been requested. I've said this before but I will say is again, it *needs* become part of the standard fetcher API rather than a hacked in afterthought which doesn't integrate well.

I already volunteered and tried to do that, but got stuck in lack of understanding how the fetcher works, and did not get any help so abandoned it in favor of keep using gitpkgv "as is". I'm still volunteering, but without help I can't do it.

Quoted 6 lines

I'm struggling since to provide the help you need, I'd nearly have to solve the problem myself. I've helped several different people understand the fetcher in the past, only to have most of them move onto other things :(. Add in the 101 other distractions I have and its frustrating for everyone including me. Can you remind me where you were at (links to the right emails would help)?

This is basically as far as I got:

http://lists.openembedded.org/pipermail/openembedded-core/2014-October/098110.html

Quoted 44 lines · Mike Looijmans

Quoted 34 lines · Richard Purdie

Quoted 14 lines · Mike Looijmans

Quoted 5 lines · Richard Purdie

On Sun, 2015-01-04 at 16:20 +0100, Mike Looijmans wrote:
Imagine you're not using gitpkgv. You set:

PV = "x.y+${SRCPV}"

Quoted 7 lines

Since SRCPV contains a revision hash, you can end up in a situation where the version changes and you cannot upgrade the package since the hash didn't 'increase'.

The PR server therefore combines with the git fetcher to add an incremental number at the start of the SRCPV string and yes, in that scenario, it acts as a PV server. This is actually working as designed.

Quoted 8 lines

Then the design is wrong. If a package chose to override PKGV manually, then the rest of the system should leave that value as is, and not touch it. Apparently the recipe author knows better, so please let him use that wisdom.

Also, if you change the architecture of the package, the PR server will reset the version counter to 0 and break the upgrade path too. That was the problem that caused me to discover this problem, the PR server is making it hard to fix arch errors in recipes.

You realise why it does that though? Imagine multiple MACHINE values being built against a PR server. Each MACHINE will have different package architecture values and different hashes. The 'PR' values should therefore be seen as different. If the system didn't do this, it would increment PR for every MACHINE change.

Ok, so you say, lets make it work against MACHINE. That fails in the case where multiple MACHINES share a package architecture :/. Its not a simple problem to try and deal with :(

Quoted 8 lines

I make no claim the system is perfect or free from bugs but these behaviours you're referring to are like this for various reasons. I'm open to changing them but we need to address the underlying problems like changing MACHINE above.

There are several other issues with changing package architecture such as the sstate files in the sysroot. There is an open bug for this and right now, I simply don't know how to solve it properly :(.

Quoted 7 lines

Yeah, I stand corrected, this is not something easily fixed.

I'd still like to have a way to override the PR server. Is there a way to make the PR server ignore a package, or at least, have it NOT modify the PKGV variable for a package?
And if not, would you accept a patch to add that functionality to the prserver system?

I've not tested it but would something like:

PRSERV_HOST_pn-XXX = ""

work?

There looks to be code to allow:

PRSERV_HOST_XXX = ""

to work too, looking at package.bbclass. Why, I have no idea :/. We should probably remove that in favour of the standard override.

Since I've just been looking at this, I'll write down an overview of what goes on:

PKGR contains ${PRAUTO}. The PKG{E,V,R} variables are only meant to be accessed at packaging time. The code in package.bbclass, package_get_auto_pr() runs at packaging time and sets PRAUTO with a value from the PR server.

The only exception to this is if the magic string "AUTOINC" is in PV. If it is, AUTOINC is replaced with the value from the PR server. You could ask why it sets PKGV with the value of PV and I honestly don't remember, that could be an oversight.

What this did mean is that we could jettison the autoincrement code from the fetcher and just have one thing doing it.

The PKGV being overwritten by PV does look like a bug and is worth looking into. Its not the PR server at fault though, its what package.bbclass is doing with the data. I would like to look at why it was added that way as I worry there was some underlying reason :/.

Quoted 18 lines

Quoted 11 lines

Quoted 10 lines

Quoted 5 lines

When you add in gitpkgv, something is obviously going wrong. gitpkgv is in meta-oe since I've refused to add it to core on the several occasions its been requested. I've said this before but I will say is again, it *needs* become part of the standard fetcher API rather than a hacked in afterthought which doesn't integrate well.

I already volunteered and tried to do that, but got stuck in lack of understanding how the fetcher works, and did not get any help so abandoned it in favor of keep using gitpkgv "as is". I'm still volunteering, but without help I can't do it.

Quoted 6 lines

I'm struggling since to provide the help you need, I'd nearly have to solve the problem myself. I've helped several different people understand the fetcher in the past, only to have most of them move onto other things :(. Add in the 101 other distractions I have and its frustrating for everyone including me. Can you remind me where you were at (links to the right emails would help)?

Quoted 3 lines

This is basically as far as I got:

http://lists.openembedded.org/pipermail/openembedded-core/2014-October/098110.html

Thanks, I do remember looking at this. I also remember getting horribly confused so I did start doing some things to try and help:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=593f14b2e3d1474d0c21d8d872dc7685163ffad2
(which was merged into master, trying to unravel some horrible function chains)

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=89a20582e4fb3b222ccfeaf068fc5cfc7ae2dbf9
(this one should get merged, its just a comment tweak in bitbake).

I also noticed this in my WIP branch:
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=00deb71e6e9a6b7f8aa5509288b59c6e4526c300
but that is perhaps best ignored, I think I had other ideas here.

So to explain what happens, you write:

PV = "x.y+{SRCPV}"

and

SRCPV = "${@bb.fetch2.get_srcrev(d)}"

and this triggers the code in fetch2/__init__.py. That function returns a string which starts with AUTOINC+ and then a set of revisions.

The problem you've run into is that PV is expanded during parsing and is used all over the place (for good reason) (e.g. in the value of WORKDIR) and fetching may not have occurred yet. That means as you observe you can't run git commands on a repo that may not yet exist (this function only uses remote commands like ls-remote if needed).

So how to solve this? You need to add a function in parallel to get_srcrev(), lets call it get_pretty_srcrev() which basically does the same thing, but calls pretty_revision() instead of sortable_revision(). You can probably parameterise get_srcrev() internally. You'd then add a pretty_revision to the git fetcher. At the metadata level you'd have something like:

PRETTYSRCPV = "${@bb.fetch2.get_pretty_srcrev(d)}"

The recipe would then set PKGV = "x.y.+${PRETTYSRCPV}" to get the gitpkgv behaviour.

All this is very approximate and I'm sure it won't be that simple but hopefully it may let you move forward.

Quoted 68 lines · Richard Purdie

Quoted 55 lines · Mike Looijmans

Quoted 43 lines · Richard Purdie

Quoted 23 lines · Mike Looijmans

Quoted 10 lines · Richard Purdie

On Sun, 2015-01-04 at 16:20 +0100, Mike Looijmans wrote:
Imagine you're not using gitpkgv. You set:

PV = "x.y+${SRCPV}"

Since SRCPV contains a revision hash, you can end up in a situation where the version changes and you cannot upgrade the package since the
hash didn't 'increase'.

Quoted 4 lines

The PR server therefore combines with the git fetcher to add an incremental number at the start of the SRCPV string and yes, in that scenario, it acts as a PV server. This is actually working as designed.

Then the design is wrong. If a package chose to override PKGV manually, then
the rest of the system should leave that value as is, and not touch it.
Apparently the recipe author knows better, so please let him use that wisdom.

Quoted 7 lines

Also, if you change the architecture of the package, the PR server will reset
the version counter to 0 and break the upgrade path too. That was the problem
that caused me to discover this problem, the PR server is making it hard to
fix arch errors in recipes.

You realise why it does that though? Imagine multiple MACHINE values being built against a PR server. Each MACHINE will have different package architecture values and different hashes. The 'PR' values should
therefore be seen as different. If the system didn't do this, it would increment PR for every MACHINE change.

Ok, so you say, lets make it work against MACHINE. That fails in the case where multiple MACHINES share a package architecture :/. Its not a simple problem to try and deal with :(

Quoted 8 lines

I make no claim the system is perfect or free from bugs but these behaviours you're referring to are like this for various reasons. I'm open to changing them but we need to address the underlying problems like changing MACHINE above.

There are several other issues with changing package architecture such as the sstate files in the sysroot. There is an open bug for this and right now, I simply don't know how to solve it properly :(.

Yeah, I stand corrected, this is not something easily fixed.

Quoted 7 lines

I'd still like to have a way to override the PR server. Is there a way to make
the PR server ignore a package, or at least, have it NOT modify the PKGV variable for a package?
And if not, would you accept a patch to add that functionality to the prserver
system?

I've not tested it but would something like:

PRSERV_HOST_pn-XXX = ""

Quoted 8 lines

work?

There looks to be code to allow:

PRSERV_HOST_XXX = ""

to work too, looking at package.bbclass. Why, I have no idea :/. We should probably remove that in favour of the standard override.

That means I could also set PRSERV_HOST inside the recipe, I guess. That might be easier to manage. Anyway, good suggestions, I'll try them. That might be useful in reducing the strain on network and flash now that 300+ packages get upgraded if someone happens to patch libc.

Quoted 13 lines

Since I've just been looking at this, I'll write down an overview of what goes on:

PKGR contains ${PRAUTO}. The PKG{E,V,R} variables are only meant to be accessed at packaging time. The code in package.bbclass, package_get_auto_pr() runs at packaging time and sets PRAUTO with a value from the PR server.

The only exception to this is if the magic string "AUTOINC" is in PV. If it is, AUTOINC is replaced with the value from the PR server. You could ask why it sets PKGV with the value of PV and I honestly don't remember, that could be an oversight.

Quoted 7 lines

What this did mean is that we could jettison the autoincrement code from the fetcher and just have one thing doing it.

The PKGV being overwritten by PV does look like a bug and is worth looking into. Its not the PR server at fault though, its what package.bbclass is doing with the data. I would like to look at why it was added that way as I worry there was some underlying reason :/.

I see what you mean, in package.bbclass in the "package_get_auto_pr" function:

if "AUTOINC" in pv:
srcpv = bb.fetch2.get_srcrev(d)
base_ver = "AUTOINC-%s" % version[:version.find(srcpv)] value = conn.getPR(base_ver, pkgarch, srcpv) d.setVar("PKGV", pv.replace("AUTOINC", str(value)))

I think this is the line that does it. It's surprising that it looks at PV in this part of the method, while it looks at PKGV for the "PR Server not active" case.

And from looking at this code, the simple way to prevent this from happening is making sure that PV does NOT contain the AUTOINC string, which SRCPV will have by default when using git, and PV usually includes SRCPV.

That would mean that if I set SRCPV to something that doesn't contain AUTOINC, this part of the packager would not fire and leave the PKGV as is. Now that would fix the issue for my particular case.