Skip to contentSkip to conversation

V2 opkg 0.1.8: respect to the arch when choose the alternatives

PATCH In review Robert Yang 4 participants 11 messages May 31, 2012 – Jun 4, 2012

lore
Share & tools
lore

Robert Yang

author
#0 lore raw

cover V2 opkg 0.1.8: respect to the arch when choose the alternatives

* Changes of V2:
- Add the test plan and more explanations in the cover letter as Saul suggested.

* Test info (PACKAGE_CLASSES = "package_ipk")
- Test to make sure the bug has been fixed
1) bitbake core-image-sato-sdk with MACHINE=qemux86
2) bitbake core-image-sato with MACHINE=crownbay

=== Test result:
Without this patch, the xserver-xorg-*_1.11.2-r3_i586.ipk would be install to the crownbay's image, this is incorrect since what crownbay needs are xserver-xorg*_1.9.3-r1.core2.ipk, the core2 pkgs would be installed correctly on crownbay's image. And check tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/temp/log.do_rootfs, no i586 pkgs should be installed.

- Test the upgrade (with nothing changed in the feeds)

    # opkg update
    1) ./tmp/sysroots/i686-linux/usr/bin/opkg-cl -f \
       /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/opkg.conf \
       -o /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/rootfs update
    # opkg upgrade
    2) ./tmp/sysroots/i686-linux/usr/bin/opkg-cl -f \
       /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/opkg.conf \
       -o /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/rootfs upgrade

=== Test result:
Nothing to do though there are xserver-xorg-*_1.11.2-r3_i586.ipk which have a higher version than the installed xserver-xorg*_1.9.3-r1.core2.ipk, becuase the priority of i586 is lower than core2 in core2's image.

  - Test the installation
    # opkg remove
    1) ./tmp/sysroots/i686-linux/usr/bin/opkg-cl -f \
       /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/opkg.conf \
       -o /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/rootfs \
       remove --force-depends xserver-xorg
    # opkg install
    2) ./tmp/sysroots/i686-linux/usr/bin/opkg-cl -f \
       /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/opkg.conf \
       -o /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/rootfs \
       install xserver-xorg

=== Test result:
The xserver-xorg_1.9.3-r1_core2.ipk will be installed, the reason is the same as the previous one

* More explanations

This patch only affects it's good_pkg_by_name behaviour, Here are its selecting priorities: (From high to low)

1, The pkg set manually
2, The good_pkg_by_name
3, The held pkg
4, Choose the higher arch one
5, Use the latest one

If of them is matched, then the left ones would be ignored, for example, if the first one matches, others would be ignored.

This patch affects the second one(good_by_pkg_name), what it did in the past: If there are both core2/pkg-1.0.apk and i586/pkg-1.1.apk match, it would use i586/pkg-1.1.apk since its version is higher and ignore the arch priorities, but we have set the arch priorities in opkg.conf clearly:

arch i586 31
arch core2 41

What we need is core2/pkg-1.0.apk since we may set the specific pkg version for the board (for example, the xorg-server for crownbay), let it respect to the arch priorities in such a case.

* Both rpm(fixed) and ipk have the similar issue, dpkg is OK.

* TODO
- In the following upgrade scene (with pkg changed in the feeds)

    # opkg remove
    1) ./tmp/sysroots/i686-linux/usr/bin/opkg-cl -f \
       /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/opkg.conf \
       -o /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/rootfs \
       remove --force-depends xserver-xorg
    # Remove xserver-xorg_1.9.3-r1_core2.ipk to make
    # xserver-xorg_1.11.2-r3_i586.ipk install to the image.
    2) mv tmp/deploy/ipk/core2/xserver-xorg_1.9.3-r1_core2.ipk tmp/bak/
    # Run opkg-make-index to re-create the feeds
    3) ./tmp/sysroots/i686-linux/usr/bin/opkg-make-index \
       -r tmp/deploy/ipk/core2/Packages \
       -p tmp/deploy/ipk/core2/Packages \
       -m tmp/deploy/ipk/core2
    # Install xserver-xorg_1.11.2-r3_i586.ipk
    4) ./tmp/sysroots/i686-linux/usr/bin/opkg-cl -f \
       /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/opkg.conf \
       -o /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/rootfs \
       install xserver-xorg
    # Add xserver-xorg_1.9.3-r1_core2.ipk back to the feeds
    5) mv tmp/bak/xserver-xorg_1.9.3-r1_core2.ipk tmp/deploy/ipk/core2/
    # Run opkg-make-index to re-create the feeds
    6) ./tmp/sysroots/i686-linux/usr/bin/opkg-make-index \
       -r tmp/deploy/ipk/core2/Packages \
       -p tmp/deploy/ipk/core2/Packages \
       -m tmp/deploy/ipk/core2
    # opkg update
    7) ./tmp/sysroots/i686-linux/usr/bin/opkg-cl -f \
       /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/opkg.conf \
       -o /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/rootfs update
    # opkg upgrade
    8) ./tmp/sysroots/i686-linux/usr/bin/opkg-cl -f \
       /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/opkg.conf \
       -o /path/to/tmp/work/crownbay-poky-linux/core-image-sato-1.0-r0/rootfs upgrade

Nothing will be done currently, since the opkg_upgrade.c will check the version again before really download the pkg and upgrade it, I'm not sure whether this is what we need, if we think that it is better, I'd like to work on it in another patch. There is another way to achieve:
opkg install --force-downgrade xserver-xorg

Then the xserver-xorg_1.9.3-r1_core2.ipk will be installed.

// Robert

The following changes since commit 363424cdc78cafa278008f973c2b4288297ebf8c:

  image.bbclass: Ensure ${S} is cleaned at the start of rootfs generation (2012-05-25 11:28:00 +0100)

are available in the git repository at:
git://git.pokylinux.org/poky-contrib robert/ipk_arch
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ipk_arch

Robert Yang (1):
opkg 0.1.8: respect to the arch when choose the alternatives

 .../opkg/opkg/respect-to-arch.patch                |   47 ++++++++++++++++++++
 meta/recipes-devtools/opkg/opkg_0.1.8.bb           |    3 +-
 meta/recipes-devtools/opkg/opkg_svn.bb             |    3 +-
 3 files changed, 51 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-devtools/opkg/opkg/respect-to-arch.patch
author
#1 lore raw

PATCH 1/1 opkg 0.1.8: respect to the arch when choose the alternatives

There is a bug if we:
1) bitbake core-image-sato-sdk MACHINE=qemux86
2) bitbake core-image-sato with MACHINE=crownbay

Then several pkgs in deploy/ipk/i586 would be installed to crownbay's image even if there is one in deploy/ipk/core2 and we have set the core2's priority higher than i586, when the version in deploy/ipk/i586 is higher. This doesn't work for us, for example, what the crownbay need is xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.

This is caused by opkg's selecting mechanism, if there are more than one candidates which have the same pkg name in the candidate list, for example, the same pkg with different versions, then it will use the last one which is the highest version in the list, this doesn't work for us, it should respect to the arch priorities in such a case.

This is for both denzil and master branch.

[YOCTO #2360]

Signed-off-by Robert Yang <[email protected]>
---
 .../opkg/opkg/respect-to-arch.patch                |   47 ++++++++++++++++++++
 meta/recipes-devtools/opkg/opkg_0.1.8.bb           |    3 +-
 meta/recipes-devtools/opkg/opkg_svn.bb             |    3 +-
 3 files changed, 51 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-devtools/opkg/opkg/respect-to-arch.patch
meta/recipes-devtools/opkg/opkg/respect-to-arch.patch new file +47 -0
new file mode 100644
index 0000000..6f4537f
@@ -0,0 +1,47 @@
+pkg_hash.c: respect to the arch priorities when good_pkg_by_name
+
+If there are more than one candidates which have the same pkg name in
+the candidate list, for example, the same pkg with different versions,
+then it will use the last one which is the highest version in the list,
+it should respect to the arch priorities in such a case.
+
+Upstream Status: Pending
+
+Signed-off-by: Robert Yang <[email protected]>
+---
+ libopkg/pkg_hash.c |   18 +++++++++++++++---
+ 1 files changed, 15 insertions(+), 3 deletions(-)
+
+diff --git a/libopkg/pkg_hash.c b/libopkg/pkg_hash.c
+index a99cf6b..cc048c8 100644
+--- a/libopkg/pkg_hash.c
++++ b/libopkg/pkg_hash.c
+@@ -376,10 +376,22 @@ pkg_hash_fetch_best_installation_candidate(abstract_pkg_t *apkg,
+           if (constraint_fcn(matching, cdata)) {
+              opkg_msg(DEBUG, "Candidate: %s %s.\n",
+ 			     matching->name, matching->version) ;
+-             good_pkg_by_name = matching;
++             /* Respect to the arch priorities when given alternatives */
++             if (good_pkg_by_name) {
++                 if (matching->arch_priority >= good_pkg_by_name->arch_priority) {
++                     good_pkg_by_name = matching;
++                     opkg_msg(DEBUG, "%s %s wins by priority.\n",
++                         matching->name, matching->version) ;
++                 } else
++                     opkg_msg(DEBUG, "%s %s wins by priority.\n",
++                         good_pkg_by_name->name, good_pkg_by_name->version) ;
++             } else
++                 good_pkg_by_name = matching;
+ 	     /* It has been provided by hand, so it is what user want */
+-             if (matching->provided_by_hand == 1)
+-                break;
++             if (matching->provided_by_hand == 1) {
++                 good_pkg_by_name = matching;
++                 break;
++             }
+           }
+      }
+ 
+-- 
+1.7.1
+
meta/recipes-devtools/opkg/opkg_0.1.8.bb +2 -1
index c206b37..09ff3b5 100644
@@ -3,6 +3,7 @@ require opkg.inc
 SRC_URI = "http://opkg.googlecode.com/files/opkg-${PV}.tar.gz \
            file://add_vercmp.patch \
            file://headerfix.patch \
+           file://respect-to-arch.patch \
           "
 
-PR = "${INC_PR}.0"
+PR = "${INC_PR}.1"
meta/recipes-devtools/opkg/opkg_svn.bb +2 -1
index c07d393..890ed52 100644
@@ -6,6 +6,7 @@ SRC_URI = "svn://opkg.googlecode.com/svn;module=trunk;proto=http \
            file://fix_installorder.patch \
            file://offline_postinstall.patch\
            file://track_parents.patch \
+           file://respect-to-arch.patch \
 "
 
 S = "${WORKDIR}/trunk"

@@ -13,4 +14,4 @@ S = "${WORKDIR}/trunk"
 SRCREV = "633"
 PV = "0.1.8+svnr${SRCPV}"
 
-PR = "${INC_PR}.0"
+PR = "${INC_PR}.1"
Signature · 1 line
1.7.1
#2 lore raw

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:

Quoted 9 lines

There is a bug if we:
1) bitbake core-image-sato-sdk MACHINE=qemux86
2) bitbake core-image-sato with MACHINE=crownbay

Then several pkgs in deploy/ipk/i586 would be installed to crownbay's image even if there is one in deploy/ipk/core2 and we have set the core2's priority higher than i586, when the version in deploy/ipk/i586 is higher. This doesn't work for us, for example, what the crownbay need is xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.

And this is working exactly as intended. Don't break opkg because your hardware driver situation sucks.

So: NAK on this patch.

author
#3 lore raw

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

Quoted 7 lines · Koen Kooi

Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:

Quoted 4 lines

There is a bug if we:
1) bitbake core-image-sato-sdk MACHINE=qemux86
2) bitbake core-image-sato with MACHINE=crownbay

Quoted 8 lines
Quoted 5 lines

Then several pkgs in deploy/ipk/i586 would be installed to crownbay's image even if there is one in deploy/ipk/core2 and we have set the core2's priority higher than i586, when the version in deploy/ipk/i586 is higher. This doesn't work for us, for example, what the crownbay need is xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.

And this is working exactly as intended. Don't break opkg because your hardware driver situation sucks.

Hi Koen,

Thanks for your great patient on this patch:-), I'm afraid that this is not working exactly as intended, since the rpm or dpkg doesn't have this issue. (I have fixed other issue for rpm, but not the same as this one, the xserver-xorg-1.9.3.rpm had installed correctly before the fix for rpm). and dpkg works well without any fix.

I'm also glad that if we won't fix it.

// Robert

Quoted 7 lines

So: NAK on this patch.
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

#4 lore raw

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

In reply to Koen Kooi
Quoted 12 lines · Koen Kooi

Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:

Quoted 9 lines

There is a bug if we:
1) bitbake core-image-sato-sdk MACHINE=qemux86
2) bitbake core-image-sato with MACHINE=crownbay

Then several pkgs in deploy/ipk/i586 would be installed to crownbay's image even if there is one in deploy/ipk/core2 and we have set the core2's priority higher than i586, when the version in deploy/ipk/i586 is higher. This doesn't work for us, for example, what the crownbay need is xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.

Quoted 4 lines

And this is working exactly as intended. Don't break opkg because your hardware driver situation sucks.

So: NAK on this patch.

I think we do have a problem here. For example, the system is ignoring a PREFERRED_VERSION directive here by building one thing and then installing another. We're also inconsistent between the dpkg/rpm and opkg backends. There is therefore definitely some kind of user experience issue at stake here since this behaviour is not obvious, expected or particularly correct.

The fact the example is hardware related is not particularly relevant, its the bigger picture I worry about. I know that hardware issue sucks and many people on this list have experienced pain due to it, we'd all like it to go away. Using this as a reason not to examine and potentially fix some problematic package manager behaviour is not right IMO though. The world isn't perfect, sucky hardware/software exists, we need to work with it.

So to be honest I'm leaning towards taking the patch. I would be interested in other people's opinions though...

Cheers,

Richard

#5 lore raw

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

Op 1 jun. 2012, om 10:17 heeft Richard Purdie het volgende geschreven:

Quoted 13 lines · Koen Kooi
Quoted 12 lines

Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:

Quoted 9 lines

There is a bug if we:
1) bitbake core-image-sato-sdk MACHINE=qemux86
2) bitbake core-image-sato with MACHINE=crownbay

Then several pkgs in deploy/ipk/i586 would be installed to crownbay's image even if there is one in deploy/ipk/core2 and we have set the core2's priority higher than i586, when the version in deploy/ipk/i586 is higher. This doesn't work for us, for example, what the crownbay need is xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.

Quoted 7 lines
Quoted 4 lines

And this is working exactly as intended. Don't break opkg because your hardware driver situation sucks.

So: NAK on this patch.

I think we do have a problem here. For example, the system is ignoring a PREFERRED_VERSION directive here

A PREFERRED_VERSION set in a machine.conf, which is the wrong place. Let's change the above build sequence to this:

1) MACHINE=qemux86 bitbake xserver-xorg
2) MACHINE=othercore2machine bitbake xserver-xorg
3) MACHINE=crownbay bitbake xserver-xorg

Oh look, I get 1.11.2 on crownbay regardless of this patch.

Quoted 8 lines

by building one thing and then
installing another. We're also inconsistent between the dpkg/rpm and opkg backends. There is therefore definitely some kind of user experience issue at stake here since this behaviour is not obvious, expected or particularly correct.

The fact the example is hardware related is not particularly relevant, its the bigger picture I worry about

I also worry about the bigger picture, especially about what Martin said about this breaking feeds.

#6 lore raw

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

Quoted 18 lines · Koen Kooi

Op 1 jun. 2012, om 10:17 heeft Richard Purdie het volgende geschreven:

Quoted 16 lines · Koen Kooi
Quoted 15 lines

Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:

Quoted 9 lines

There is a bug if we:
1) bitbake core-image-sato-sdk MACHINE=qemux86
2) bitbake core-image-sato with MACHINE=crownbay

Then several pkgs in deploy/ipk/i586 would be installed to crownbay's image even if there is one in deploy/ipk/core2 and we have set the core2's priority higher than i586, when the version in deploy/ipk/i586 is higher. This doesn't work for us, for example, what the crownbay need is xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.

And this is working exactly as intended. Don't break opkg because your hardware driver situation sucks.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

So: NAK on this patch.

I think we do have a problem here. For example, the system is ignoring a PREFERRED_VERSION directive here

A PREFERRED_VERSION set in a machine.conf, which is the wrong place.

Its strongly not recommended. You can do it and if things are setup correctly, we do support machine specific alterations however...

Quoted 7 lines

Let's change the above build sequence to this:

1) MACHINE=qemux86 bitbake xserver-xorg
2) MACHINE=othercore2machine bitbake xserver-xorg
3) MACHINE=crownbay bitbake xserver-xorg

Oh look, I get 1.11.2 on crownbay regardless of this patch.

I've been assuming this xserver is marked as machine specific. If its not, that is a bug and we can fix that.

Quoted 11 lines
Quoted 8 lines

by building one thing and then
installing another. We're also inconsistent between the dpkg/rpm and opkg backends. There is therefore definitely some kind of user experience issue at stake here since this behaviour is not obvious, expected or particularly correct.

The fact the example is hardware related is not particularly relevant, its the bigger picture I worry about

I also worry about the bigger picture, especially about what Martin said about this breaking feeds.

As far as I understood Martin's comments, this actually helps avoid some of the issues he's been experiencing with feeds?

Martin has a problem where machines are ending up with unoptimised versions of code on them and it would be good to fix opkg behaviour to do what people expect...

Cheers,

Richard

#7 lore raw

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

Op 1 jun. 2012, om 12:02 heeft Richard Purdie het volgende geschreven:

Quoted 31 lines · Koen Kooi
Quoted 18 lines

Op 1 jun. 2012, om 10:17 heeft Richard Purdie het volgende geschreven:

Quoted 16 lines · Koen Kooi
Quoted 15 lines

Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:

Quoted 9 lines

There is a bug if we:
1) bitbake core-image-sato-sdk MACHINE=qemux86
2) bitbake core-image-sato with MACHINE=crownbay

Then several pkgs in deploy/ipk/i586 would be installed to crownbay's image even if there is one in deploy/ipk/core2 and we have set the core2's priority higher than i586, when the version in deploy/ipk/i586 is higher. This doesn't work for us, for example, what the crownbay need is xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.

And this is working exactly as intended. Don't break opkg because your hardware driver situation sucks.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

So: NAK on this patch.

I think we do have a problem here. For example, the system is ignoring a PREFERRED_VERSION directive here

A PREFERRED_VERSION set in a machine.conf, which is the wrong place.

Its strongly not recommended. You can do it and if things are setup correctly, we do support machine specific alterations however...

Quoted 2 lines

Let's change the above build sequence to this:

Quoted 8 lines
Quoted 5 lines

1) MACHINE=qemux86 bitbake xserver-xorg
2) MACHINE=othercore2machine bitbake xserver-xorg
3) MACHINE=crownbay bitbake xserver-xorg

Oh look, I get 1.11.2 on crownbay regardless of this patch.

I've been assuming this xserver is marked as machine specific. If its not, that is a bug and we can fix that.

the X server not machine specific and that is NOT a bug. The recipe for the DDX only works with a specific X version. The real problem here is that xf86-video-something (again, not machine specific, think pci cards) has a strict requirement on the x server version. This is now purely a DISTRO problem. A few possible solutions are:

1) Lock X to 1.9.3 for all compatible x86 archs in your DISTRO
2) Lock X to 1.11.2 for all compatible x86 archs in your DISTRO, live with the crownbay breakage
3) Only use a package manager with no notion of compatible archs (dpkg) or with strong version pinning (rpm)
4) Be really careful with the build sequence and structure your feed configs to not have compatible archs in their feed URIs
5) remove 'x11' from DISTRO_FEATURES

This is a point where you cannot make everybody happy. But as you say below, let's decouple the above problem from the arch vs version discussion.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

by building one thing and then
installing another. We're also inconsistent between the dpkg/rpm and opkg backends. There is therefore definitely some kind of user experience issue at stake here since this behaviour is not obvious, expected or particularly correct.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

The fact the example is hardware related is not particularly relevant, its the bigger picture I worry about

I also worry about the bigger picture, especially about what Martin said about this breaking feeds.

As far as I understood Martin's comments, this actually helps avoid some of the issues he's been experiencing with feeds?

No, it allows you to add a package-of-death. Example:

broken-app_1.0_machine.ipk is in the feeds, being machine specific broken-apps author fixes the machine specific bits properly broken-app_2.0_i586.ipk hits the feeds and is not picked up.

Quoted 3 lines

Martin has a problem where machines are ending up with unoptimised versions of code on them and it would be good to fix opkg behaviour to do what people expect...

Yes, feed updates are not atomic, but this patch breaks moving packages between archs like the broken-app example above. In angstrom a machine doesn't see feeds with compatible archs at runtime, e.g. beagleboard sees beagleboard, armv7a and noarch URIs. This reduces these kind of problems to images built with OE. I must note that this setup for feed URIs was done out of bandwidth, storage and ram concerns (good old ipkg), not to avoid Martins problems :)

regards,

Koen

author
#8 lore raw

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

Quoted 65 lines · Koen Kooi

Op 1 jun. 2012, om 12:02 heeft Richard Purdie het volgende geschreven:

Quoted 31 lines · Koen Kooi
Quoted 18 lines

Op 1 jun. 2012, om 10:17 heeft Richard Purdie het volgende geschreven:

Quoted 16 lines · Koen Kooi
Quoted 15 lines

Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:

Quoted 9 lines

There is a bug if we:
1) bitbake core-image-sato-sdk MACHINE=qemux86
2) bitbake core-image-sato with MACHINE=crownbay

Then several pkgs in deploy/ipk/i586 would be installed to crownbay's image even if there is one in deploy/ipk/core2 and we have set the core2's priority higher than i586, when the version in deploy/ipk/i586 is higher. This doesn't work for us, for example, what the crownbay need is xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.

And this is working exactly as intended. Don't break opkg because your hardware driver situation sucks.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

So: NAK on this patch.

I think we do have a problem here. For example, the system is ignoring a PREFERRED_VERSION directive here

A PREFERRED_VERSION set in a machine.conf, which is the wrong place.

Its strongly not recommended. You can do it and if things are setup correctly, we do support machine specific alterations however...

Quoted 2 lines

Let's change the above build sequence to this:

Quoted 8 lines
Quoted 5 lines

1) MACHINE=qemux86 bitbake xserver-xorg
2) MACHINE=othercore2machine bitbake xserver-xorg
3) MACHINE=crownbay bitbake xserver-xorg

Oh look, I get 1.11.2 on crownbay regardless of this patch.

I've been assuming this xserver is marked as machine specific. If its not, that is a bug and we can fix that.

the X server not machine specific and that is NOT a bug. The recipe for the DDX only works with a specific X version. The real problem here is that xf86-video-something (again, not machine specific, think pci cards) has a strict requirement on the x server version. This is now purely a DISTRO problem. A few possible solutions are:

1) Lock X to 1.9.3 for all compatible x86 archs in your DISTRO
2) Lock X to 1.11.2 for all compatible x86 archs in your DISTRO, live with the crownbay breakage
3) Only use a package manager with no notion of compatible archs (dpkg) or with strong version pinning (rpm)
4) Be really careful with the build sequence and structure your feed configs to not have compatible archs in their feed URIs
5) remove 'x11' from DISTRO_FEATURES

This is a point where you cannot make everybody happy. But as you say below, let's decouple the above problem from the arch vs version discussion.

Quoted 12 lines
Quoted 11 lines
Quoted 8 lines

by building one thing and then
installing another. We're also inconsistent between the dpkg/rpm and opkg backends. There is therefore definitely some kind of user experience issue at stake here since this behaviour is not obvious, expected or particularly correct.

The fact the example is hardware related is not particularly relevant, its the bigger picture I worry about

I also worry about the bigger picture, especially about what Martin said about this breaking feeds.

Quoted 8 lines
Quoted 2 lines

As far as I understood Martin's comments, this actually helps avoid some of the issues he's been experiencing with feeds?

No, it allows you to add a package-of-death. Example:

broken-app_1.0_machine.ipk is in the feeds, being machine specific broken-apps author fixes the machine specific bits properly broken-app_2.0_i586.ipk hits the feeds and is not picked up.

I think that the broken-app_2.0_i586.ipk came unexpectedly,
there was a broken-app_1.0_machine.ipk (maybe also broken-app_1.0_i586.ipk, but it didn't matter), if it has been fixed to version 2.0, then there should be also a broken-app_2.0_machine.ipk (and it would be picked up).

// Robert

Quoted 16 lines
Quoted 4 lines

Martin has a problem where machines are ending up with unoptimised versions of code on them and it would be good to fix opkg behaviour to do what people expect...

Yes, feed updates are not atomic, but this patch breaks moving packages between archs like the broken-app example above. In angstrom a machine doesn't see feeds with compatible archs at runtime, e.g. beagleboard sees beagleboard, armv7a and noarch URIs. This reduces these kind of problems to images built with OE. I must note that this setup for feed URIs was done out of bandwidth, storage and ram concerns (good old ipkg), not to avoid Martins problems :)

regards,

Koen
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

#9 lore raw

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

Quoted 73 lines · Robert Yang
Quoted 70 lines · Koen Kooi

Op 1 jun. 2012, om 12:02 heeft Richard Purdie het volgende geschreven:

Quoted 31 lines · Koen Kooi
Quoted 18 lines

Op 1 jun. 2012, om 10:17 heeft Richard Purdie het volgende geschreven:

Quoted 16 lines · Koen Kooi
Quoted 15 lines

Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:

Quoted 9 lines

There is a bug if we:
1) bitbake core-image-sato-sdk MACHINE=qemux86
2) bitbake core-image-sato with MACHINE=crownbay

Then several pkgs in deploy/ipk/i586 would be installed to crownbay's image even if there is one in deploy/ipk/core2 and we have set the core2's priority higher than i586, when the version in deploy/ipk/i586 is higher. This doesn't work for us, for example, what the crownbay need is xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.

And this is working exactly as intended. Don't break opkg because your hardware driver situation sucks.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

So: NAK on this patch.

I think we do have a problem here. For example, the system is ignoring a PREFERRED_VERSION directive here

A PREFERRED_VERSION set in a machine.conf, which is the wrong place.

Its strongly not recommended. You can do it and if things are setup correctly, we do support machine specific alterations however...

Quoted 2 lines

Let's change the above build sequence to this:

Quoted 8 lines
Quoted 5 lines

1) MACHINE=qemux86 bitbake xserver-xorg
2) MACHINE=othercore2machine bitbake xserver-xorg
3) MACHINE=crownbay bitbake xserver-xorg

Oh look, I get 1.11.2 on crownbay regardless of this patch.

I've been assuming this xserver is marked as machine specific. If its not, that is a bug and we can fix that.

the X server not machine specific and that is NOT a bug. The recipe for the DDX only works with a specific X version. The real problem here is that xf86-video-something (again, not machine specific, think pci cards) has a strict requirement on the x server version. This is now purely a DISTRO problem. A few possible solutions are:

1) Lock X to 1.9.3 for all compatible x86 archs in your DISTRO
2) Lock X to 1.11.2 for all compatible x86 archs in your DISTRO, live with the crownbay breakage
3) Only use a package manager with no notion of compatible archs (dpkg) or with strong version pinning (rpm)
4) Be really careful with the build sequence and structure your feed configs to not have compatible archs in their feed URIs
5) remove 'x11' from DISTRO_FEATURES

This is a point where you cannot make everybody happy. But as you say below, let's decouple the above problem from the arch vs version discussion.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

by building one thing and then
installing another. We're also inconsistent between the dpkg/rpm and opkg backends. There is therefore definitely some kind of user experience issue at stake here since this behaviour is not obvious, expected or particularly correct.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

The fact the example is hardware related is not particularly relevant, its the bigger picture I worry about

I also worry about the bigger picture, especially about what Martin said about this breaking feeds.

As far as I understood Martin's comments, this actually helps avoid some of the issues he's been experiencing with feeds?

No, it allows you to add a package-of-death. Example:

Quoted 8 lines
Quoted 3 lines

broken-app_1.0_machine.ipk is in the feeds, being machine specific broken-apps author fixes the machine specific bits properly broken-app_2.0_i586.ipk hits the feeds and is not picked up.

I think that the broken-app_2.0_i586.ipk came unexpectedly,
there was a broken-app_1.0_machine.ipk (maybe also broken-app_1.0_i586.ipk, but it didn't matter), if it has been fixed to version 2.0, then there should be also a broken-app_2.0_machine.ipk (and it would be picked up).

If broken-app developers are now detecting machine capabilities (or whatever) since 2.0 in runtime, then we don't need to build broken-app for every single machine we support, so changing PACKAGE_ARCH from MACHINE_ARCH to TUNE_PKGARCH is right thing to do to save compile time and feed disk space.

Same use-case broken by this patch reported here:
http://lists.linuxtogo.org/pipermail/openembedded-core/2012-May/023154.html

Cheers,

Quoted 24 lines

// Robert

Quoted 7 lines
Quoted 4 lines

Martin has a problem where machines are ending up with unoptimised versions of code on them and it would be good to fix opkg behaviour to do what people expect...

Yes, feed updates are not atomic, but this patch breaks moving packages between archs like the broken-app example above. In angstrom a machine doesn't see feeds with compatible archs at runtime, e.g. beagleboard sees beagleboard, armv7a and noarch URIs. This reduces these kind of problems to images built with OE. I must note that this setup for feed URIs was done out of bandwidth, storage and ram concerns (good old ipkg), not to avoid Martins problems :)

Quoted 9 lines

regards,

Koen
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#10 lore raw

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

Op 4 jun. 2012, om 12:39 heeft Martin Jansa het volgende geschreven:

Quoted 83 lines · Robert Yang
Quoted 73 lines · Koen Kooi
Quoted 70 lines

Op 1 jun. 2012, om 12:02 heeft Richard Purdie het volgende geschreven:

Quoted 31 lines · Koen Kooi
Quoted 18 lines

Op 1 jun. 2012, om 10:17 heeft Richard Purdie het volgende geschreven:

Quoted 16 lines · Koen Kooi
Quoted 15 lines

Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:

Quoted 9 lines

There is a bug if we:
1) bitbake core-image-sato-sdk MACHINE=qemux86
2) bitbake core-image-sato with MACHINE=crownbay

Then several pkgs in deploy/ipk/i586 would be installed to crownbay's image even if there is one in deploy/ipk/core2 and we have set the core2's priority higher than i586, when the version in deploy/ipk/i586 is higher. This doesn't work for us, for example, what the crownbay need is xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.

And this is working exactly as intended. Don't break opkg because your hardware driver situation sucks.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

So: NAK on this patch.

I think we do have a problem here. For example, the system is ignoring a PREFERRED_VERSION directive here

A PREFERRED_VERSION set in a machine.conf, which is the wrong place.

Its strongly not recommended. You can do it and if things are setup correctly, we do support machine specific alterations however...

Quoted 2 lines

Let's change the above build sequence to this:

Quoted 8 lines
Quoted 5 lines

1) MACHINE=qemux86 bitbake xserver-xorg
2) MACHINE=othercore2machine bitbake xserver-xorg
3) MACHINE=crownbay bitbake xserver-xorg

Oh look, I get 1.11.2 on crownbay regardless of this patch.

I've been assuming this xserver is marked as machine specific. If its not, that is a bug and we can fix that.

the X server not machine specific and that is NOT a bug. The recipe for the DDX only works with a specific X version. The real problem here is that xf86-video-something (again, not machine specific, think pci cards) has a strict requirement on the x server version. This is now purely a DISTRO problem. A few possible solutions are:

1) Lock X to 1.9.3 for all compatible x86 archs in your DISTRO
2) Lock X to 1.11.2 for all compatible x86 archs in your DISTRO, live with the crownbay breakage
3) Only use a package manager with no notion of compatible archs (dpkg) or with strong version pinning (rpm)
4) Be really careful with the build sequence and structure your feed configs to not have compatible archs in their feed URIs
5) remove 'x11' from DISTRO_FEATURES

This is a point where you cannot make everybody happy. But as you say below, let's decouple the above problem from the arch vs version discussion.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

by building one thing and then
installing another. We're also inconsistent between the dpkg/rpm and opkg backends. There is therefore definitely some kind of user experience issue at stake here since this behaviour is not obvious, expected or particularly correct.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

The fact the example is hardware related is not particularly relevant, its the bigger picture I worry about

I also worry about the bigger picture, especially about what Martin said about this breaking feeds.

As far as I understood Martin's comments, this actually helps avoid some of the issues he's been experiencing with feeds?

No, it allows you to add a package-of-death. Example:

Quoted 8 lines
Quoted 3 lines

broken-app_1.0_machine.ipk is in the feeds, being machine specific broken-apps author fixes the machine specific bits properly broken-app_2.0_i586.ipk hits the feeds and is not picked up.

I think that the broken-app_2.0_i586.ipk came unexpectedly,
there was a broken-app_1.0_machine.ipk (maybe also broken-app_1.0_i586.ipk, but it didn't matter), if it has been fixed to version 2.0, then there should be also a broken-app_2.0_machine.ipk (and it would be picked up).

Quoted 5 lines

If broken-app developers are now detecting machine capabilities (or whatever) since 2.0 in runtime, then we don't need to build broken-app for every single machine we support, so changing PACKAGE_ARCH from MACHINE_ARCH to TUNE_PKGARCH is right thing to do to save compile time and feed disk space.

Exactly

Quoted 2 lines

Same use-case broken by this patch reported here:
http://lists.linuxtogo.org/pipermail/openembedded-core/2012-May/023154.html

I borrowed heavily from your post since both Richard and Robert don't seem to see the damage this will do to feeds when a package/recipe changes archs.

regards,

Koen