opkg/rpm inconsistencies in multilib image installations

tl;dr: multilib apparently defines its own root directory underneath ${IMAGE_ROOT} which includes copies of /etc files like passwd that are not known to pseudo. opkg and rpm differ in whether useradd scripts are run when a multilib package is installed, resulting in failures to add users/groups during multilib install when using opkg because a sanity check thinks they already exist because pseudo is looking somewhere else.

Details:

Ross pointed out in http://lists.openembedded.org/pipermail/openembedded-core/2014-November/099032.html that the patches to prevent using host /etc files in pseudo failed under multilib and asked me to look into it.

Recall that with these patches PSEUDO_PASSWD (where pseudo looks up users and groups for getpwnam(3) etc) is set in bitbake.conf to:

PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${STAGING_DIR_NATIVE}"

where STAGING_DIR_NATIVE holds the fallback files that provide only enough entries to get fs-perms.txt to work before the target staging /etc exists.

Because images are different and with some packagers base-passwd can't be installed unless user root can be resolved, in an (updated) image.bbclass we use this:

# Prefer image, but use the fallback files for lookups if the image ones # aren't yet available.
PSEUDO_PASSWD = "${IMAGE_ROOTFS}:${STAGING_DIR_NATIVE}"

Adding this line to perform_groupadd in useradd_base.bbclass
bbnote "rootdir $rootdir
PSEUDO_PASSWD=${PSEUDO_PASSWD}=$PSEUDO_PASSWD"

and adding linebreaks for readability, I'm finding that when lib32-dbus-1 is installed with PACKAGE_CLASSES = "package_ipk" we have:

rootdir
/prj/oe/omap/build-ab-multilib/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/multilib/lib32

PSEUDO_PASSWD
=/prj/oe/omap/build-ab-multilib/tmp/sysroots/lib32-qemux86-64 :/prj/oe/omap/build-ab-multilib/tmp/sysroots/x86_64-linux =/prj/oe/omap/build-ab-multilib/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/rootfs

:/prj/oe/omap/build-ab-multilib/tmp/sysroots/x86_64-linux

In other words, useradd.bbclass (1) thinks the etc directory is in $D, the bitbake variable PSEUDO_PASSWD during build (2) thinks it's in the multilib 32-bit target staging directory, and the PSEUDO_PASSWD actually running (3) thinks it's in IMAGE_ROOT.

Because of this, useradd.bbclass can't find netdev in $D/etc/group so invokes groupadd which does a sanity check and complains that netdev already exists in ${IMAGE_ROOT}/etc/group and things crumble to a stop.

Interestingly the build+install works with PACKAGE_CLASSES = "package_rpm" which shows this for the install of dbus-1-1.8.2-r0.3.core2_64.rpm:

rootdir
/prj/oe/omap/build-ab-multilib/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/rootfs

PSEUDO_PASSWD
=/prj/oe/omap/build-ab-multilib/tmp/sysroots/qemux86-64 :/prj/oe/omap/build-ab-multilib/tmp/sysroots/x86_64-linux =/prj/oe/omap/build-ab-multilib/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/rootfs

:/prj/oe/omap/build-ab-multilib/tmp/sysroots/x86_64-linux

and either does not install dbus-1-1.8.2-r0.3.lib32_x86.rpm or somehow prevents the groupadd scripts from being run.

Whatever's going on here, RPM and OPKG appear to differ in the environment and selected packages for multilib images, which surely isn't right. I didn't check deb.

Once somebody reconciles the packager behaviors, I'll return to deal with any remaining changes to pseudo. If we have to allow multilib to create users/groups in a non-standard root directory it's gonna get messy.

Peter

Quoted 7 lines · Peter A. Bigot

tl;dr: multilib apparently defines its own root directory underneath ${IMAGE_ROOT} which includes copies of /etc files like passwd that are not known to pseudo. opkg and rpm differ in whether useradd scripts are run when a multilib package is installed, resulting in failures to add users/groups during multilib install when using opkg because a sanity check thinks they already exist because pseudo is looking somewhere else.

Actually, I'm going to retract this whole thing, since I'm now unable to reproduce any of the problems described here. Possibly because of changes to the pseudo patches that fixed problems I thought were unrelated to this behavior.

Sorry for the noise.

Quoted 9 lines · Peter A. Bigot

Quoted 7 lines · Peter A. Bigot

tl;dr: multilib apparently defines its own root directory underneath ${IMAGE_ROOT} which includes copies of /etc files like passwd that are not known to pseudo. opkg and rpm differ in whether useradd scripts are run when a multilib package is installed, resulting in failures to add users/groups during multilib install when using opkg because a sanity check thinks they already exist because pseudo is looking somewhere else.

Quoted 6 lines

Actually, I'm going to retract this whole thing, since I'm now unable to reproduce any of the problems described here. Possibly because of changes to the pseudo patches that fixed problems I thought were unrelated to this behavior.

Sorry for the noise.

FWIW you are right in that the way multilib works with rpm is different to ipk/deb, you can end up with two different rootfs directories which are then reconciled. I do have some concerns about what would happen to the passwd/group files in this scenario since they could easily become out of sync. I suspect we'd have to create a pathological configuration to manage that but its probably possible. I *think* the system would at least error out if that pathological case occured.

That said, I suspect this problem exists before your patchset so isn't a new issue.

Cheers,

Richard