glibc: make nscd optional
PATCH 1/7 glibc: make nscd optional
NSCD itself is already packaged into its own package, but if it is never going to be needed then it can be disabled and it won't be called from glibc at all. Add a PACKAGECONFIG for NSCD that is enabled by default.
As a side effect if NSCD is disable, glibc with and without systemd is binary identical, which helps sstate reuse.
--- meta/recipes-core/glibc/glibc_2.32.bb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
meta/recipes-core/glibc/glibc_2.32.bb +3 -1
@@ -69,13 +69,15 @@ EXTRA_OECONF = "--enable-kernel=${OLDEST_KERNEL} \ --enable-stackguard-randomization \ --disable-crypt \ --with-default-link \ - --enable-nscd \ ${@bb.utils.contains_any('SELECTED_OPTIMIZATION', '-O0 -Og', '--disable-werror', '', d)} \ ${GLIBCPIE} \ ${GLIBC_EXTRA_OECONF}" EXTRA_OECONF += "${@get_libc_fpu_setting(bb, d)}" +PACKAGECONFIG ??= "nscd" +PACKAGECONFIG[nscd] = "--enable-nscd,--disable-nscd" + do_patch_append() { bb.build.exec_func('do_fix_readlib_c', d) }
Signature · 1 line
2.25.1
PATCH 2/7 utils: respect scheduler affinity in cpu_count()
cpu_count() returns multiprocessing.cpu_count() but that is simply returns os.cpu_count() so we could use that directly.
However this returns the number of CPUs on the host, not the number of usable CPUs on the host. If the user is using scheduler affinity then the number of usable CPUs may be less, so when determining how many cores we can use check the affinity instead.
--- meta/lib/oe/utils.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
meta/lib/oe/utils.py +1 -2
@@ -249,8 +249,7 @@ def trim_version(version, num_parts=2): return trimmed def cpu_count(at_least=1): - import multiprocessing - cpus = multiprocessing.cpu_count() + cpus = len(os.sched_getaffinity(0)) return max(cpus, at_least) def execute_pre_post_process(d, cmds):
Signature · 1 line
2.25.1
PATCH 3/7 rpm: disable libarchive use
In reply to Ross Burtonlibarchive is only needed for the rpm2archive tool, as this is of limited use and libarchive is a non-trivial build dependency make this optional and disabled by default.
--- meta/recipes-devtools/rpm/rpm_4.15.1.bb | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
meta/recipes-devtools/rpm/rpm_4.15.1.bb +4 -3
@@ -49,7 +49,7 @@ SRCREV = "ab2179452c5be276a6b96c591afded485c7e58c3" S = "${WORKDIR}/git" -DEPENDS = "openssl libarchive db file popt xz bzip2 dbus elfutils python3" +DEPENDS = "openssl db file popt xz bzip2 dbus elfutils python3" DEPENDS_append_class-native = " file-replacement-native bzip2-replacement-native" inherit autotools gettext pkgconfig python3native
@@ -74,6 +74,7 @@ BBCLASSEXTEND = "native nativesdk" PACKAGECONFIG ??= "" PACKAGECONFIG[imaevm] = "--with-imaevm,,ima-evm-utils" +PACKAGECONFIG[rpm2archive] = "--with-archive,--without-archive,libarchive" ASNEEDED = ""
@@ -95,7 +96,7 @@ WRAPPER_TOOLS = " \ do_install_append_class-native() { for tool in ${WRAPPER_TOOLS}; do - create_wrapper ${D}$tool \ + test -x ${D}$tool && create_wrapper ${D}$tool \ RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \ RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \ MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \
@@ -105,7 +106,7 @@ do_install_append_class-native() { do_install_append_class-nativesdk() { for tool in ${WRAPPER_TOOLS}; do - create_wrapper ${D}$tool \ + test -x ${D}$tool && create_wrapper ${D}$tool \ RPM_CONFIGDIR='`dirname $''realpath`'/${@os.path.relpath(d.getVar('libdir'), d.getVar('bindir'))}/rpm \ RPM_ETCCONFIGDIR='$'{RPM_ETCCONFIGDIR-'`dirname $''realpath`'/${@os.path.relpath(d.getVar('sysconfdir'), d.getVar('bindir'))}/..} \ MAGIC='`dirname $''realpath`'/${@os.path.relpath(d.getVar('datadir'), d.getVar('bindir'))}/misc/magic.mgc \
Signature · 1 line
2.25.1
PATCH 4/7 rpm: add PACKAGECONFIG for the systemd inhibit plugin
In reply to Ross BurtonRPM ships a systemd inhibit plugin, that will tell systemd to inhibit a reboot or sleep during a package upgrades.
For native RPM this is entirely useless, and for target it's only useful if you're using systemd+logind+rpm.
This plugin uses DBus which means it depends on expat -> cmake-native -> libarchive-native curl-native, which is quite a dependency tree to need in early build (required to build packages via rpm-native).
It was previously forcibly disabled for native packages but the build dependency on DBus remained. Remove that logic and add a PACKAGECONFIG for the plugin that is only enabled for target builds with systemd and explicitly disabled for native/nativesdk builds.
--- meta/recipes-devtools/rpm/rpm_4.15.1.bb | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)
meta/recipes-devtools/rpm/rpm_4.15.1.bb +9 -8
@@ -49,7 +49,7 @@ SRCREV = "ab2179452c5be276a6b96c591afded485c7e58c3" S = "${WORKDIR}/git" -DEPENDS = "openssl db file popt xz bzip2 dbus elfutils python3" +DEPENDS = "openssl db file popt xz bzip2 elfutils python3" DEPENDS_append_class-native = " file-replacement-native bzip2-replacement-native" inherit autotools gettext pkgconfig python3native
@@ -62,18 +62,19 @@ EXTRA_OECONF_append = " --without-lua --enable-python --with-crypto=openssl" EXTRA_OECONF_append_libc-musl = " --disable-nls --disable-openmp" # --sysconfdir prevents rpm from attempting to access machine-specific configuration in sysroot/etc; we need to have it in rootfs -# # --localstatedir prevents rpm from writing its database to native sysroot when building images -# -# Disable dbus for native, so that rpm doesn't attempt to inhibit shutdown via session dbus even when plugins support is enabled. -# Also disable plugins by default for native. -EXTRA_OECONF_append_class-native = " --sysconfdir=/etc --localstatedir=/var --disable-plugins" -EXTRA_OECONF_append_class-nativesdk = " --sysconfdir=/etc --disable-plugins" +EXTRA_OECONF_append_class-native = " --sysconfdir=/etc --localstatedir=/var" +EXTRA_OECONF_append_class-nativesdk = " --sysconfdir=/etc" BBCLASSEXTEND = "native nativesdk" -PACKAGECONFIG ??= "" +PACKAGECONFIG ??= "${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'inhibit', '', d)}" +# The inhibit plugin serves no purpose outside of the target +PACKAGECONFIG_remove_class-native = "inhibit" +PACKAGECONFIG_remove_class-nativesdk = "inhibit" + PACKAGECONFIG[imaevm] = "--with-imaevm,,ima-evm-utils" +PACKAGECONFIG[inhibit] = "--enable-inhibit-plugin,--disable-inhibit-plugin,dbus" PACKAGECONFIG[rpm2archive] = "--with-archive,--without-archive,libarchive" ASNEEDED = ""
Signature · 1 line
2.25.1
PATCH 5/7 utils: add umask changing context manager
In reply to Ross BurtonAdd a umask context manager which can be used to temporarily change the umask in a 'with' block.
--- bitbake/lib/bb/utils.py | 11 +++++++++++ 1 file changed, 11 insertions(+)
bitbake/lib/bb/utils.py +11 -0
@@ -944,6 +944,17 @@ def which(path, item, direction = 0, history = False, executable=False): return "", hist return "" +@contextmanager +def umask(new_mask): + """ + Context manager to set the umask to a specific mask, and restore it afterwards. + """ + current_mask = os.umask(new_mask) + try: + yield + finally: + os.umask(current_mask) + def to_boolean(string, default=None): if not string: return default
Signature · 1 line
2.25.1
PATCH 6/7 siggen: use correct umask when writing siginfo
In reply to Ross BurtonWe try to write sstate with group-write permissions so that sstate-cache can be shared between multiple users. However the siginfo files are created in various tasks which may set their own umask (such as do_populate_sysroot, 0022). This results in no group write permission on the intermediate directories, which is fatal when sharing a cache.
Fix this by wrapping the siginfo mkdir in a umask change to 0002.
--- bitbake/lib/bb/siggen.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
bitbake/lib/bb/siggen.py +2 -1
@@ -358,7 +358,8 @@ class SignatureGeneratorBasic(SignatureGenerator): else: sigfile = stampbase + "." + task + ".sigbasedata" + "." + self.basehash[tid] - bb.utils.mkdirhier(os.path.dirname(sigfile)) + with bb.utils.umask(0o002): + bb.utils.mkdirhier(os.path.dirname(sigfile)) data = {} data['task'] = task
Signature · 1 line
2.25.1
PATCH 7/7 sstate: set mode explicitly when creating directories in sstate-cache
In reply to Ross BurtonWhen creating directories in the sstate-cache, explicitly set the mode passed to mkdir to 0775 so that the directories are group writable, as otherwise they cannot be shared with other users.
--- meta/classes/sstate.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/sstate.bbclass +1 -1
@@ -787,7 +787,7 @@ sstate_create_package () { return fi - mkdir -p `dirname ${SSTATE_PKG}` + mkdir --mode=0775 -p `dirname ${SSTATE_PKG}` TFILE=`mktemp ${SSTATE_PKG}.XXXXXXXX` # Use pigz if available
Signature · 1 line
2.25.1
Re: [OE-core] [PATCH 7/7] sstate: set mode explicitly when creating directories in sstate-cache
I worry that this could be problematic for other ways. Security requirements for an org or even users who want to share the sstate case 0777 etc.
Wouldn't it be better to warn the user that their umask won't allow them to share this with others, and give them instructions and opening the umask?
--Mark
Quoted 27 lines · Ross Burton
When creating directories in the sstate-cache, explicitly set the mode passed to mkdir to 0775 so that the directories are group writable, as otherwise they cannot be shared with other users.
Signed-off-by Ross Burton <[email protected]>--- meta/classes/sstate.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)meta/classes/sstate.bbclass +1
-1@@ -787,7 +787,7 @@ sstate_create_package () { return fi - mkdir -p `dirname ${SSTATE_PKG}` + mkdir --mode=0775 -p `dirname ${SSTATE_PKG}` TFILE=`mktemp ${SSTATE_PKG}.XXXXXXXX` # Use pigz if available
Re: [OE-core] [PATCH 7/7] sstate: set mode explicitly when creating directories in sstate-cache
Quoted 5 lines · Mark Hatle
I worry that this could be problematic for other ways. Security requirements for an org or even users who want to share the sstate case 0777 etc.
Wouldn't it be better to warn the user that their umask won't allow them to share this with others, and give them instructions and opening the umask?
FWIW, ten lines lower we chmod 664 the actual data, so this is just in keeping with existing behaviour.
Ross
Quoted 24 lines · Ross Burton
NSCD itself is already packaged into its own package, but if it is never going to be needed then it can be disabled and it won't be called from glibc at all. Add a PACKAGECONFIG for NSCD that is enabled by default.
As a side effect if NSCD is disable, glibc with and without systemd is binary identical, which helps sstate reuse.
Signed-off-by Ross Burton <[email protected]>--- meta/recipes-core/glibc/glibc_2.32.bb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)meta/recipes-core/glibc/glibc_2.32.bb +0
-1@@ -69,13 +69,15 @@ EXTRA_OECONF = "--enable-kernel=${OLDEST_KERNEL} \ --enable-stackguard-randomization \ --disable-crypt \ --with-default-link \ - --enable-nscd \ ${@bb.utils.contains_any('SELECTED_OPTIMIZATION', '-O0 -Og', '--disable-werror', '', d)} \ ${GLIBCPIE} \
meta/recipes-core/glibc/glibc_2.32.bb EXTRA_OECONF = "--enable-kernel=${OLDEST_KERNEL} \ · line 83
Quoted 10 lines
meta/recipes-core/glibc/glibc_2.32.bb +3
-0${GLIBC_EXTRA_OECONF}" EXTRA_OECONF += "${@get_libc_fpu_setting(bb, d)}" +PACKAGECONFIG ??= "nscd" +PACKAGECONFIG[nscd] = "--enable-nscd,--disable-nscd" + do_patch_append() { bb.build.exec_func('do_fix_readlib_c', d) }
The patch looks good to me but
I wonder if this goes beyond glibc and more like a distro features, where you can also ignore nsswitch.conf and
packages being aware of it.
Quoted 6 lines
--
2.25.1