Skip to contentSkip to conversation

package/util-linux: build programs and libraries in separate packages

RFC PATCH Superseded unixmania at gmail.com 2 participants 2 messages May 9, 2019

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

RFC PATCH v2 1/1 package/util-linux: build programs and libraries in separate packages

From: Carlos Santos <[email protected]>

The findmount and lsblk utilities need udev to work correctly but cannot be built with udev support because the packages providing libudev (eudev and systemd) depend on util-linux, creating a chicken-egg problem. Solve it by means of the following changes:

- Split util-linux into three packages:
- util-linux-libs, providing lib{blkid,fdisk,mount,smartcols,uuid}.
- util-linux-programs, providing both the aforementioned libs and the programs.
- util-linux, a dummy package that drives configuration and building of the other ones.
- Add blind selections for -libs and -programs, i.e. they are indirectly selected according to the util-linux options.
- Make util-linux have build dependencies on util-linux-{libs,programs} if they are selected.
- host-util-linux has a build depencency on either host-util-linux-libs or host-util-linux-programs (not on both, since they are installed on the same destination).
- Make eudev and systemd have build dependencies on util-linux-libs. This can be extended to other packages in the future but is not needed right now because the configuration options are backward-compatible.
- Make util-linux-programs have an optional build dependency on the package that provides libudev (either eudev or systemd), if it is selected.

util-linux-libs is installed on STAGING_DIR by default and on TARGET_DIR if util-linux-programs is not selected. Conversely, util-linux-programs installs on TARGET_DIR by default and on STAGING_DIR if util-linux-libs is not selected. This prevents installing the libraries twice on the same destination, which would confuse check-uniq-files.

With this approach we don't need to patch configuration files neither change other packages besides eudev and systemd. Other packages that require util-linux libraries and whose libraries can be used by util-linux programs can be updated later. We also don't need to change any existing defcconfig, since all configuration options are kept in the dummy util-linux package.

The main drawback of this approach is that util-linux-rebuild, as wel as -reinstall, -reconfigure and even -dirclean targets do not have real effect. It's necessary to use util-linux-libs-rebuild, for instance, but this is a reasonable price to pay for the solution.

---
Changes v1->v2:
- Prevent double installation of the libraries.
- Drop the intrusive patches and AUTORECONF
- Move the the programs to a separate package
- Convert util-linux into a dummy package that drives the other ones.
- Keep configurations and dependencies backward-compatible, except for eudev and systemd.
- Make BR2_PACKAGE_UTIL_LINUX_LIBS a blind option

---
 package/eudev/Config.in                       |   1 +
 package/eudev/eudev.mk                        |   3 +-
 package/systemd/Config.in                     |   1 +
 package/systemd/systemd.mk                    |   4 +-
 package/util-linux/Config.in                  |  59 ++++
 .../util-linux-libs/util-linux-libs.hash      |   1 +
 .../util-linux-libs/util-linux-libs.mk        |  91 ++++++
 ...se-poweroff-if-shutdown-is-not-found.patch |   0
 ...tty-fix-output-of-escaped-characters.patch |   0
 ...rch-fix-obscure-sparc32bash-use-case.patch |   0
 ...alue-of_HAVE_STRUCT_TERMIOS_C_ISPEED.patch |   0
 .../{ => util-linux-programs}/su.pam          |   0
 .../util-linux-programs.hash                  |   1 +
 .../util-linux-programs.mk                    | 261 +++++++++++++++++
 package/util-linux/util-linux.mk              | 265 ++----------------
 15 files changed, 440 insertions(+), 247 deletions(-)
 create mode 120000 package/util-linux/util-linux-libs/util-linux-libs.hash
 create mode 100644 package/util-linux/util-linux-libs/util-linux-libs.mk
 rename package/util-linux/{ => util-linux-programs}/0001-rtcwake-use-poweroff-if-shutdown-is-not-found.patch (100%)
 rename package/util-linux/{ => util-linux-programs}/0002-agetty-fix-output-of-escaped-characters.patch (100%)
 rename package/util-linux/{ => util-linux-programs}/0003-setarch-fix-obscure-sparc32bash-use-case.patch (100%)
 rename package/util-linux/{ => util-linux-programs}/0004-ldattach-Check-for-value-of_HAVE_STRUCT_TERMIOS_C_ISPEED.patch (100%)
 rename package/util-linux/{ => util-linux-programs}/su.pam (100%)
 create mode 120000 package/util-linux/util-linux-programs/util-linux-programs.hash
 create mode 100644 package/util-linux/util-linux-programs/util-linux-programs.mk
package/eudev/Config.in +1 -0
index 2220265a55..2df94fb68c 100644
@@ -6,6 +6,7 @@ config BR2_PACKAGE_EUDEV
 	depends on !BR2_STATIC_LIBS # kmod
 	select BR2_PACKAGE_HAS_UDEV
 	select BR2_PACKAGE_UTIL_LINUX
+	select BR2_PACKAGE_UTIL_LINUX_LIBS
 	select BR2_PACKAGE_UTIL_LINUX_LIBBLKID
 	select BR2_PACKAGE_KMOD
 	help
package/eudev/eudev.mk +2 -1
index d88e010c28..5590049fcc 100644
@@ -18,7 +18,8 @@ EUDEV_CONF_OPTS = \
 	--enable-kmod \
 	--enable-blkid
 
-EUDEV_DEPENDENCIES = host-gperf host-pkgconf util-linux kmod
+# eudev requires only the util-linux libraries at build time
+EUDEV_DEPENDENCIES = host-gperf host-pkgconf util-linux-libs kmod
 EUDEV_PROVIDES = udev
 
 ifeq ($(BR2_ROOTFS_MERGED_USR),)
package/systemd/Config.in +1 -0
index 92ffc7cc5a..3aa5ced46c 100644
@@ -23,6 +23,7 @@ menuconfig BR2_PACKAGE_SYSTEMD
 	select BR2_PACKAGE_DBUS # runtime dependency only
 	select BR2_PACKAGE_LIBCAP
 	select BR2_PACKAGE_UTIL_LINUX
+	select BR2_PACKAGE_UTIL_LINUX_LIBS
 	select BR2_PACKAGE_UTIL_LINUX_LIBBLKID
 	select BR2_PACKAGE_UTIL_LINUX_LIBMOUNT
 	select BR2_PACKAGE_UTIL_LINUX_BINARIES
package/systemd/systemd.mk +3 -1
index edc4a9ecc1..292a97fb40 100644
@@ -9,13 +9,15 @@ SYSTEMD_SITE = $(call github,systemd,systemd,v$(SYSTEMD_VERSION))
 SYSTEMD_LICENSE = LGPL-2.1+, GPL-2.0+ (udev), Public Domain (few source files, see README)
 SYSTEMD_LICENSE_FILES = LICENSE.GPL2 LICENSE.LGPL2.1 README
 SYSTEMD_INSTALL_STAGING = YES
+
+# systemd requires only the util-linux libraries at build time
 SYSTEMD_DEPENDENCIES = \
 	$(if $(BR2_PACKAGE_BASH_COMPLETION),bash-completion) \
 	host-gperf \
 	host-intltool \
 	kmod \
 	libcap \
-	util-linux
+	util-linux-libs
 
 SYSTEMD_PROVIDES = udev
 
package/util-linux/Config.in +59 -0

368 lines not on this page · raw message

package/util-linux/util-linux-libs/util-linux-libs.hash new file +1 -0
new file mode 120000
index 0000000000..dc1b2f866a
@@ -0,0 +1 @@
+../util-linux.hash
\ No newline at end of file
package/util-linux/util-linux-libs/util-linux-libs.mk new file +91 -0

91 lines not on this page · raw message

package/util-linux/util-linux-programs/0001-rtcwake-use-poweroff-if-shutdown-is-not-found.patch renamed from package/util-linux/0001-rtcwake-use-poweroff-if-shutdown-is-not-found.patch +0 -0
similarity index 100%
rename from package/util-linux/0001-rtcwake-use-poweroff-if-shutdown-is-not-found.patch
rename to package/util-linux/util-linux-programs/0001-rtcwake-use-poweroff-if-shutdown-is-not-found.patch
package/util-linux/util-linux-programs/0002-agetty-fix-output-of-escaped-characters.patch renamed from package/util-linux/0002-agetty-fix-output-of-escaped-characters.patch +0 -0
similarity index 100%
rename from package/util-linux/0002-agetty-fix-output-of-escaped-characters.patch
rename to package/util-linux/util-linux-programs/0002-agetty-fix-output-of-escaped-characters.patch
package/util-linux/util-linux-programs/0003-setarch-fix-obscure-sparc32bash-use-case.patch renamed from package/util-linux/0003-setarch-fix-obscure-sparc32bash-use-case.patch +0 -0
similarity index 100%
rename from package/util-linux/0003-setarch-fix-obscure-sparc32bash-use-case.patch
rename to package/util-linux/util-linux-programs/0003-setarch-fix-obscure-sparc32bash-use-case.patch
package/util-linux/util-linux-programs/0004-ldattach-Check-for-value-of_HAVE_STRUCT_TERMIOS_C_ISPEED.patch renamed from package/util-linux/0004-ldattach-Check-for-value-of_HAVE_STRUCT_TERMIOS_C_ISPEED.patch +0 -0
similarity index 100%
rename from package/util-linux/0004-ldattach-Check-for-value-of_HAVE_STRUCT_TERMIOS_C_ISPEED.patch
rename to package/util-linux/util-linux-programs/0004-ldattach-Check-for-value-of_HAVE_STRUCT_TERMIOS_C_ISPEED.patch
package/util-linux/util-linux-programs/su.pam renamed from package/util-linux/su.pam +0 -0
similarity index 100%
rename from package/util-linux/su.pam
rename to package/util-linux/util-linux-programs/su.pam
package/util-linux/util-linux-programs/util-linux-programs.hash new file +1 -0
new file mode 120000
index 0000000000..dc1b2f866a
@@ -0,0 +1 @@
+../util-linux.hash
\ No newline at end of file
package/util-linux/util-linux-programs/util-linux-programs.mk new file +261 -0

261 lines not on this page · raw message

package/util-linux/util-linux.mk +20 -245

268 lines not on this page · raw message

Signature · 1 line
2.20.1
#1 lore raw

RFC PATCH v2 1/1 package/util-linux: build programs and libraries in separate packages

Carlos, All,

On 2019-05-09 11:30 -0300, unixmania at gmail.com spake thusly:

Quoted 18 lines

From: Carlos Santos <[email protected]>

The findmount and lsblk utilities need udev to work correctly but cannot be built with udev support because the packages providing libudev (eudev and systemd) depend on util-linux, creating a chicken-egg problem. Solve it by means of the following changes:

- Split util-linux into three packages:
- util-linux-libs, providing lib{blkid,fdisk,mount,smartcols,uuid}.
- util-linux-programs, providing both the aforementioned libs and the programs.
- util-linux, a dummy package that drives configuration and building of the other ones.
- Add blind selections for -libs and -programs, i.e. they are indirectly selected according to the util-linux options.
- Make util-linux have build dependencies on util-linux-{libs,programs} if they are selected.
- host-util-linux has a build depencency on either host-util-linux-libs

Quoted 8 lines

or host-util-linux-programs (not on both, since they are installed on the same destination).
- Make eudev and systemd have build dependencies on util-linux-libs. This can be extended to other packages in the future but is not needed right now because the configuration options are backward-compatible.
- Make util-linux-programs have an optional build dependency on the package that provides libudev (either eudev or systemd), if it is selected.

To be honest, I was at first very reluctant at lloking into that patch, and your explanations of it make it appear more nasty than it really is. It *is* a sneaky trick you're playing with the install-target/staging conditions, and I am not too happy with it, that's for sure.

But otherwise, it is interesting what you came up with.

I even think you could have gone a bit further, and completely hide the split, so packages would still only have to know about util-linux and never have to worry about the backing -libs and -programs. See below...

Quoted 13 lines

util-linux-libs is installed on STAGING_DIR by default and on TARGET_DIR if util-linux-programs is not selected. Conversely, util-linux-programs installs on TARGET_DIR by default and on STAGING_DIR if util-linux-libs is not selected. This prevents installing the libraries twice on the same destination, which would confuse check-uniq-files.

With this approach we don't need to patch configuration files neither change other packages besides eudev and systemd. Other packages that require util-linux libraries and whose libraries can be used by util-linux programs can be updated later. We also don't need to change any existing defcconfig, since all configuration options are kept in the dummy util-linux package.

Quoted 4 lines

The main drawback of this approach is that util-linux-rebuild, as wel as -reinstall, -reconfigure and even -dirclean targets do not have real effect. It's necessary to use util-linux-libs-rebuild, for instance, but this is a reasonable price to pay for the solution.

I agree that this drawback is not too big of a problem. It is unavoidable, whatever the solution we choose, this one or the previous.

[--SNIP--]

package/eudev/Config.in config BR2_PACKAGE_EUDEV · line 9

Quoted 9 lines
package/eudev/Config.in +1 -0
index 2220265a55..2df94fb68c 100644
@@ -6,6 +6,7 @@ config BR2_PACKAGE_EUDEV
 	depends on !BR2_STATIC_LIBS # kmod
 	select BR2_PACKAGE_HAS_UDEV
 	select BR2_PACKAGE_UTIL_LINUX
+	select BR2_PACKAGE_UTIL_LINUX_LIBS

This select can go away altogether if ...

package/eudev/Config.in config BR2_PACKAGE_EUDEV · line 10

Quoted 1 line
package/eudev/Config.in +0 -0
 	select BR2_PACKAGE_UTIL_LINUX_LIBBLKID

... the BR2_PACKAGE_UTIL_LINUX_LIBBLKID (and the others) do the select, like you did for programs.

I.e. a package that wants a library just has to do:

config BR2_PACKAGE_EUDEV
select BR2_PACKAGE_UTIL_LINUX
select BR2_PACKAGE_UTIL_LINUX_LIBBLKID

and then:

config BR2_PACKAGE_UTIL_LINUX_LIBBLKID
select BR2_PACKAGE_UTIL_LINUX_LIBS

So, the complexity and duality of util-linux.

Quoted 21 lines
package/eudev/Config.in +0 -0
 	select BR2_PACKAGE_KMOD
 	help
package/eudev/eudev.mk +2 -1
index d88e010c28..5590049fcc 100644
@@ -18,7 +18,8 @@ EUDEV_CONF_OPTS = \
 	--enable-kmod \
 	--enable-blkid
 
-EUDEV_DEPENDENCIES = host-gperf host-pkgconf util-linux kmod
+# eudev requires only the util-linux libraries at build time
+EUDEV_DEPENDENCIES = host-gperf host-pkgconf util-linux-libs kmod
 EUDEV_PROVIDES = udev
 
 ifeq ($(BR2_ROOTFS_MERGED_USR),)
package/systemd/Config.in +0 -0
index 92ffc7cc5a..3aa5ced46c 100644

package/systemd/Config.in menuconfig BR2_PACKAGE_SYSTEMD · line 26

Quoted 4 lines
package/systemd/Config.in +1 -0
@@ -23,6 +23,7 @@ menuconfig BR2_PACKAGE_SYSTEMD
 	select BR2_PACKAGE_DBUS # runtime dependency only
 	select BR2_PACKAGE_LIBCAP
 	select BR2_PACKAGE_UTIL_LINUX
+	select BR2_PACKAGE_UTIL_LINUX_LIBS

Ditto as eudev, of course.

Quoted 29 lines
package/systemd/Config.in +0 -0
 	select BR2_PACKAGE_UTIL_LINUX_LIBBLKID
 	select BR2_PACKAGE_UTIL_LINUX_LIBMOUNT
 	select BR2_PACKAGE_UTIL_LINUX_BINARIES
package/systemd/systemd.mk +3 -1
index edc4a9ecc1..292a97fb40 100644
@@ -9,13 +9,15 @@ SYSTEMD_SITE = $(call github,systemd,systemd,v$(SYSTEMD_VERSION))
 SYSTEMD_LICENSE = LGPL-2.1+, GPL-2.0+ (udev), Public Domain (few source files, see README)
 SYSTEMD_LICENSE_FILES = LICENSE.GPL2 LICENSE.LGPL2.1 README
 SYSTEMD_INSTALL_STAGING = YES
+
+# systemd requires only the util-linux libraries at build time
 SYSTEMD_DEPENDENCIES = \
 	$(if $(BR2_PACKAGE_BASH_COMPLETION),bash-completion) \
 	host-gperf \
 	host-intltool \
 	kmod \
 	libcap \
-	util-linux
+	util-linux-libs
 
 SYSTEMD_PROVIDES = udev
 
package/util-linux/Config.in +0 -0
index a5a137bfe0..6fa873ac40 100644

package/util-linux/Config.in menuconfig BR2_PACKAGE_UTIL_LINUX · line 17

Quoted 9 lines
package/util-linux/Config.in +3 -0
@@ -9,6 +9,9 @@ menuconfig BR2_PACKAGE_UTIL_LINUX
 
 if BR2_PACKAGE_UTIL_LINUX
 
+config BR2_PACKAGE_UTIL_LINUX_LIBS
+	bool
+
 config BR2_PACKAGE_UTIL_LINUX_LIBBLKID
 	bool "libblkid"
 	depends on BR2_USE_MMU # fork()

Here you would add the select BR2_PACKAGE_UTIL_LINUX_LIBS (and on the other libs symbols, too).

[--SNIP--]

Quoted 24 lines
package/util-linux/util-linux-libs/util-linux-libs.hash new file +1 -0
new file mode 120000
index 0000000000..dc1b2f866a
@@ -0,0 +1 @@
+../util-linux.hash
\ No newline at end of file
package/util-linux/util-linux-libs/util-linux-libs.mk new file +10 -0
new file mode 100644
index 0000000000..b6af22f2ce
@@ -0,0 +1,91 @@
+################################################################################
+#
+# util-linux-libs
+#
+################################################################################
+
+UTIL_LINUX_LIBS_VERSION = $(UTIL_LINUX_VERSION)
+UTIL_LINUX_LIBS_SOURCE = $(UTIL_LINUX_SOURCE)
+UTIL_LINUX_LIBS_SITE = $(UTIL_LINUX_SITE)
+UTIL_LINUX_LIBS_DL_SUBDIR = $(UTIL_LINUX_DL_SUBDIR)

package/util-linux/util-linux-libs/util-linux-libs.mk line 20

Quoted 10 lines
package/util-linux/util-linux-libs/util-linux-libs.mk new file +10 -0
+
+# README.licensing claims that some files are GPL-2.0 only, but this is not true.
+# Some files are GPL-3.0+ but only in tests. rfkill uses an ISC-style license.
+UTIL_LINUX_LIBS_LICENSE = LGPL-2.1+ (libblkid, libfdisk, libmount, libsmartcols), BSD-3-Clause (libuuid)
+UTIL_LINUX_LIBS_LICENSE_FILES = README.licensing \
+	Documentation/licenses/COPYING.BSD-3-Clause \
+	Documentation/licenses/COPYING.LGPL-2.1-or-later
+UTIL_LINUX_LIBS_INSTALL_STAGING = YES
+# Prevent installing the libraries twice on TARGET_DIR
+UTIL_LINUX_LIBS_INSTALL_TARGET = if($(BR2_PACKAGE_UTIL_LINUX_PROGRAMS),NO,YES)

How can that even work? It's gonna be either 'if(y,NO,YES)' or 'if(,NO,YES)' , but never just 'YES' or just 'NO'.

Note that you have the exact same issue in the -programs case...

[--SNIP--]

Quoted 18 lines
package/util-linux/util-linux-programs/util-linux-programs.mk new file +12 -0
new file mode 100644
index 0000000000..e215fe7187
@@ -0,0 +1,261 @@
+################################################################################
+#
+# util-linux-programs
+#
+################################################################################
+
+UTIL_LINUX_PROGRAMS_VERSION = $(UTIL_LINUX_VERSION)
+UTIL_LINUX_PROGRAMS_SOURCE = $(UTIL_LINUX_SOURCE)
+UTIL_LINUX_PROGRAMS_SITE = $(UTIL_LINUX_SITE)
+UTIL_LINUX_PROGRAMS_DL_SUBDIR = $(UTIL_LINUX_DL_SUBDIR)
+
+# README.licensing claims that some files are GPL-2.0 only, but this is not true.

package/util-linux/util-linux-programs/util-linux-programs.mk line 22

Quoted 10 lines
package/util-linux/util-linux-programs/util-linux-programs.mk new file +10 -0
+# Some files are GPL-3.0+ but only in tests. rfkill uses an ISC-style license.
+UTIL_LINUX_PROGRAMS_LICENSE = GPL-2.0+, BSD-4-Clause, LGPL-2.1+ (libblkid, libfdisk, libmount, libsmartcols), BSD-3-Clause (libuuid) ISC (rfkill)
+UTIL_LINUX_PROGRAMS_LICENSE_FILES = README.licensing \
+	Documentation/licenses/COPYING.BSD-3-Clause \
+	Documentation/licenses/COPYING.BSD-4-Clause-UC \
+	Documentation/licenses/COPYING.GPL-2.0-or-later \
+	Documentation/licenses/COPYING.ISC \
+	Documentation/licenses/COPYING.LGPL-2.1-or-later
+# Prevent installing the libraries twice on STAGING_DIR
+UTIL_LINUX_PROGRAMS_INSTALL_STAGING = if($(BR2_PACKAGE_UTIL_LINUX_LIBS),NO,YES)

... here.

So, this is otherwise a big patch, and I obviously did not review it in its entirety.

But otherwise it looks OK-ish, even if I am still not entirely happy with this trick...

Still, I'd like we get some feedback from upstream about a way to actually split the package. Did you get any? I tried looking at their github tracker, and could not see any relevant activity...

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'