Skip to contentSkip to conversation

btrfs-progs related improvements

PATCH New Thomas Petazzoni 2 participants 12 messages Aug 21, 2018 – Aug 22, 2018

lore
Share & tools
lore
#0 lore raw

cover btrfs-progs related improvements

Hello,

Following the addition of host-btrfs-progs recently by Robert J. Heywood, I wondered why e2fsprogs was needed as a dependency. Indeed, why would btrfs need the library of a completely different filesystem?

Turns out it's because btrfs-progs can optionally build a tool called btrfs-convert, to convert an ext2 or reiserfs filesystem to a btrfs one, and this tool needs e2fsprogs as a dependency to support ext2 filesystem conversion. But this is completely optional.

For the host variant, I don't think it makes sense to support that at all, so we forcibly disable building btrfs-convert.

For the target variant, e2fsprogs is turned into an automatic optional
dependency: when available, btrfs-convert is compiled with ext2 support, otherwise btrfs-convert is not built at all. We don't take into account reiserfs support because we don't have the corresponding libraries packaged in Buildroot.

While doing this work on the host variant, I noticed the e2fsprogs and lzo host libraries were only available as static libraries. So I went ahead and fixed that as well.

Best regards,

Thomas

Thomas Petazzoni (4):
lzo: build shared library for the host
e2fsprogs: build shared library for the host
btrfs-progs: drop dependency on e2fsprogs by not building
btrfs-convert
btrfs-progs: make e2fsprogs an optional dependency

 package/btrfs-progs/Config.in      |  1 -
 package/btrfs-progs/btrfs-progs.mk | 20 +++++++++++++++++---
 package/e2fsprogs/e2fsprogs.mk     |  3 ++-
 package/lzo/lzo.mk                 |  2 ++
 4 files changed, 21 insertions(+), 5 deletions(-)
Signature · 1 line
2.14.4
#1 lore raw

PATCH 1/4 lzo: build shared library for the host

By default, the lzo package builds and installs only a static library. For the target variant, we pass the appropriate ENABLE_STATIC/ENABLE_SHARED options, but not for the host package, and therefore a static lzo library gets installed.

However, on the host, our policy is to build shared libraries and not static libraries, as visible in the default configure options passed to host package in pkg-autotools.mk (--enable-shared --disable-static). Let's do the same with lzo.

---
 package/lzo/lzo.mk | 2 ++
 1 file changed, 2 insertions(+)
package/lzo/lzo.mk +2 -0
index 27ca459acf..0ead92725a 100644
@@ -23,5 +23,7 @@ else
 LZO_CONF_OPTS += -DENABLE_STATIC=OFF
 endif
 
+HOST_LZO_CONF_OPTS += -DENABLE_SHARED=ON -DENABLE_STATIC=OFF
+
 $(eval $(cmake-package))
 $(eval $(host-cmake-package))
Signature · 1 line
2.14.4
#2 lore raw

PATCH 1/4 lzo: build shared library for the host

Thomas, All,

On 2018-08-21 23:59 +0200, Thomas Petazzoni spake thusly:

Quoted 11 lines

By default, the lzo package builds and installs only a static library. For the target variant, we pass the appropriate ENABLE_STATIC/ENABLE_SHARED options, but not for the host package, and therefore a static lzo library gets installed.

However, on the host, our policy is to build shared libraries and not static libraries, as visible in the default configure options passed to host package in pkg-autotools.mk (--enable-shared --disable-static). Let's do the same with lzo.

Regards,
Yann E. MORIN.

Quoted 23 lines
---
 package/lzo/lzo.mk | 2 ++
 1 file changed, 2 insertions(+)
package/lzo/lzo.mk +2 -0
index 27ca459acf..0ead92725a 100644
@@ -23,5 +23,7 @@ else
 LZO_CONF_OPTS += -DENABLE_STATIC=OFF
 endif
 
+HOST_LZO_CONF_OPTS += -DENABLE_SHARED=ON -DENABLE_STATIC=OFF
+
 $(eval $(cmake-package))
 $(eval $(host-cmake-package))
Signature · 6 lines
2.14.4

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#3 lore raw

PATCH 2/4 e2fsprogs: build shared library for the host

In reply to Thomas Petazzoni

By default, the e2fsprogs package builds and installs only static libraries, unless --enable-elf-shlibs is passed. For the target variant, we pass the appropriate --enable-elf-shlibs/--disable-elf-shlibs options, but not for the host package, and therefore static e2fsprogs libraries get installed.

However, on the host, our policy is to build shared libraries and not static libraries, as visible in the default configure options passed to host package in pkg-autotools.mk (--enable-shared --disable-static). Let's do the same with e2fsprogs.

---
 package/e2fsprogs/e2fsprogs.mk | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
package/e2fsprogs/e2fsprogs.mk +2 -1
index efc10d0e99..65f9774afb 100644
@@ -27,7 +27,8 @@ HOST_E2FSPROGS_CONF_OPTS = \
 	--disable-libblkid \
 	--disable-libuuid \
 	--enable-symlink-install \
-	--disable-testio-debug
+	--disable-testio-debug \
+	--enable-elf-shlibs
 
 # Set the binary directories to "/bin" and "/sbin", as busybox does,
 # so that we do not end up with two versions of e2fs tools.
Signature · 1 line
2.14.4
#4 lore raw

PATCH 2/4 e2fsprogs: build shared library for the host

Thomas, All,

On 2018-08-21 23:59 +0200, Thomas Petazzoni spake thusly:

Quoted 21 lines

By default, the e2fsprogs package builds and installs only static libraries, unless --enable-elf-shlibs is passed. For the target variant, we pass the appropriate --enable-elf-shlibs/--disable-elf-shlibs options, but not for the host package, and therefore static e2fsprogs libraries get installed.

However, on the host, our policy is to build shared libraries and not static libraries, as visible in the default configure options passed to host package in pkg-autotools.mk (--enable-shared --disable-static). Let's do the same with e2fsprogs.

---
 package/e2fsprogs/e2fsprogs.mk | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
package/e2fsprogs/e2fsprogs.mk +0 -0
index efc10d0e99..65f9774afb 100644

package/e2fsprogs/e2fsprogs.mk HOST_E2FSPROGS_CONF_OPTS = \ · line 31

Quoted 6 lines
package/e2fsprogs/e2fsprogs.mk +2 -1
@@ -27,7 +27,8 @@ HOST_E2FSPROGS_CONF_OPTS = \
 	--disable-libblkid \
 	--disable-libuuid \
 	--enable-symlink-install \
-	--disable-testio-debug
+	--disable-testio-debug \
+	--enable-elf-shlibs

I lknow this is not your fault, but I find it more readable when the --disable are all grouped together, and the --enable ones as well.

It also helps spot potentialy issues when the ordering is the same btw host and target variants.

Regards,
Yann E. MORIN.

Quoted 9 lines
package/e2fsprogs/e2fsprogs.mk +0 -0
 # Set the binary directories to "/bin" and "/sbin", as busybox does,
 # so that we do not end up with two versions of e2fs tools.
Signature · 6 lines
2.14.4

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#5 lore raw

PATCH 2/4 e2fsprogs: build shared library for the host

Hello,

Quoted 5 lines · Yann E. MORIN

I lknow this is not your fault, but I find it more readable when the --disable are all grouped together, and the --enable ones as well.

It also helps spot potentialy issues when the ordering is the same btw host and target variants.

I have pushed an additional commit fixing this for the host configure options.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#6 lore raw

PATCH 3/4 btrfs-progs: drop dependency on e2fsprogs by not building btrfs-convert

In reply to Thomas Petazzoni

The btrfs-convert tool is used to convert an existing ext2 or reiserfs filesystem into a btrfs filesystem. On the host, this is not really useful, so let's disable building this tool, which allows to drop the host-e2fsprogs dependency.

The host-util-linux dependency becomes necessary: it was previously brought as a second-order dependency of host-e2fsprogs, but since we no longer depend on host-e2fsprogs, we now need to explicitly depend on host-util-linux.

---
 package/btrfs-progs/btrfs-progs.mk | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
package/btrfs-progs/btrfs-progs.mk +6 -2
index 3129ecc714..2f9f4127b6 100644
@@ -21,8 +21,12 @@ BTRFS_PROGS_INSTALL_TARGET_OPTS = DESTDIR=$(TARGET_DIR) install-static
 BTRFS_PROGS_INSTALL_STAGING_OPTS = DESTDIR=$(STAGING_DIR) install-static
 endif
 
-HOST_BTRFS_PROGS_DEPENDENCIES = host-e2fsprogs host-lzo host-zlib
-HOST_BTRFS_PROGS_CONF_OPTS = --disable-backtrace --disable-zstd --disable-python
+HOST_BTRFS_PROGS_DEPENDENCIES = host-util-linux host-lzo host-zlib
+HOST_BTRFS_PROGS_CONF_OPTS = \
+	--disable-backtrace \
+	--disable-zstd \
+	--disable-python \
+	--disable-convert
 
 $(eval $(autotools-package))
 $(eval $(host-autotools-package))
Signature · 1 line
2.14.4
#7 lore raw

PATCH 3/4 btrfs-progs: drop dependency on e2fsprogs by not building btrfs-convert

Thomas, All,

On 2018-08-21 23:59 +0200, Thomas Petazzoni spake thusly:

Quoted 11 lines

The btrfs-convert tool is used to convert an existing ext2 or reiserfs filesystem into a btrfs filesystem. On the host, this is not really useful, so let's disable building this tool, which allows to drop the host-e2fsprogs dependency.

The host-util-linux dependency becomes necessary: it was previously brought as a second-order dependency of host-e2fsprogs, but since we no longer depend on host-e2fsprogs, we now need to explicitly depend on host-util-linux.

Regards,
Yann E. MORIN.

Quoted 30 lines
---
 package/btrfs-progs/btrfs-progs.mk | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
package/btrfs-progs/btrfs-progs.mk +6 -2
index 3129ecc714..2f9f4127b6 100644
@@ -21,8 +21,12 @@ BTRFS_PROGS_INSTALL_TARGET_OPTS = DESTDIR=$(TARGET_DIR) install-static
 BTRFS_PROGS_INSTALL_STAGING_OPTS = DESTDIR=$(STAGING_DIR) install-static
 endif
 
-HOST_BTRFS_PROGS_DEPENDENCIES = host-e2fsprogs host-lzo host-zlib
-HOST_BTRFS_PROGS_CONF_OPTS = --disable-backtrace --disable-zstd --disable-python
+HOST_BTRFS_PROGS_DEPENDENCIES = host-util-linux host-lzo host-zlib
+HOST_BTRFS_PROGS_CONF_OPTS = \
+	--disable-backtrace \
+	--disable-zstd \
+	--disable-python \
+	--disable-convert
 
 $(eval $(autotools-package))
 $(eval $(host-autotools-package))
Signature · 6 lines
2.14.4

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#8 lore raw

PATCH 4/4 btrfs-progs: make e2fsprogs an optional dependency

In reply to Thomas Petazzoni

e2fsprogs is only needed to build the btrfs-convert program, that allows to convert an existing ext2 filesystem into a btrfs filesystem. Not everybody needs to do that and making this dependency optional is nicer, so this is what this patch does.

Note that btrfs-progs also supports converting from reiserfs, which is why the --with-convert option supports a list of filesystems. Since Buildroot has no package for the reiserfs library, we for now only support the ext2 case, with e2fsprogs as a dependency.

---
 package/btrfs-progs/Config.in      |  1 -
 package/btrfs-progs/btrfs-progs.mk | 12 +++++++++++-
 2 files changed, 11 insertions(+), 2 deletions(-)
package/btrfs-progs/Config.in +0 -1
index ad86bab580..35dfc39193 100644
@@ -2,7 +2,6 @@ config BR2_PACKAGE_BTRFS_PROGS
 	bool "btrfs-progs"
 	depends on BR2_USE_MMU # util-linux
 	depends on BR2_TOOLCHAIN_HAS_THREADS
-	select BR2_PACKAGE_E2FSPROGS
 	select BR2_PACKAGE_LZO
 	select BR2_PACKAGE_UTIL_LINUX
 	select BR2_PACKAGE_UTIL_LINUX_LIBBLKID
package/btrfs-progs/btrfs-progs.mk +11 -1
index 2f9f4127b6..33a6be324f 100644
@@ -7,7 +7,7 @@
 BTRFS_PROGS_VERSION = 4.16.1
 BTRFS_PROGS_SITE = $(BR2_KERNEL_MIRROR)/linux/kernel/people/kdave/btrfs-progs
 BTRFS_PROGS_SOURCE = btrfs-progs-v$(BTRFS_PROGS_VERSION).tar.xz
-BTRFS_PROGS_DEPENDENCIES = host-pkgconf e2fsprogs lzo util-linux zlib
+BTRFS_PROGS_DEPENDENCIES = host-pkgconf lzo util-linux zlib
 BTRFS_PROGS_CONF_OPTS = --disable-backtrace --disable-zstd --disable-python
 BTRFS_PROGS_LICENSE = GPL-2.0, LGPL-3.0+ (libbtrfsutil)
 BTRFS_PROGS_LICENSE_FILES = COPYING libbtrfsutil/COPYING \
@@ -21,6 +21,16 @@ BTRFS_PROGS_INSTALL_TARGET_OPTS = DESTDIR=$(TARGET_DIR) install-static
 BTRFS_PROGS_INSTALL_STAGING_OPTS = DESTDIR=$(STAGING_DIR) install-static
 endif
 
+# convert also supports conversion from reiserfs, which needs some
+# reiserfs libraries, but we have no package for them in Buildroot, so
+# we keep things simple and only handle ext2.
+ifeq ($(BR2_PACKAGE_E2FSPROGS),y)
+BTRFS_PROGS_CONF_OPTS += --enable-convert --with-convert=ext2
+BTRFS_PROGS_DEPENDENCIES += e2fsprogs
+else
+BTRFS_PROGS_CONF_OPTS += --disable-convert
+endif
+
 HOST_BTRFS_PROGS_DEPENDENCIES = host-util-linux host-lzo host-zlib
 HOST_BTRFS_PROGS_CONF_OPTS = \
 	--disable-backtrace \
Signature · 1 line
2.14.4
#9 lore raw

PATCH 4/4 btrfs-progs: make e2fsprogs an optional dependency

Thomas, All,

On 2018-08-21 23:59 +0200, Thomas Petazzoni spake thusly:

Quoted 11 lines

e2fsprogs is only needed to build the btrfs-convert program, that allows to convert an existing ext2 filesystem into a btrfs filesystem. Not everybody needs to do that and making this dependency optional is nicer, so this is what this patch does.

Note that btrfs-progs also supports converting from reiserfs, which is why the --with-convert option supports a list of filesystems. Since Buildroot has no package for the reiserfs library, we for now only support the ext2 case, with e2fsprogs as a dependency.

Regards,
Yann E. MORIN.

Quoted 54 lines
---
 package/btrfs-progs/Config.in      |  1 -
 package/btrfs-progs/btrfs-progs.mk | 12 +++++++++++-
 2 files changed, 11 insertions(+), 2 deletions(-)
package/btrfs-progs/Config.in +0 -1
index ad86bab580..35dfc39193 100644
@@ -2,7 +2,6 @@ config BR2_PACKAGE_BTRFS_PROGS
 	bool "btrfs-progs"
 	depends on BR2_USE_MMU # util-linux
 	depends on BR2_TOOLCHAIN_HAS_THREADS
-	select BR2_PACKAGE_E2FSPROGS
 	select BR2_PACKAGE_LZO
 	select BR2_PACKAGE_UTIL_LINUX
 	select BR2_PACKAGE_UTIL_LINUX_LIBBLKID
package/btrfs-progs/btrfs-progs.mk +11 -1
index 2f9f4127b6..33a6be324f 100644
@@ -7,7 +7,7 @@
 BTRFS_PROGS_VERSION = 4.16.1
 BTRFS_PROGS_SITE = $(BR2_KERNEL_MIRROR)/linux/kernel/people/kdave/btrfs-progs
 BTRFS_PROGS_SOURCE = btrfs-progs-v$(BTRFS_PROGS_VERSION).tar.xz
-BTRFS_PROGS_DEPENDENCIES = host-pkgconf e2fsprogs lzo util-linux zlib
+BTRFS_PROGS_DEPENDENCIES = host-pkgconf lzo util-linux zlib
 BTRFS_PROGS_CONF_OPTS = --disable-backtrace --disable-zstd --disable-python
 BTRFS_PROGS_LICENSE = GPL-2.0, LGPL-3.0+ (libbtrfsutil)
 BTRFS_PROGS_LICENSE_FILES = COPYING libbtrfsutil/COPYING \
@@ -21,6 +21,16 @@ BTRFS_PROGS_INSTALL_TARGET_OPTS = DESTDIR=$(TARGET_DIR) install-static
 BTRFS_PROGS_INSTALL_STAGING_OPTS = DESTDIR=$(STAGING_DIR) install-static
 endif
 
+# convert also supports conversion from reiserfs, which needs some
+# reiserfs libraries, but we have no package for them in Buildroot, so
+# we keep things simple and only handle ext2.
+ifeq ($(BR2_PACKAGE_E2FSPROGS),y)
+BTRFS_PROGS_CONF_OPTS += --enable-convert --with-convert=ext2
+BTRFS_PROGS_DEPENDENCIES += e2fsprogs
+else
+BTRFS_PROGS_CONF_OPTS += --disable-convert
+endif
+
 HOST_BTRFS_PROGS_DEPENDENCIES = host-util-linux host-lzo host-zlib
 HOST_BTRFS_PROGS_CONF_OPTS = \
 	--disable-backtrace \
Signature · 6 lines
2.14.4

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#10 lore raw

cover btrfs-progs related improvements

In reply to Thomas Petazzoni

Thomas, All,

On 2018-08-21 23:59 +0200, Thomas Petazzoni spake thusly:

Quoted 19 lines

Following the addition of host-btrfs-progs recently by Robert J. Heywood, I wondered why e2fsprogs was needed as a dependency. Indeed, why would btrfs need the library of a completely different filesystem?

Turns out it's because btrfs-progs can optionally build a tool called btrfs-convert, to convert an ext2 or reiserfs filesystem to a btrfs one, and this tool needs e2fsprogs as a dependency to support ext2 filesystem conversion. But this is completely optional.

For the host variant, I don't think it makes sense to support that at all, so we forcibly disable building btrfs-convert.

For the target variant, e2fsprogs is turned into an automatic optional
dependency: when available, btrfs-convert is compiled with ext2 support, otherwise btrfs-convert is not built at all. We don't take into account reiserfs support because we don't have the corresponding libraries packaged in Buildroot.

Quoted 3 lines

While doing this work on the host variant, I noticed the e2fsprogs and lzo host libraries were only available as static libraries. So I went ahead and fixed that as well.

There is one thing you forgot to do: enable support for zstd, now that we have a package for it. ;-p

Regards,
Yann E. MORIN.

Quoted 24 lines

Best regards,

Thomas

Thomas Petazzoni (4):
lzo: build shared library for the host
e2fsprogs: build shared library for the host
btrfs-progs: drop dependency on e2fsprogs by not building
btrfs-convert
btrfs-progs: make e2fsprogs an optional dependency

 package/btrfs-progs/Config.in      |  1 -
 package/btrfs-progs/btrfs-progs.mk | 20 +++++++++++++++++---
 package/e2fsprogs/e2fsprogs.mk     |  3 ++-
 package/lzo/lzo.mk                 |  2 ++
 4 files changed, 21 insertions(+), 5 deletions(-)
Signature · 6 lines
2.14.4

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#11 lore raw

cover btrfs-progs related improvements

In reply to Thomas Petazzoni

Hello,

Quoted 6 lines · Thomas Petazzoni

Thomas Petazzoni (4):
lzo: build shared library for the host
e2fsprogs: build shared library for the host
btrfs-progs: drop dependency on e2fsprogs by not building
btrfs-convert
btrfs-progs: make e2fsprogs an optional dependency

All applied to next. Thanks Yann for the review!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com