Skip to contentSkip to conversation

package/zfs: fix cross-compilation

PATCH In review Michael Zimmermann 6 participants 7 messages Jul 20, 2022 – Jul 27, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 package/zfs: fix cross-compilation

Currently, the kernel module gets built using the host-compiler(from the user distro, not from buildroot). That might work when target and host arch are the same but during cross compilation that results in the message Unable to build an empty module. because tests for compiling modules fail due to missing headers.

---
 package/zfs/zfs.mk | 3 +++
 1 file changed, 3 insertions(+)
package/zfs/zfs.mk +3 -0
index 07ce3295ec..30f966bdeb 100644
@@ -57,6 +57,9 @@ else
 ZFS_CONF_OPTS += --disable-pam
 endif
 
+ZFS_CONF_ENV += $(LINUX_MAKE_FLAGS)
+ZFS_MAKE_ENV += $(LINUX_MAKE_FLAGS)
+
 # ZFS userland tools are unfunctional without the Linux kernel modules.
 ZFS_MODULE_SUBDIRS = \
 	module/avl \
Signature · 6 lines
2.37.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw
Quoted 2 lines · Michael Zimmermann

Currently, the kernel module gets built using the host-compiler(from the user distro, not from buildroot).

That shouldn't happen. The kernel-module infrastructure passes
LINUX_MAKE_FLAGS in the environment already. And if the module were built during the userspace build stage, that one already has the cross-compiler as well through the autotools infrastructure.

Can you give a bit more detail about the config where you get the issue, and
how exactly it fails?

Regards,
Arnout

Quoted 15 lines

That might work when target and host
arch are the same but during cross compilation that results in the message Unable to build an empty module. because tests for compiling modules fail due to missing headers.

---
  package/zfs/zfs.mk | 3 +++
  1 file changed, 3 insertions(+)
package/zfs/zfs.mk +0 -0
index 07ce3295ec..30f966bdeb 100644

package/zfs/zfs.mk else · line 65

Quoted 9 lines
package/zfs/zfs.mk +3 -0
@@ -57,6 +57,9 @@ else
  ZFS_CONF_OPTS += --disable-pam
  endif
 
+ZFS_CONF_ENV += $(LINUX_MAKE_FLAGS)
+ZFS_MAKE_ENV += $(LINUX_MAKE_FLAGS)
+
  # ZFS userland tools are unfunctional without the Linux kernel modules.
  ZFS_MODULE_SUBDIRS = \
  	module/avl \

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#2 lore raw
Quoted 9 lines · Arnout Vandecappelle
Quoted 2 lines · Michael Zimmermann

Currently, the kernel module gets built using the host-compiler(from the user distro, not from buildroot).

 That shouldn't happen. The kernel-module infrastructure passes LINUX_MAKE_FLAGS in the environment already. And if the module were built during the userspace build stage, that one already has the cross-compiler as well through the autotools infrastructure.

 Can you give a bit more detail about the config where you get the issue, and how exactly it fails?

 It's actually easy to reproduce, just change the architecture to anything other than x86.

 I've looked around a bit and it seems to me that zfs already builds the kernel module from the autotools infrastructure, so we're actually building it twice. I think we should therefore remove the kernel-module infra for this package, and instead add a dependency on linux and copy this part:

# If the package is enabled, ensure the kernel will support modules ifeq ($$(BR2_PACKAGE_$(2)),y)
LINUX_NEEDS_MODULES = y
endif

 Regards,
 Arnout

Quoted 14 lines

 Regards,
 Arnout

Quoted 10 lines

That might work when target and host
arch are the same but during cross compilation that results in the message Unable to build an empty module. because tests for compiling modules fail due to missing headers.

---
  package/zfs/zfs.mk | 3 +++

  1 file changed, 3 insertions(+)
Quoted 13 lines
Quoted 10 lines
package/zfs/zfs.mk +0 -0
index 07ce3295ec..30f966bdeb 100644
@@ -57,6 +57,9 @@ else

  ZFS_CONF_OPTS += --disable-pam
  endif
  +ZFS_CONF_ENV += $(LINUX_MAKE_FLAGS)

package/zfs/zfs.mk +2 -0
+ZFS_MAKE_ENV += $(LINUX_MAKE_FLAGS)
+
Quoted 3 lines

  # ZFS userland tools are unfunctional without the Linux kernel modules. ZFS_MODULE_SUBDIRS = \
      module/avl \

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#3 lore raw

Michael, All,

On 2022-07-23 15:41 +0200, Arnout Vandecappelle spake thusly:

Quoted 10 lines · Arnout Vandecappelle
Quoted 3 lines · Michael Zimmermann
Quoted 2 lines

Currently, the kernel module gets built using the host-compiler(from the user distro, not from buildroot).

 It's actually easy to reproduce, just change the architecture to anything other than x86.

 I've looked around a bit and it seems to me that zfs already builds the kernel module from the autotools infrastructure, so we're actually building it twice.

I checked, and it is not possible to tell the zfs buildsystem to not build it.

Quoted 7 lines

I think we should therefore remove the kernel-module infra for this package, and instead add a dependency on linux and copy this part:

# If the package is enabled, ensure the kernel will support modules ifeq ($$(BR2_PACKAGE_$(2)),y)
LINUX_NEEDS_MODULES = y
endif

... and add linux to ZFS_DEPDENDENCIES

So, I've marked the patch as changes requested in patchwork.

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#4 lore raw

Yann, All,

Noticed the same issue on x86-64 without cross-compiling. Here again the host compiler was used instead of toolchain. The build failed because the host compiler was older and did not support some fancy flags.

Host: Debian 11.7 x86_64
Host gcc: 10.2.1
Toolchain: x86-64--glibc--bleeding-edge-2022.08-1
Toolchain gcc: 12.2.0

The patch proposed by Michael Zimmermann fixed the issue.

Signature · 1 line
Alexander.
Quoted 25 lines · Yann E. MORIN

Michael, All,

On 2022-07-23 15:41 +0200, Arnout Vandecappelle spake thusly:

Quoted 10 lines · Arnout Vandecappelle
Quoted 3 lines · Michael Zimmermann
Quoted 2 lines

Currently, the kernel module gets built using the host-compiler(from the user distro, not from buildroot).

 It's actually easy to reproduce, just change the architecture to anything other than x86.

 I've looked around a bit and it seems to me that zfs already builds the kernel module from the autotools infrastructure, so we're actually building it twice.

I checked, and it is not possible to tell the zfs buildsystem to not build it.

Quoted 7 lines

I think we should therefore remove the kernel-module infra for this package, and instead add a dependency on linux and copy this part:

# If the package is enabled, ensure the kernel will support modules ifeq ($$(BR2_PACKAGE_$(2)),y)
LINUX_NEEDS_MODULES = y
endif

Quoted 18 lines

... and add linux to ZFS_DEPDENDENCIES

So, I've marked the patch as changes requested in patchwork.

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#5 lore raw

PATCH 1/1 package/zfs: not using kernel-module infra

In reply to Yann E. MORIN

zfs already builds the kernel module from the autotools infrastructure.

---
 package/zfs/Config.in | 1 +
 package/zfs/zfs.mk    | 5 +++--
 2 files changed, 4 insertions(+), 2 deletions(-)
package/zfs/Config.in +1 -0
index 6209c5865b..3bd9f8ab02 100644
@@ -11,6 +11,7 @@ config BR2_PACKAGE_ZFS
 	depends on BR2_PACKAGE_HAS_UDEV
 	depends on BR2_USE_MMU  # util-linux (libblkid)
 	depends on BR2_TOOLCHAIN_HAS_THREADS  # libtirpc
+	select BR2_LINUX_NEEDS_MODULES # not using kernel-module infra
 	select BR2_PACKAGE_UTIL_LINUX
 	select BR2_PACKAGE_UTIL_LINUX_LIBBLKID
 	select BR2_PACKAGE_UTIL_LINUX_LIBUUID
package/zfs/zfs.mk +3 -2
index e99413e611..50c1866045 100644
@@ -15,7 +15,7 @@ ZFS_CPE_ID_PRODUCT = openzfs
 # 0001-removal-of-LegacyVersion-broke-ax_python_dev.m4.patch
 ZFS_AUTORECONF = YES
 
-ZFS_DEPENDENCIES = libaio openssl udev util-linux zlib libcurl
+ZFS_DEPENDENCIES = libaio openssl udev util-linux zlib libcurl linux
 
 # sysvinit installs only a commented-out modules-load.d/ config file
 ZFS_CONF_OPTS = \
@@ -87,5 +87,6 @@ define ZFS_LINUX_CONFIG_FIXUPS
 	$(call KCONFIG_ENABLE_OPT,CONFIG_ZLIB_INFLATE)
 endef
 
-$(eval $(kernel-module))
+# ZFS autotools will compile the kernel modules, so there is not needed to execute
+# $(eval $(kernel-module))
 $(eval $(autotools-package))
Signature · 6 lines
2.41.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#6 lore raw

Re: [Buildroot] [PATCH] package/zfs: not using kernel-module infra

Quoted 7 lines · On Thu, 8 Jun 2023 19:50:16 +0200 José Luis Salvador Rufo…

zfs already builds the kernel module from the autotools infrastructure.

---
 package/zfs/Config.in | 1 +
 package/zfs/zfs.mk    | 5 +++--
 2 files changed, 4 insertions(+), 2 deletions(-)

Applied to master, thanks.

Thomas

Signature · 7 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot