Skip to contentSkip to conversation

package/kmod: add support for compressed modules (branch yem/host-kmod-z)

PATCH New Yann E. MORIN 4 participants 11 messages May 3, 2020 – Oct 6, 2020

lore
Share & tools
lore
#0 lore raw

v2 cover package/kmod: add support for compressed modules (branch yem/host-kmod-z)

Hello All!

This series adds conditional compression support to the host-kmod, so that we correctly handle compressed kernel modules.

Regards,
Yann E. MORIN.

The following changes since commit de70d9b933c23b88b7b6f07561a62fd813274e9c

package/ne10: drop arch-related comment (2020-05-03 10:07:05 +0200)

are available in the git repository at:

git://git.buildroot.org/~ymorin/git/buildroot.git

for you to fetch changes up to d66d07fe82ae1c515eb9a010f1df2cbcc10f1a87

package/lmod: add compression support for the host variant (2020-05-03 15:06:48 +0200)

----------------------------------------------------------------
Yann E. MORIN (3):
package/kmod: explicitly disable compression when not enabled package/kmod: add option for host variant package/lmod: add compression support for the host variant

 linux/Config.in             |  1 +
 package/Config.in.host      |  1 +
 package/kmod/Config.in.host | 16 ++++++++++++++++
 package/kmod/kmod.mk        | 18 ++++++++++++++++++
 4 files changed, 36 insertions(+)
 create mode 100644 package/kmod/Config.in.host
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#1 lore raw

PATCH v2 1/3 package/kmod: explicitly disable compression when not enabled

---
 package/kmod/kmod.mk | 4 ++++
 1 file changed, 4 insertions(+)
package/kmod/kmod.mk +4 -0
index e2dfea5c7b..cdbfc5a469 100644
@@ -35,11 +35,15 @@ endif
 ifeq ($(BR2_PACKAGE_ZLIB),y)
 KMOD_DEPENDENCIES += zlib
 KMOD_CONF_OPTS += --with-zlib
+else
+KMOD_CONF_OPTS += --without-zlib
 endif
 
 ifeq ($(BR2_PACKAGE_XZ),y)
 KMOD_DEPENDENCIES += xz
 KMOD_CONF_OPTS += --with-xz
+else
+KMOD_CONF_OPTS += --without-xz
 endif
 
 ifeq ($(BR2_PACKAGE_PYTHON)$(BR2_PACKAGE_PYTHON3),y)
Signature · 1 line
2.20.1
#2 lore raw

PATCH v2 1/3 package/kmod: explicitly disable compression when not enabled

Yann,

Quoted 3 lines · Yann E. MORIN

Tested support for a compressed and signed modules (found [1] was needed to allow modinfo to display signed module details)

BR2_aarch64=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_VERSION=y BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.16.7" BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/aarch64-virt/linux.config" BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y BR2_PACKAGE_BUSYBOX_SHOW_OTHERS=y
BR2_PACKAGE_XZ=y
BR2_PACKAGE_LIBZIP=y
BR2_PACKAGE_OPENSSL=y # modinfo requires so it can display signature info
BR2_PACKAGE_KMOD=y
BR2_PACKAGE_KMOD_TOOLS=y
BR2_TARGET_ROOTFS_EXT2=y
# BR2_TARGET_ROOTFS_TAR is not set
BR2_PACKAGE_HOST_KMOD_GZ=y
BR2_PACKAGE_HOST_KMOD_XZ=y

Linux Kernel options enabled

       CONFIG_MODULES=y
       CONFIG_MODULE_UNLOAD=y
       CONFIG_MODVERSIONS=y
       CONFIG_MODULE_SRCVERSION_ALL=y
       CONFIG_MODULE_SIG=y
       CONFIG_MODULE_SIG_FORCE=y
       CONFIG_MODULE_SIG_SHA256=y
       CONFIG_MODULE_COMPRESS=y
       CONFIG_MODULE_COMPRESS_XZ=y

[1] http://patchwork.ozlabs.org/project/buildroot/patch/20200514182806.56592-1-matthew.weber at rockwellcollins.com/

#3 lore raw

PATCH v2 1/3 package/kmod: explicitly disable compression when not enabled

In reply to Yann E. MORIN

Kris,

Quoted 30 lines · Yann E. MORIN
---
 package/kmod/kmod.mk | 4 ++++
 1 file changed, 4 insertions(+)
package/kmod/kmod.mk +4 -1
index e2dfea5c7b..cdbfc5a469 100644
@@ -35,11 +35,15 @@ endif
 ifeq ($(BR2_PACKAGE_ZLIB),y)
 KMOD_DEPENDENCIES += zlib
 KMOD_CONF_OPTS += --with-zlib
+else
+KMOD_CONF_OPTS += --without-zlib
 endif
 
 ifeq ($(BR2_PACKAGE_XZ),y)
 KMOD_DEPENDENCIES += xz
 KMOD_CONF_OPTS += --with-xz
+else
+KMOD_CONF_OPTS += --without-xz
 endif
 
 ifeq ($(BR2_PACKAGE_PYTHON)$(BR2_PACKAGE_PYTHON3),y)
--

2.20.1

Quoted 4 lines

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Reflecting this for your review

#4 lore raw

PATCH v2 1/3 package/kmod: explicitly disable compression when not enabled

Yann,

Quoted 3 lines · Yann E. MORIN
Quoted 2 lines

Tested support for compressed kernel modules. Did not test signed modules.

BR2_arm=y
BR2_cortex_a9=y
BR2_ARM_ENABLE_NEON=y
BR2_ARM_ENABLE_VFP=y
BR2_ARM_FPU_VFPV3=y
BR2_CCACHE=y
BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
BR2_KERNEL_HEADERS_4_14=y
BR2_GCC_VERSION_8_X=y
BR2_TOOLCHAIN_BUILDROOT_CXX=y
BR2_GCC_ENABLE_LTO=y
BR2_GCC_ENABLE_OPENMP=y
BR2_TARGET_GENERIC_HOSTNAME="tsa38x"
BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_MDEV=y
BR2_TARGET_TZ_INFO=y
BR2_ROOTFS_OVERLAY="$(BR2_EXTERNAL_TECHNOLOGIC_PATH)/board/tsa38x/rootfs_overlay" BR2_ROOTFS_POST_BUILD_SCRIPT="$(BR2_EXTERNAL_TECHNOLOGIC_PATH)/board/tsa38x/post-build.sh" BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_TARBALL=y
BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION="$(call github,embeddedarm,linux-a38x,9007c40f288d3eb01689c66ac0fad166d63c1958)/linux-9007c40f288d3eb01689c66ac0fad166d63c1958.tar.gz" BR2_LINUX_KERNEL_DEFCONFIG="tsa38x"
BR2_LINUX_KERNEL_DTS_SUPPORT=y
BR2_LINUX_KERNEL_INTREE_DTS_NAME="armada-385-ts7840"
BR2_LINUX_KERNEL_INSTALL_TARGET=y
BR2_PACKAGE_LINUX_TOOLS_GPIO=y
BR2_PACKAGE_LINUX_TOOLS_IIO=y
BR2_PACKAGE_BUSYBOX_SHOW_OTHERS=y
BR2_PACKAGE_BUSYBOX_WATCHDOG=y
BR2_PACKAGE_BUSYBOX_WATCHDOG_PERIOD="30"
BR2_PACKAGE_BZIP2=y
BR2_PACKAGE_XZ=y
BR2_PACKAGE_LINUX_FIRMWARE=y
BR2_PACKAGE_LINUX_FIRMWARE_IMX_SDMA=y
BR2_PACKAGE_LIBZIP=y
BR2_PACKAGE_OPENSSL=y
BR2_PACKAGE_LIBOPENSSL_BIN=y
BR2_PACKAGE_EXPAT=y
BR2_PACKAGE_LIBEVENT=y
BR2_PACKAGE_LIBGLIB2=y
# BR2_PACKAGE_IFUPDOWN_SCRIPTS is not set
BR2_PACKAGE_KMOD=y
BR2_PACKAGE_KMOD_TOOLS=y
BR2_TARGET_ROOTFS_TAR_XZ=y
BR2_PACKAGE_HOST_KMOD_XZ=y
BR2_PACKAGE_HOST_UBOOT_TOOLS=y
BR2_PACKAGE_HOST_UTIL_LINUX=y

Relevant Linux Kernel options

CONFIG_MODULES_USE_ELF_REL=y
CONFIG_ARCH_HAS_STRICT_KERNEL_RWX=y
CONFIG_STRICT_KERNEL_RWX=y
CONFIG_ARCH_HAS_STRICT_MODULE_RWX=y
CONFIG_STRICT_MODULE_RWX=y
CONFIG_MODULES=y
CONFIG_MODULE_FORCE_LOAD=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
CONFIG_MODVERSIONS=y
# CONFIG_MODULE_SIG is not set
CONFIG_MODULE_COMPRESS=y
CONFIG_MODULE_COMPRESS_XZ=y

Signature · 5 lines
Best Regards,
________________________________________________________________
Kris Bahnsen, Technologic Systems     | voice: (480) 837-5200
16525 East Laser Drive                | fax: (480) 837-5300
Fountain Hills, AZ 85268              | web: www.embeddedARM.com
#5 lore raw

PATCH v2 2/3 package/kmod: add option for host variant

In reply to Yann E. MORIN

Curently, host-kmod has no option to enable it, because only the kernel depends on it, and this is unconditional (because we can't know if modules will be enabled in the kernel config).

But we're soon to add options to enable various features of kmod, so we'll need a place where to show those features.

Cc Yegor Yefremov <[email protected]>
Cc Lucian Buga <[email protected]>
---
 linux/Config.in             | 1 +
 package/Config.in.host      | 1 +
 package/kmod/Config.in.host | 6 ++++++
 3 files changed, 8 insertions(+)
 create mode 100644 package/kmod/Config.in.host
linux/Config.in +1 -0
index d7373ab25a..231d5a5962 100644
@@ -2,6 +2,7 @@ menu "Kernel"
 
 config BR2_LINUX_KERNEL
 	bool "Linux Kernel"
+	select BR2_PACKAGE_HOST_KMOD # Unconditional, even if modules not enabled
 	select BR2_PACKAGE_HOST_IMAGEMAGICK if BR2_LINUX_KERNEL_CUSTOM_LOGO_PATH != ""
 	help
 	  Enable this option if you want to build a Linux kernel for
package/Config.in.host +1 -0
index f1246c708f..140ffa7198 100644
@@ -38,6 +38,7 @@ menu "Host utilities"
 	source "package/imx-usb-loader/Config.in.host"
 	source "package/jq/Config.in.host"
 	source "package/jsmin/Config.in.host"
+	source "package/kmod/Config.in.host"
 	source "package/libp11/Config.in.host"
 	source "package/lld/Config.in.host"
 	source "package/lpc3250loader/Config.in.host"
package/kmod/Config.in.host new file +6 -0
new file mode 100644
index 0000000000..ea99a790f6
@@ -0,0 +1,6 @@
+config BR2_PACKAGE_HOST_KMOD
+	bool "host kmod"
+	help
+	  kmod is a set of tools to handle common tasks with Linux
+	  kernel modules like insert, remove, list, check properties,
+	  resolve dependencies and aliases.
Signature · 1 line
2.20.1
#6 lore raw

PATCH v2 2/3 package/kmod: add option for host variant

Yann,

Quoted 11 lines · Yann E. MORIN

Curently, host-kmod has no option to enable it, because only the kernel depends on it, and this is unconditional (because we can't know if modules will be enabled in the kernel config).

But we're soon to add options to enable various features of kmod, so we'll need a place where to show those features.

Cc Yegor Yefremov <[email protected]>
Cc Lucian Buga <[email protected]>
#7 lore raw

PATCH v2 3/3 package/lmod: add compression support for the host variant

In reply to Yann E. MORIN

The kernel may install compressed modules. At the end of the buid, we then run depmod, to generate modules.dep and a few assorted files, so that loadign modules works properly on the target (loading by alias, loading dependencies...)

However, depmod needs support for compressed modules, or it would generate empty modules.dep et al.

Systematically adding support for gz and xz, and the required deps, was deemed too much of a burden [0], so we add too options to enable either or both compression.

[0] http://lists.busybox.net/pipermail/buildroot/2018-April/218410.html

Yegor Yefremov <[email protected]>
Cc: Signed-off-by: Yann E. MORIN <[email protected]>
Cc: Thomas Petazzoni <[email protected]>
Cc: Lucian Buga <[email protected]>

---
Changes v1 -> v2:
- make compression support optional (Thomas)

---
 package/kmod/Config.in.host | 10 ++++++++++
 package/kmod/kmod.mk        | 14 ++++++++++++++
 2 files changed, 24 insertions(+)
package/kmod/Config.in.host +10 -0
index ea99a790f6..810fac3156 100644
@@ -4,3 +4,13 @@ config BR2_PACKAGE_HOST_KMOD
 	  kmod is a set of tools to handle common tasks with Linux
 	  kernel modules like insert, remove, list, check properties,
 	  resolve dependencies and aliases.
+
+if BR2_PACKAGE_HOST_KMOD
+
+config BR2_PACKAGE_HOST_KMOD_GZ
+	bool "support gzip-compresed modules"
+
+config BR2_PACKAGE_HOST_KMOD_XZ
+	bool "support xz-compresed modules"
+
+endif
package/kmod/kmod.mk +14 -0
index cdbfc5a469..979c3e87e6 100644
@@ -75,6 +75,20 @@ else
 KMOD_CONF_OPTS += --disable-tools
 endif
 
+ifeq ($(BR2_PACKAGE_HOST_KMOD_GZ),y)
+HOST_KMOD_DEPENDENCIES += host-zlib
+HOST_KMOD_CONF_OPTS += --with-zlib
+else
+HOST_KMOD_CONF_OPTS += --without-zlib
+endif
+
+ifeq ($(BR2_PACKAGE_HOST_KMOD_XZ),y)
+HOST_KMOD_DEPENDENCIES += host-xz
+HOST_KMOD_CONF_OPTS += --with-xz
+else
+HOST_KMOD_CONF_OPTS += --without-xz
+endif
+
 # We only install depmod, since that's the only tool used for the
 # host.
 define HOST_KMOD_INSTALL_TOOLS
Signature · 1 line
2.20.1
#8 lore raw

PATCH v2 3/3 package/lmod: add compression support for the host variant

Yann,

Quoted 13 lines · Yann E. MORIN

The kernel may install compressed modules. At the end of the buid, we then run depmod, to generate modules.dep and a few assorted files, so that loadign modules works properly on the target (loading by alias, loading dependencies...)

However, depmod needs support for compressed modules, or it would generate empty modules.dep et al.

Systematically adding support for gz and xz, and the required deps, was deemed too much of a burden [0], so we add too options to enable either or both compression.

Quoted 6 lines

[0] http://lists.busybox.net/pipermail/buildroot/2018-April/218410.html

Yegor Yefremov <[email protected]>
Cc: Signed-off-by: Yann E. MORIN <[email protected]>
Cc: Thomas Petazzoni <[email protected]>
Cc: Lucian Buga <[email protected]>

#9 lore raw

PATCH v2 3/3 package/lmod: add compression support for the host variant

In reply to Yann E. MORIN

Yann,

Quoted 2 lines · Yann E. MORIN

The kernel may install compressed modules. At the end of the buid, we

buid -> build

Quoted 2 lines

then run depmod, to generate modules.dep and a few assorted files, so that loadign modules works properly on the target (loading by alias,

loading -> loading

Quoted 29 lines

loading dependencies...)

However, depmod needs support for compressed modules, or it would generate empty modules.dep et al.

Systematically adding support for gz and xz, and the required deps, was deemed too much of a burden [0], so we add too options to enable either or both compression.

[0] http://lists.busybox.net/pipermail/buildroot/2018-April/218410.html

Yegor Yefremov <[email protected]>
Cc: Signed-off-by: Yann E. MORIN <[email protected]>
Cc: Thomas Petazzoni <[email protected]>
Cc: Lucian Buga <[email protected]>

---
Changes v1 -> v2:
- make compression support optional (Thomas)

---
 package/kmod/Config.in.host | 10 ++++++++++
 package/kmod/kmod.mk        | 14 ++++++++++++++
 2 files changed, 24 insertions(+)
package/kmod/Config.in.host +0 -0
index ea99a790f6..810fac3156 100644

package/kmod/Config.in.host config BR2_PACKAGE_HOST_KMOD · line 11

Quoted 8 lines
package/kmod/Config.in.host +5 -0
@@ -4,3 +4,13 @@ config BR2_PACKAGE_HOST_KMOD
          kmod is a set of tools to handle common tasks with Linux
          kernel modules like insert, remove, list, check properties,
          resolve dependencies and aliases.
+
+if BR2_PACKAGE_HOST_KMOD
+
+config BR2_PACKAGE_HOST_KMOD_GZ
+       bool "support gzip-compresed modules"

gzip-compresed -> gzip-compressed

package/kmod/Config.in.host config BR2_PACKAGE_HOST_KMOD · line 14

Quoted 3 lines
package/kmod/Config.in.host +3 -0
+
+config BR2_PACKAGE_HOST_KMOD_XZ
+       bool "support xz-compresed modules"

xz-compresed -> xz-compressed

-Matt

#10 lore raw

v2 cover package/kmod: add support for compressed modules (branch yem/host-kmod-z)

In reply to Yann E. MORIN
Quoted 4 lines · On Sun, 3 May 2020 15:09:58 +0200 "Yann E. MORIN" <[email protected]>…

Yann E. MORIN (3):
package/kmod: explicitly disable compression when not enabled package/kmod: add option for host variant package/lmod: add compression support for the host variant

Thanks, I've applied after taking into account the review comments from Matt Weber.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com