Skip to contentSkip to conversation

Add a kernel module loading mechanism

PATCH New unixmania at gmail.com 4 participants 12 messages Apr 18, 2020 – Apr 21, 2020

lore
Share & tools
lore
#0 lore raw

v2 cover Add a kernel module loading mechanism

From: Carlos Santos <[email protected]>

Currently there are two ways to load kernel modules in Buildroot:

- via systemd (systemd-modules-load.service)
- via OpenRC (/etc/init.d/modules)

The systemd mechanism uses configuration files, each one containing a list of kernel module names to load, separated by newlines. Empty lines and lines whose first non-whitespace character is # or ; are ignored. Files are loaded in the following order:

/etc/modules-load.d/*.conf
/run/modules-load.d/*.conf
/usr/lib/modules-load.d/*.conf

OpenRC uses /etc/modules-load.d/*.conf, only, and does not ignore lines beginning with ';'.

In order to provide a compatible mechanism for sysvinit/busybox init systems, this series does the following:

Patch #1 adds a S02modules-load init script to kmod (which provides the "modprobe" utility). It roughly mimics the systemd service.

Patch #2 adds the S02modules-load init script to Busybox. It's installed only if kmod is not selected and modprobe is included in buildroot.

Patchs #3..5 modify packages dmraid, owfs and ti-sgx-km to use the modules-load mechanism.

Package ti-gfx should be modified too, but I did not find a board configuration to test it.

ABOUT THE REASON OF THIS SERIES

Since late 2017 I have been working on porting libvirt to Buildroot, initially as part of my work at DATACOM and since April 2019 as a pet project, using hardware kindly donated by DATCOM[1]. libvirt is a big and complex package that has many build and run time dependencies and requires several kernel modules to work.

In previous versions of the package, e.g. [2], the modules were loaded by the S92libvirtd init script. The next patch, however (to be submitted soon) will add support for both OpenRC and systemd init systems, so I needed a commom mechanism to load the modules. That was how the patches for kmod and busybox were born. The ones for dmraid, owfs an ti-sgx-km are just a consequence of my "scratch the itch" syndrome. :-)

[1] https://www.datacom.com.br/en/produtos/network-appliance
[2] https://patchwork.ozlabs.org/patch/1183162/
---
Carlos Santos (5):
package/kmod: add modules-load init script package/busybox: add modules-load init script package/dmraid: use modules-load to load the kernel module package/owfs: use modules-load to load the kernel module package/ti-sgx-km: use modules-load to load the kernel module

 package/busybox/S11modules-load |   1 +
 package/busybox/busybox.mk      |  12 ++++
 package/dmraid/S20dmraid        |   3 -
 package/dmraid/dmraid.mk        |  12 ++++
 package/kmod/S11modules-load    | 115 ++++++++++++++++++++++++++++++++
 package/kmod/kmod.mk            |  10 +++
 package/owfs/S60owfs            |   2 -
 package/owfs/owfs.mk            |  11 +++
 package/ti-sgx-km/ti-sgx-km.mk  |  10 +++
 package/ti-sgx-um/S80ti-sgx     |  19 +-----
 10 files changed, 173 insertions(+), 22 deletions(-)
 create mode 120000 package/busybox/S11modules-load
 create mode 100644 package/kmod/S11modules-load
Signature · 1 line
2.18.2
#1 lore raw

PATCH v2 1/5 package/kmod: add modules-load init script

From: Carlos Santos <[email protected]>

Use some scripting to mimic the systemd "modules-load" and the OpenRC "modules" services (load kernel modules based on static configuration).

The configuration files should simply contain a list of kernel module names to load, separated by newlines. Empty lines and lines whose first non-whitespace character is # or ; are ignored.

The configuration directory list and file format is the same as the one described in modules-load.d(5). Files are loaded in the following order:

/etc/modules-load.d/*.conf
/run/modules-load.d/*.conf
/usr/lib/modules-load.d/*.conf

This roughly mimics the logic used by systemd but the files are not sorted by their filename in lexicographic order as systemd does.

Notice that OpenRC uses /etc/modules-load.d/*.conf, only, and does not ignore lines beginning with ';'.

The file redirections do the following:

- stdout is redirected to syslog with facility.level "kern.info"
- stderr is redirected to syslog with facility.level "kern.err"
- file dscriptor 4 is used to pass the result to the "start" function.

---
CC: "Yann E. MORIN" <[email protected]>
---
Changes v1->v2:
- Redirect output to syslog only if /usr/bin/logger exists, as made in package/procps-ng/S02sysctl.
- Use the MODULES_LOAD_ARGS variable, optionally set in /etc/default/modules-load.
- Rename S02modules-load to S11modules-load to ensure that it runs after S10mdev and S10udev, which both are going to trigger cold-plug events that may in turn trigger module loading, as observed by Yann E. MORIN.
- Prevent the installation of S11modules-load with openrc, which already provides the /etc/init.d/modules script.
- Reorganize the comments in S11modules-load.

---
 package/kmod/S11modules-load | 115 +++++++++++++++++++++++++++++++++++
 package/kmod/kmod.mk         |  10 +++
 2 files changed, 125 insertions(+)
 create mode 100644 package/kmod/S11modules-load
package/kmod/S11modules-load new file +115 -0
new file mode 100644
index 0000000000..9010230e23
@@ -0,0 +1,115 @@
+#!/bin/sh
+#
+# Use some scripting to mimic the systemd "modules-load" and the OpenRC
+# "modules" services (load kernel modules based on static configuration).
+#
+# The configuration files should simply contain a list of kernel module
+# names to load, separated by newlines. Empty lines and lines whose first
+# non-whitespace character is # or ; are ignored.
+#
+# The configuration directory list and file format is the same as the one
+# described in modules-load.d(5). Files are loaded in the following order:
+#
+#   /etc/modules-load.d/*.conf
+#   /run/modules-load.d/*.conf
+#   /usr/lib/modules-load.d/*.conf
+#
+# This roughly mimics the logic used by systemd but the files are not sorted
+# by their filename in lexicographic order as systemd does.
+#
+# Notice that OpenRC uses /etc/modules-load.d/*.conf, only, and does not
+# ignore lines beginning with ';'.
+#
+
+PROGRAM="modules-load"
+
+MODULES_LOAD_ARGS=""
+
+# Add "-q" to MODULES_LOAD_ARGS to disable error messages.
+# shellcheck source=/dev/null
+[ -r "/etc/default/$PROGRAM" ] && . "/etc/default/$PROGRAM"
+
+# Files are read from directories in the MODULES_SOURCES list, in the given
+# order. A file may be used more than once, since there can be multiple
+# symlinks to it. No attempt is made to prevent this.
+MODULES_SOURCES="/etc/modules-load.d/ /run/modules-load.d/ /usr/lib/modules-load.d/"
+
+# If the logger utility is available all messages are sent to syslog, except
+# for the final status. The file redirections do the following:
+#
+# - stdout is redirected to syslog with facility.level "kern.info"
+# - stderr is redirected to syslog with facility.level "kern.err"
+# - file dscriptor 4 is used to pass the result to the "start" function.
+#
+run_logger() {
+	# shellcheck disable=SC2086 # we need the word splitting
+	find $MODULES_SOURCES -maxdepth 1 -name '*.conf' -print0 2> /dev/null | \
+	xargs -0 -r -n 1 readlink -f | {
+		prog_status="OK"
+		while :; do
+			read -r file || {
+				echo "$prog_status" >&4
+				break
+			}
+			echo "* Applying $file ..."
+			while :; do
+				read -r mod || break
+				case "$mod" in
+					''|'#'*|';'*) ;;
+					*) /sbin/modprobe $MODULES_LOAD_ARGS "$mod" || prog_status="FAIL"
+				esac
+			done < "$file"
+		done 2>&1 >&3 | /usr/bin/logger -t "$PROGRAM" -p kern.err
+	} 3>&1 | /usr/bin/logger -t "$PROGRAM" -p kern.info
+}
+
+# If logger is not available all messages are sent to stdout/stderr.
+run_std() {
+	# shellcheck disable=SC2086 # we need the word splitting
+	find $MODULES_SOURCES -maxdepth 1 -name '*.conf' -print0 2> /dev/null | \
+	xargs -0 -r -n 1 readlink -f | {
+		prog_status="OK"
+		while :; do
+			read -r file || {
+				echo "$prog_status" >&4
+				break
+			}
+			echo "* Applying $file ..."
+			while :; do
+				read -r mod || break
+				case "$mod" in
+					''|'#'*|';'*) ;;
+					*) /sbin/modprobe $MODULES_LOAD_ARGS "$mod" || prog_status="FAIL"
+				esac
+			done < "$file"
+		done
+	}
+}
+
+if [ -x /usr/bin/logger ]; then
+	run_program="run_logger"
+else
+	run_program="run_std"
+fi
+
+start() {
+	printf '%s %s: ' "$1" "$PROGRAM"
+	status=$("$run_program" 4>&1)
+	echo "$status"
+	if [ "$status" = "OK" ]; then
+		return 0
+	fi
+	return 1
+}
+
+case "$1" in
+	start)
+		start "Running";;
+	restart|reload)
+		start "Rerunning";;
+	stop)
+		:;;
+	*)
+		echo "Usage: $0 {start|stop|restart|reload}"
+		exit 1
+esac
package/kmod/kmod.mk +10 -0
index e2dfea5c7b..9b6735e2d0 100644
@@ -60,6 +60,16 @@ else
 KMOD_BIN_PATH = ../usr/bin/kmod
 endif
 
+# Avoid installing S11modules-load, since openrc provides /etc/init.d/modules.
+define KMOD_INSTALL_INIT_OPENRC
+	@:
+endef
+
+define KMOD_INSTALL_INIT_SYSV
+	$(INSTALL) -m 0755 -D package/kmod/S11modules-load \
+		$(TARGET_DIR)/etc/init.d/S11modules-load
+endef
+
 define KMOD_INSTALL_TOOLS
 	for i in depmod insmod lsmod modinfo modprobe rmmod; do \
 		ln -sf $(KMOD_BIN_PATH) $(TARGET_DIR)/sbin/$$i; \
Signature · 1 line
2.18.2
#2 lore raw

PATCH v2 1/5 package/kmod: add modules-load init script

Carlos, All,

On 2020-04-18 19:14 -0300, unixmania at gmail.com spake thusly:

Quoted 9 lines

From: Carlos Santos <[email protected]>

Use some scripting to mimic the systemd "modules-load" and the OpenRC "modules" services (load kernel modules based on static configuration).

The configuration files should simply contain a list of kernel module names to load, separated by newlines. Empty lines and lines whose first non-whitespace character is # or ; are ignored.

Quoted 6 lines

The configuration directory list and file format is the same as the one described in modules-load.d(5). Files are loaded in the following order:

/etc/modules-load.d/*.conf
/run/modules-load.d/*.conf
/usr/lib/modules-load.d/*.conf

I don't think we should have to support all these locations. After all with Buildroot, only the files installed by packages should make sense, as well as the administrator-provided files.

Besides, as you say yourself, the same-named files do not override one the others, as would otherwise be expected.

So, let's just handle the location where packages will install their files, and a global one for people to fill with an overlay or whatever.

As per the man page for modules-load.d:

Packages should install their configuration files in /usr/lib/

so let's do just that (and not /etc/ as you use in followup patches). If a package installs its own foo.conf list, and we want to override that, let's just overwrite the file installed by the package directly; this solves the limitation that same-named files do not override each others.

Quoted 11 lines

This roughly mimics the logic used by systemd but the files are not sorted by their filename in lexicographic order as systemd does.

Notice that OpenRC uses /etc/modules-load.d/*.conf, only, and does not ignore lines beginning with ';'.

The file redirections do the following:

- stdout is redirected to syslog with facility.level "kern.info"
- stderr is redirected to syslog with facility.level "kern.err"
- file dscriptor 4 is used to pass the result to the "start" function.

As I already replied in a previous iteration, this is overly complex, and IMHO totally unwarranted. More comments below, please read on...

Quoted 104 lines

---
CC: "Yann E. MORIN" <[email protected]>
---
Changes v1->v2:
- Redirect output to syslog only if /usr/bin/logger exists, as made in package/procps-ng/S02sysctl.
- Use the MODULES_LOAD_ARGS variable, optionally set in /etc/default/modules-load.
- Rename S02modules-load to S11modules-load to ensure that it runs after S10mdev and S10udev, which both are going to trigger cold-plug events that may in turn trigger module loading, as observed by Yann E. MORIN.
- Prevent the installation of S11modules-load with openrc, which already provides the /etc/init.d/modules script.
- Reorganize the comments in S11modules-load.

---
 package/kmod/S11modules-load | 115 +++++++++++++++++++++++++++++++++++
 package/kmod/kmod.mk         |  10 +++
 2 files changed, 125 insertions(+)
 create mode 100644 package/kmod/S11modules-load
package/kmod/S11modules-load new file +77 -0
new file mode 100644
index 0000000000..9010230e23
@@ -0,0 +1,115 @@
+#!/bin/sh
+#
+# Use some scripting to mimic the systemd "modules-load" and the OpenRC
+# "modules" services (load kernel modules based on static configuration).
+#
+# The configuration files should simply contain a list of kernel module
+# names to load, separated by newlines. Empty lines and lines whose first
+# non-whitespace character is # or ; are ignored.
+#
+# The configuration directory list and file format is the same as the one
+# described in modules-load.d(5). Files are loaded in the following order:
+#
+#   /etc/modules-load.d/*.conf
+#   /run/modules-load.d/*.conf
+#   /usr/lib/modules-load.d/*.conf
+#
+# This roughly mimics the logic used by systemd but the files are not sorted
+# by their filename in lexicographic order as systemd does.
+#
+# Notice that OpenRC uses /etc/modules-load.d/*.conf, only, and does not
+# ignore lines beginning with ';'.
+#
+
+PROGRAM="modules-load"
+
+MODULES_LOAD_ARGS=""
+
+# Add "-q" to MODULES_LOAD_ARGS to disable error messages.
+# shellcheck source=/dev/null
+[ -r "/etc/default/$PROGRAM" ] && . "/etc/default/$PROGRAM"
+
+# Files are read from directories in the MODULES_SOURCES list, in the given
+# order. A file may be used more than once, since there can be multiple
+# symlinks to it. No attempt is made to prevent this.
+MODULES_SOURCES="/etc/modules-load.d/ /run/modules-load.d/ /usr/lib/modules-load.d/"
+
+# If the logger utility is available all messages are sent to syslog, except
+# for the final status. The file redirections do the following:
+#
+# - stdout is redirected to syslog with facility.level "kern.info"
+# - stderr is redirected to syslog with facility.level "kern.err"
+# - file dscriptor 4 is used to pass the result to the "start" function.
+#
+run_logger() {
+	# shellcheck disable=SC2086 # we need the word splitting
+	find $MODULES_SOURCES -maxdepth 1 -name '*.conf' -print0 2> /dev/null | \
+	xargs -0 -r -n 1 readlink -f | {
+		prog_status="OK"
+		while :; do
+			read -r file || {
+				echo "$prog_status" >&4
+				break
+			}
+			echo "* Applying $file ..."
+			while :; do
+				read -r mod || break
+				case "$mod" in
+					''|'#'*|';'*) ;;
+					*) /sbin/modprobe $MODULES_LOAD_ARGS "$mod" || prog_status="FAIL"
+				esac
+			done < "$file"
+		done 2>&1 >&3 | /usr/bin/logger -t "$PROGRAM" -p kern.err
+	} 3>&1 | /usr/bin/logger -t "$PROGRAM" -p kern.info
+}
+
+# If logger is not available all messages are sent to stdout/stderr.
+run_std() {
+	# shellcheck disable=SC2086 # we need the word splitting
+	find $MODULES_SOURCES -maxdepth 1 -name '*.conf' -print0 2> /dev/null | \
+	xargs -0 -r -n 1 readlink -f | {
+		prog_status="OK"
+		while :; do
+			read -r file || {
+				echo "$prog_status" >&4
+				break
+			}
+			echo "* Applying $file ..."

package/kmod/S11modules-load line 87

Quoted 10 lines
package/kmod/S11modules-load new file +10 -0
+			while :; do
+				read -r mod || break
+				case "$mod" in
+					''|'#'*|';'*) ;;
+					*) /sbin/modprobe $MODULES_LOAD_ARGS "$mod" || prog_status="FAIL"
+				esac
+			done < "$file"
+		done
+	}
+}

So you are totally duplicating a complex piece of code just for the sake of logging of echoing to stdout/err? I don't think this duplication makes sense.

Besides, as I already explained, this code is much more complicated than it should be.

First, let's consider that this is sysv-init: don't try to be too smart with that, and just send everything to stdout/stderr. People who want more complex behaviour will have to use a more complex init system.

Second, let's consider each module to load the action to be tested.

    #!/bin/sh

    MODULES_DIR="/usr/lib/modules-load.d /etc/modules-load.d"

    [ -r "/etc/default/modules" ] && . "/etc/default/modules"

    start() {
        for d in ${MODULES_DIR}; do
            for mod_list in "${d}"/*; do
                [ -e "${mod_list}" ] || continue
                for m in $(sed -r -e 's/[;#].*$//' "${mod_list}"); do
                    printf 'Loading module %s: ' "${m}"
                    modprobe "${m}"
                    if [ ${?} -eq 0 ]; then
                        printf 'OK\n'
                    else
                        printf 'FAIL\n'
                    fi
                done
            done
        done
    }

    case "${1}" in
        start)i
            start;;
        stop|reload|restart) ;;
        *)
            echo "Usage: $0 {start|stop|restart|reload}"
            exit 1
            ;;
    esac

Notice that this correctly handles the fact that a directory may have no file in it: [ -e "${mod_list}" ] will test false in that case.

With the sed expression, we correctly drop comments.

And finally, the shell expansion of the result of the sed, will properly form space-separated words on which we can iterate. If the file was empty or only comments or only empty lines, the list will be empty, and the loop wil not iterate even once.

This is much easier to review, understand and maintain.

Note: I'm pretty sure we could do without the middle loop, at the expense of a slight sanity check, but that would be an unwarranted optimisation as well:

    for d in ${MODULES_DIR}; do
        for m in $(sed -r -e 's/[;#].*$//' "${d}"/* 2>/dev/null); do
            [...]
        done
    done

But as I said, that would be unwarranted; I only talked about it for the sake of illustration...

Regards,
Yann E. MORIN.

PS. I know you already argued that we had such complexity in other
startup scripts. I don't think that is a reason to continue the trend, quite the opposite: we should fix those other scripts to be simpler.
YEM.

Quoted 51 lines
package/kmod/S11modules-load new file +27 -0
+if [ -x /usr/bin/logger ]; then
+	run_program="run_logger"
+else
+	run_program="run_std"
+fi
+
+start() {
+	printf '%s %s: ' "$1" "$PROGRAM"
+	status=$("$run_program" 4>&1)
+	echo "$status"
+	if [ "$status" = "OK" ]; then
+		return 0
+	fi
+	return 1
+}
+
+case "$1" in
+	start)
+		start "Running";;
+	restart|reload)
+		start "Rerunning";;
+	stop)
+		:;;
+	*)
+		echo "Usage: $0 {start|stop|restart|reload}"
+		exit 1
+esac
package/kmod/kmod.mk +10 -0
index e2dfea5c7b..9b6735e2d0 100644
@@ -60,6 +60,16 @@ else
 KMOD_BIN_PATH = ../usr/bin/kmod
 endif
 
+# Avoid installing S11modules-load, since openrc provides /etc/init.d/modules.
+define KMOD_INSTALL_INIT_OPENRC
+	@:
+endef
+
+define KMOD_INSTALL_INIT_SYSV
+	$(INSTALL) -m 0755 -D package/kmod/S11modules-load \
+		$(TARGET_DIR)/etc/init.d/S11modules-load
+endef
+
 define KMOD_INSTALL_TOOLS
 	for i in depmod insmod lsmod modinfo modprobe rmmod; do \
 		ln -sf $(KMOD_BIN_PATH) $(TARGET_DIR)/sbin/$$i; \
Signature · 1 line
2.18.2
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#3 lore raw

PATCH v2 1/5 package/kmod: add modules-load init script

In reply to unixmania at gmail.com

Carlos,

Quoted 6 lines · On Sat, Apr 18, 2020 at 5:16 PM <[email protected]> wrote:

From: Carlos Santos <[email protected]>

Use some scripting to mimic the systemd "modules-load" and the OpenRC "modules" services (load kernel modules based on static configuration).

Thanks for sending this. I know I carry a lot of kmod loading scripts that all look the same and should have been upstreamed :-)

Just a note, I noticed this debian example script for sysv and it can also handle options. Usually having the ability to provide module options is one of the reasons to break out to use modules in some of our use cases.
https://github.com/SebKuzminsky/kmod/blob/master/debian/kmod.init

Quoted 193 lines

The configuration files should simply contain a list of kernel module names to load, separated by newlines. Empty lines and lines whose first non-whitespace character is # or ; are ignored.

The configuration directory list and file format is the same as the one described in modules-load.d(5). Files are loaded in the following order:

/etc/modules-load.d/*.conf
/run/modules-load.d/*.conf
/usr/lib/modules-load.d/*.conf

This roughly mimics the logic used by systemd but the files are not sorted by their filename in lexicographic order as systemd does.

Notice that OpenRC uses /etc/modules-load.d/*.conf, only, and does not ignore lines beginning with ';'.

The file redirections do the following:

- stdout is redirected to syslog with facility.level "kern.info"
- stderr is redirected to syslog with facility.level "kern.err"
- file dscriptor 4 is used to pass the result to the "start" function.

---
CC: "Yann E. MORIN" <[email protected]>
---
Changes v1->v2:
- Redirect output to syslog only if /usr/bin/logger exists, as made in package/procps-ng/S02sysctl.
- Use the MODULES_LOAD_ARGS variable, optionally set in /etc/default/modules-load.
- Rename S02modules-load to S11modules-load to ensure that it runs after S10mdev and S10udev, which both are going to trigger cold-plug events that may in turn trigger module loading, as observed by Yann E. MORIN.
- Prevent the installation of S11modules-load with openrc, which already provides the /etc/init.d/modules script.
- Reorganize the comments in S11modules-load.

---
 package/kmod/S11modules-load | 115 +++++++++++++++++++++++++++++++++++
 package/kmod/kmod.mk         |  10 +++
 2 files changed, 125 insertions(+)
 create mode 100644 package/kmod/S11modules-load
package/kmod/S11modules-load new file +115 -0
new file mode 100644
index 0000000000..9010230e23
@@ -0,0 +1,115 @@
+#!/bin/sh
+#
+# Use some scripting to mimic the systemd "modules-load" and the OpenRC
+# "modules" services (load kernel modules based on static configuration).
+#
+# The configuration files should simply contain a list of kernel module
+# names to load, separated by newlines. Empty lines and lines whose first
+# non-whitespace character is # or ; are ignored.
+#
+# The configuration directory list and file format is the same as the one
+# described in modules-load.d(5). Files are loaded in the following order:
+#
+#   /etc/modules-load.d/*.conf
+#   /run/modules-load.d/*.conf
+#   /usr/lib/modules-load.d/*.conf
+#
+# This roughly mimics the logic used by systemd but the files are not sorted
+# by their filename in lexicographic order as systemd does.
+#
+# Notice that OpenRC uses /etc/modules-load.d/*.conf, only, and does not
+# ignore lines beginning with ';'.
+#
+
+PROGRAM="modules-load"
+
+MODULES_LOAD_ARGS=""
+
+# Add "-q" to MODULES_LOAD_ARGS to disable error messages.
+# shellcheck source=/dev/null
+[ -r "/etc/default/$PROGRAM" ] && . "/etc/default/$PROGRAM"
+
+# Files are read from directories in the MODULES_SOURCES list, in the given
+# order. A file may be used more than once, since there can be multiple
+# symlinks to it. No attempt is made to prevent this.
+MODULES_SOURCES="/etc/modules-load.d/ /run/modules-load.d/ /usr/lib/modules-load.d/"
+
+# If the logger utility is available all messages are sent to syslog, except
+# for the final status. The file redirections do the following:
+#
+# - stdout is redirected to syslog with facility.level "kern.info"
+# - stderr is redirected to syslog with facility.level "kern.err"
+# - file dscriptor 4 is used to pass the result to the "start" function.
+#
+run_logger() {
+       # shellcheck disable=SC2086 # we need the word splitting
+       find $MODULES_SOURCES -maxdepth 1 -name '*.conf' -print0 2> /dev/null | \
+       xargs -0 -r -n 1 readlink -f | {
+               prog_status="OK"
+               while :; do
+                       read -r file || {
+                               echo "$prog_status" >&4
+                               break
+                       }
+                       echo "* Applying $file ..."
+                       while :; do
+                               read -r mod || break
+                               case "$mod" in
+                                       ''|'#'*|';'*) ;;
+                                       *) /sbin/modprobe $MODULES_LOAD_ARGS "$mod" || prog_status="FAIL"
+                               esac
+                       done < "$file"
+               done 2>&1 >&3 | /usr/bin/logger -t "$PROGRAM" -p kern.err
+       } 3>&1 | /usr/bin/logger -t "$PROGRAM" -p kern.info
+}
+
+# If logger is not available all messages are sent to stdout/stderr.
+run_std() {
+       # shellcheck disable=SC2086 # we need the word splitting
+       find $MODULES_SOURCES -maxdepth 1 -name '*.conf' -print0 2> /dev/null | \
+       xargs -0 -r -n 1 readlink -f | {
+               prog_status="OK"
+               while :; do
+                       read -r file || {
+                               echo "$prog_status" >&4
+                               break
+                       }
+                       echo "* Applying $file ..."
+                       while :; do
+                               read -r mod || break
+                               case "$mod" in
+                                       ''|'#'*|';'*) ;;
+                                       *) /sbin/modprobe $MODULES_LOAD_ARGS "$mod" || prog_status="FAIL"
+                               esac
+                       done < "$file"
+               done
+       }
+}
+
+if [ -x /usr/bin/logger ]; then
+       run_program="run_logger"
+else
+       run_program="run_std"
+fi
+
+start() {
+       printf '%s %s: ' "$1" "$PROGRAM"
+       status=$("$run_program" 4>&1)
+       echo "$status"
+       if [ "$status" = "OK" ]; then
+               return 0
+       fi
+       return 1
+}
+
+case "$1" in
+       start)
+               start "Running";;
+       restart|reload)
+               start "Rerunning";;
+       stop)
+               :;;
+       *)
+               echo "Usage: $0 {start|stop|restart|reload}"
+               exit 1
+esac
package/kmod/kmod.mk +10 -1
index e2dfea5c7b..9b6735e2d0 100644
@@ -60,6 +60,16 @@ else
 KMOD_BIN_PATH = ../usr/bin/kmod
 endif
 
+# Avoid installing S11modules-load, since openrc provides /etc/init.d/modules.
+define KMOD_INSTALL_INIT_OPENRC
+       @:
+endef
+
+define KMOD_INSTALL_INIT_SYSV
+       $(INSTALL) -m 0755 -D package/kmod/S11modules-load \
+               $(TARGET_DIR)/etc/init.d/S11modules-load
+endef
+
 define KMOD_INSTALL_TOOLS
        for i in depmod insmod lsmod modinfo modprobe rmmod; do \
                ln -sf $(KMOD_BIN_PATH) $(TARGET_DIR)/sbin/$$i; \
--

2.18.2

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Signature · 30 lines
Matthew Weber | Associate Director Software Engineer | Commercial Avionics

COLLINS AEROSPACE

400 Collins Road NE, Cedar Rapids, Iowa 52498, USA

Tel: +1 319 295 7349 | FAX: +1 319 263 6099

matthew.weber at collins.com | collinsaerospace.com



CONFIDENTIALITY WARNING: This message may contain proprietary and/or
privileged information of Collins Aerospace and its affiliated
companies. If you are not the intended recipient, please 1) Do not
disclose, copy, distribute or use this message or its contents. 2)
Advise the sender by return email. 3) Delete all copies (including all
attachments) from your computer. Your cooperation is greatly
appreciated.


Any export restricted material should be shared using my
matthew.weber at corp.rockwellcollins.com address.


ALPHA BRAVO COLLINS | Aerospace Redefined

         __ l __

 \- - - -o-(_)-o- - - -/
#4 lore raw

PATCH v2 2/5 package/busybox: add modules-load init script

In reply to unixmania at gmail.com

From: Carlos Santos <[email protected]>

Use some scripting to mimic the systemd "modules-load" and the OpenRC "modules" services (load kernel modules based on static configuration).

At the moment package/busybox/S02modules-load is a symlink to the kmod script, since it works with both versions of the "modprobe" utility.

package/busybox/

---
CC: "Yann E. MORIN" <[email protected]>
---
Changes v1->v2:
- Rename S02modules-load to S11modules-load to ensure that it runs after S10mdev and S10udev, which both are going to trigger cold-plug events that may in turn trigger module loading, as observed by Yann E. MORIN.

---
 package/busybox/S11modules-load |  1 +
 package/busybox/busybox.mk      | 12 ++++++++++++
 2 files changed, 13 insertions(+)
 create mode 120000 package/busybox/S11modules-load
package/busybox/S11modules-load new file +1 -0
new file mode 120000
index 0000000000..44623673ff
@@ -0,0 +1 @@
+../kmod/S11modules-load
\ No newline at end of file
package/busybox/busybox.mk +12 -0
index 5d5f3e92bd..3512bf77b5 100644
@@ -264,6 +264,17 @@ define BUSYBOX_INSTALL_LOGGING_SCRIPT
 endef
 endif
 
+# Only install our modules-load script if no other package does it.
+ifeq ($(BR2_PACKAGE_KMOD_TOOLS),)
+define BUSYBOX_INSTALL_MODULES_SCRIPT
+	if grep -q CONFIG_MODPROBE=y $(@D)/.config; \
+	then \
+		$(INSTALL) -m 0755 -D package/busybox/S11modules-load \
+			$(TARGET_DIR)/etc/init.d/S11modules-load; \
+	fi
+endef
+endif
+
 # Only install our sysctl scripts if no other package does it.
 ifeq ($(BR2_PACKAGE_PROCPS_NG),)
 define BUSYBOX_INSTALL_SYSCTL_SCRIPT
@@ -368,6 +379,7 @@ define BUSYBOX_INSTALL_INIT_SYSV
 	$(BUSYBOX_INSTALL_MDEV_SCRIPT)
 	$(BUSYBOX_INSTALL_LOGGING_SCRIPT)
 	$(BUSYBOX_INSTALL_WATCHDOG_SCRIPT)
+	$(BUSYBOX_INSTALL_MODULES_SCRIPT)
 	$(BUSYBOX_INSTALL_SYSCTL_SCRIPT)
 	$(BUSYBOX_INSTALL_TELNET_SCRIPT)
 endef
Signature · 1 line
2.18.2
#5 lore raw

PATCH v2 2/5 package/busybox: add modules-load init script

Carlos, All,

On 2020-04-18 19:14 -0300, unixmania at gmail.com spake thusly:

Quoted 31 lines

From: Carlos Santos <[email protected]>

Use some scripting to mimic the systemd "modules-load" and the OpenRC "modules" services (load kernel modules based on static configuration).

At the moment package/busybox/S02modules-load is a symlink to the kmod script, since it works with both versions of the "modprobe" utility.

package/busybox/

---
CC: "Yann E. MORIN" <[email protected]>
---
Changes v1->v2:
- Rename S02modules-load to S11modules-load to ensure that it runs after S10mdev and S10udev, which both are going to trigger cold-plug events that may in turn trigger module loading, as observed by Yann E. MORIN.

---
 package/busybox/S11modules-load |  1 +
 package/busybox/busybox.mk      | 12 ++++++++++++
 2 files changed, 13 insertions(+)
 create mode 120000 package/busybox/S11modules-load
package/busybox/S11modules-load new file +0 -0
new file mode 120000
index 0000000000..44623673ff

package/busybox/S11modules-load line 1

Quoted 2 lines
package/busybox/S11modules-load new file +1 -0
@@ -0,0 +1 @@
+../kmod/S11modules-load
\ No newline at end of file

I like that the script is shared between the two packages.

But please make that state prominent in the header of the file, with something like:

#!/bin/sh
# NOTE: this file is used both by kmod and busybox

Quoted 8 lines
package/busybox/busybox.mk +0 -0
index 5d5f3e92bd..3512bf77b5 100644
@@ -264,6 +264,17 @@ define BUSYBOX_INSTALL_LOGGING_SCRIPT
 endef
 endif
 

package/busybox/busybox.mk define BUSYBOX_INSTALL_LOGGING_SCRIPT · line 276

Quoted 10 lines
package/busybox/busybox.mk +10 -0
+# Only install our modules-load script if no other package does it.
+ifeq ($(BR2_PACKAGE_KMOD_TOOLS),)
+define BUSYBOX_INSTALL_MODULES_SCRIPT
+	if grep -q CONFIG_MODPROBE=y $(@D)/.config; \
+	then \
+		$(INSTALL) -m 0755 -D package/busybox/S11modules-load \
+			$(TARGET_DIR)/etc/init.d/S11modules-load; \
+	fi
+endef
+endif

So far, the heuristic to install a file from busybox was to install it in no-clobber mode, and guarantee the build ordering.

Since the build ordering is already guaranteed, we should only have to test if the startup script already exists before we isntall it.

Regards,
Yann E. MORIN.

Quoted 14 lines
package/busybox/busybox.mk +1 -0
 # Only install our sysctl scripts if no other package does it.
 ifeq ($(BR2_PACKAGE_PROCPS_NG),)
 define BUSYBOX_INSTALL_SYSCTL_SCRIPT
@@ -368,6 +379,7 @@ define BUSYBOX_INSTALL_INIT_SYSV
 	$(BUSYBOX_INSTALL_MDEV_SCRIPT)
 	$(BUSYBOX_INSTALL_LOGGING_SCRIPT)
 	$(BUSYBOX_INSTALL_WATCHDOG_SCRIPT)
+	$(BUSYBOX_INSTALL_MODULES_SCRIPT)
 	$(BUSYBOX_INSTALL_SYSCTL_SCRIPT)
 	$(BUSYBOX_INSTALL_TELNET_SCRIPT)
 endef
Signature · 1 line
2.18.2
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#6 lore raw

PATCH v2 3/5 package/dmraid: use modules-load to load the kernel module

In reply to unixmania at gmail.com

From: Carlos Santos <[email protected]>

This is compatible with BusyBox/sysvinit (via S02modules-load), OpenRC and systemd. It also prevents trying to load the module each time the init script is executed.

---
Tested on QEMU with a kernel config with MD=y and DM_<FOO>=m. Only the module loading was tested, since there is no RAID device.

---
 package/dmraid/S20dmraid |  3 ---
 package/dmraid/dmraid.mk | 12 ++++++++++++
 2 files changed, 12 insertions(+), 3 deletions(-)
package/dmraid/S20dmraid +0 -3
index b3bfdcc84b..ff325154c6 100644
@@ -2,9 +2,6 @@
 
 set -e
 
-# try to load module in case that hasn't been done yet
-modprobe dm-mod >/dev/null 2>&1
-
 case "$1" in
 	start|"")
 		echo "Setting up DMRAID devices..."
package/dmraid/dmraid.mk +12 -0
index 0382cd4d99..c8facbdabd 100644
@@ -21,4 +21,16 @@ define DMRAID_INSTALL_INIT_SYSV
 		$(TARGET_DIR)/etc/init.d/S20dmraid
 endef
 
+ifeq ($(BR2_LINUX_KERNEL),y)
+define DMRAID_GEN_MODULES_CONF
+	$(INSTALL) -d -m 755 $(TARGET_DIR)/etc/modules-load.d
+	{ \
+		find $(TARGET_DIR)/lib/modules/$(LINUX_VERSION_PROBED)/kernel/drivers/md -name 'dm[_-]mod.ko*'; \
+	} 2> /dev/null | { \
+		sed 's:^.*/::; s:\.ko[^ ]*::'; \
+	} > $(TARGET_DIR)/etc/modules-load.d/20-dmraid.conf
+endef
+DMRAID_TARGET_FINALIZE_HOOKS += DMRAID_GEN_MODULES_CONF
+endif
+
 $(eval $(autotools-package))
Signature · 1 line
2.18.2
#7 lore raw

PATCH v2 3/5 package/dmraid: use modules-load to load the kernel module

CArlos, All,

On 2020-04-18 19:14 -0300, unixmania at gmail.com spake thusly:

Quoted 38 lines

From: Carlos Santos <[email protected]>

This is compatible with BusyBox/sysvinit (via S02modules-load), OpenRC and systemd. It also prevents trying to load the module each time the init script is executed.

---
Tested on QEMU with a kernel config with MD=y and DM_<FOO>=m. Only the module loading was tested, since there is no RAID device.

---
 package/dmraid/S20dmraid |  3 ---
 package/dmraid/dmraid.mk | 12 ++++++++++++
 2 files changed, 12 insertions(+), 3 deletions(-)
package/dmraid/S20dmraid +0 -3
index b3bfdcc84b..ff325154c6 100644
@@ -2,9 +2,6 @@
 
 set -e
 
-# try to load module in case that hasn't been done yet
-modprobe dm-mod >/dev/null 2>&1
-
 case "$1" in
 	start|"")
 		echo "Setting up DMRAID devices..."
package/dmraid/dmraid.mk +1 -0
index 0382cd4d99..c8facbdabd 100644
@@ -21,4 +21,16 @@ define DMRAID_INSTALL_INIT_SYSV
 		$(TARGET_DIR)/etc/init.d/S20dmraid
 endef
 
+ifeq ($(BR2_LINUX_KERNEL),y)

package/dmraid/dmraid.mk define DMRAID_INSTALL_INIT_SYSV · line 34

Quoted 10 lines
package/dmraid/dmraid.mk +10 -0
+define DMRAID_GEN_MODULES_CONF
+	$(INSTALL) -d -m 755 $(TARGET_DIR)/etc/modules-load.d
+	{ \
+		find $(TARGET_DIR)/lib/modules/$(LINUX_VERSION_PROBED)/kernel/drivers/md -name 'dm[_-]mod.ko*'; \
+	} 2> /dev/null | { \
+		sed 's:^.*/::; s:\.ko[^ ]*::'; \
+	} > $(TARGET_DIR)/etc/modules-load.d/20-dmraid.conf
+endef
+DMRAID_TARGET_FINALIZE_HOOKS += DMRAID_GEN_MODULES_CONF
+endif

Why do we need this complex code? Furthermore, we'll have to repeat that complexity in all packages that want to provide modules; that's not nice... :-(

Can't we simply have a modules table, like we have the device amd user tables? Something like:

define DMRAID_MODULES
dm-stuff
dm-blurb
dm-blabla
endef

And then have the package and rootfs infrastructures handle that like the other tables?

Regards,
Yann E. MORIN.

Quoted 8 lines
package/dmraid/dmraid.mk +0 -0
 $(eval $(autotools-package))
Signature · 6 lines
2.18.2

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#8 lore raw

PATCH v2 3/5 package/dmraid: use modules-load to load the kernel module

Quoted 11 lines · On Sun, 19 Apr 2020 12:42:39 +0200 "Yann E. MORIN" <[email protected]>…

Can't we simply have a modules table, like we have the device amd user tables? Something like:

define DMRAID_MODULES
dm-stuff
dm-blurb
dm-blabla
endef

And then have the package and rootfs infrastructures handle that like the other tables?

If we're going to do that, then we don't need an infra at all, package/dmraid/ can simply contain a file with the list of modules, which gets installed to /usr/lib/modules-load.d/.

I think the idea here was to avoid maintaining the list of dmraid modules, and ensure it is generated automatically from the list of installed modules. But even that is not guaranteed to work 100%, for example if one day dmraid adds a module that doesn't match the dm[_-]mod.ko* expression.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#9 lore raw

PATCH v2 3/5 package/dmraid: use modules-load to load the kernel module

Thomas, Carlos, All,

On 2020-04-19 14:49 +0200, Thomas Petazzoni spake thusly:

Quoted 12 lines
Quoted 9 lines · On Sun, 19 Apr 2020 12:42:39 +0200 "Yann E. MORIN" <[email protected]>…

Can't we simply have a modules table, like we have the device amd user tables? Something like:

define DMRAID_MODULES
dm-stuff
dm-blurb
dm-blabla
endef

Quoted 6 lines
Quoted 2 lines

And then have the package and rootfs infrastructures handle that like the other tables?

If we're going to do that, then we don't need an infra at all, package/dmraid/ can simply contain a file with the list of modules, which gets installed to /usr/lib/modules-load.d/.

Right, adding an infra just to copy a file would over-engineering.

Quoted 5 lines

I think the idea here was to avoid maintaining the list of dmraid modules, and ensure it is generated automatically from the list of installed modules. But even that is not guaranteed to work 100%, for example if one day dmraid adds a module that doesn't match the dm[_-]mod.ko* expression.

But suddenly, it strikes me as odd that the dmraid package has to list the modules built by the kernel.

Those modules should be auto-loaded on-demand, no?

For dmraid, we may load just the main module, dm-mod, as was done so far, and leave to the user the responsibility to provide a module file to load whatever exact module they need?

I would be OK for a long list, but this complex code repeated over and over again for each package that needs to load modules, is just ugly to me...

Also, this created files during the target-finalize step, and so those files are not accounted to the package that installed them.

In the end, I guess such a functionality is mostly useful for those out-of-tree modules that do not behave properly, and need manual loading. In which case, those packages know what modules to list, and they can do so during the installation step (directly or as part of a post-install hook). Using target-finalize is not so nice...

Regards,
Yann E. MORIN.

Quoted 5 lines

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#10 lore raw

PATCH v2 4/5 package/owfs: use modules-load to load the kernel module

In reply to unixmania at gmail.com

From: Carlos Santos <[email protected]>

This is compatible with BusyBox/sysvinit (via S02modules-load), OpenRC and systemd. It also prevents trying to load the module each time the init script is executed.

---
Tested on QEMU with a kernel config with FUSE_FS=m. Only the module loading was tested, since there is no 1-wire device.

---
 package/owfs/S60owfs |  2 --
 package/owfs/owfs.mk | 11 +++++++++++
 2 files changed, 11 insertions(+), 2 deletions(-)
package/owfs/S60owfs +0 -2
index feed676f8a..46b7ec775d 100644
@@ -10,8 +10,6 @@ OWFS_ARGS=""
 case "$1" in
 start)
 	printf "Starting ${NAME}: "
-	# Fuse may be in a module, so try to load it
-	modprobe -q fuse && printf "[fuse] "
 	mkdir -p ${OWFS_MOUNTPOINT}
 	start-stop-daemon -S -x ${DAEMON} -- \
 		--pid_file ${PID_F} -m ${OWFS_MOUNTPOINT} ${OWFS_DEVICES} \
package/owfs/owfs.mk +11 -0
index ffc0b3098d..a09a4bb37b 100644
@@ -35,6 +35,17 @@ define OWFS_CREATE_MOUNTPOINT
 	mkdir -p $(TARGET_DIR)/dev/1wire
 endef
 OWFS_POST_INSTALL_TARGET_HOOKS += OWFS_CREATE_MOUNTPOINT
+ifeq ($(BR2_LINUX_KERNEL),y)
+define OWFS_GEN_MODULES_CONF
+	$(INSTALL) -d -m 755 $(TARGET_DIR)/etc/modules-load.d
+	{ \
+		find $(TARGET_DIR)/lib/modules/$(LINUX_VERSION_PROBED)/kernel/fs/fuse -name 'fuse.ko*'; \
+	} 2> /dev/null | { \
+		sed 's:^.*/::; s:\.ko[^ ]*::'; \
+	} > $(TARGET_DIR)/etc/modules-load.d/60-owfs.conf
+endef
+OWFS_TARGET_FINALIZE_HOOKS += OWFS_GEN_MODULES_CONF
+endif
 else
 OWFS_CONF_OPTS += --disable-owfs
 endif
Signature · 1 line
2.18.2
#11 lore raw

PATCH v2 5/5 package/ti-sgx-km: use modules-load to load the kernel module

In reply to unixmania at gmail.com

From: Carlos Santos <[email protected]>

This is compatible with BusyBox/sysvinit (via S02modules-load), OpenRC and systemd. It also prevents trying to load the module each time the init script is executed.

Remove the insmod command from the ti-sgx-um init script.

---
Tested on a BeagleBone Black card using beaglebone_qt5_defconfig.

---
 package/ti-sgx-km/ti-sgx-km.mk | 10 ++++++++++
 package/ti-sgx-um/S80ti-sgx    | 19 ++-----------------
 2 files changed, 12 insertions(+), 17 deletions(-)
package/ti-sgx-km/ti-sgx-km.mk +10 -0
index ae294c300a..7f627f276b 100644
@@ -33,4 +33,14 @@ define TI_SGX_KM_INSTALL_TARGET_CMDS
 		kbuild_install -C $(@D)/$(TI_SGX_KM_SUBDIR)
 endef
 
+define TI_SGX_KM_GEN_MODULES_CONF
+	$(INSTALL) -d -m 755 $(TARGET_DIR)/etc/modules-load.d
+	{ \
+		find $(TARGET_DIR)/lib/modules/$(LINUX_VERSION_PROBED)/extra -name 'pvrsrvkm.ko*'; \
+	} 2> /dev/null | { \
+		sed 's:^.*/::; s:\.ko[^ ]*::'; \
+	} > $(TARGET_DIR)/etc/modules-load.d/50-ti-sgx-km.conf
+endef
+TI_SGX_KM_TARGET_FINALIZE_HOOKS += TI_SGX_KM_GEN_MODULES_CONF
+
 $(eval $(generic-package))
package/ti-sgx-um/S80ti-sgx +2 -17
index 2630a0576a..0f183b6fc5 100644
@@ -1,24 +1,9 @@
 #!/bin/sh
 
-pvrsrvkm_ko="/lib/modules/$(/bin/uname -r)/extra/pvrsrvkm.ko"
-
-pvr_loaded() {
-	/sbin/lsmod | /bin/grep -q '^\<pvrsrvkm\>'
-}
-
-pvr_load() {
-	/sbin/insmod "$pvrsrvkm_ko" > /dev/null 2>&1
-}
-
 start() {
-	printf 'Loading pvrsrvkm module: '
-	pvr_loaded || pvr_load
+	printf 'Starting PowerVR services: '
+	/usr/bin/pvrsrvctl --start --no-module > /dev/null 2>&1
 	status=$?
-	if [ "$status" -eq 0 ]; then
-		printf 'Starting PowerVR services: '
-		/usr/bin/pvrsrvctl --start --no-module > /dev/null 2>&1
-		status=$?
-	fi
 	if [ "$status" -eq 0 ]; then
 		echo "OK"
 	else
Signature · 1 line
2.18.2