Skip to contentSkip to conversation

package/kmod - fix host build dependecies

PATCH New Lucian Buga 2 participants 7 messages May 3, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 package/kmod - fix host build dependecies

When kernel is configured with CONFIG_MODULE_COMPRESS, host-kmod must be built with zlib / xz support to be able to properly generate dependency files for target.

---
 package/kmod/kmod.mk | 4 ++++
 1 file changed, 4 insertions(+)
package/kmod/kmod.mk +4 -0
index e2dfea5c7b..7149658ac7 100644
@@ -35,11 +35,15 @@ endif
 ifeq ($(BR2_PACKAGE_ZLIB),y)
 KMOD_DEPENDENCIES += zlib
 KMOD_CONF_OPTS += --with-zlib
+HOST_KMOD_DEPENDENCIES += host-zlib
+HOST_KMOD_CONF_OPTS += --with-zlib
 endif
 
 ifeq ($(BR2_PACKAGE_XZ),y)
 KMOD_DEPENDENCIES += xz
 KMOD_CONF_OPTS += --with-xz
+HOST_KMOD_DEPENDENCIES += host-xz
+HOST_KMOD_CONF_OPTS += --with-xz
 endif
 
 ifeq ($(BR2_PACKAGE_PYTHON)$(BR2_PACKAGE_PYTHON3),y)
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/1 package/kmod - fix host build dependecies

Lucian, All,

+Thomas +Peter: question for you.

On 2020-05-03 09:51 +0300, Lucian Buga spake thusly:

Quoted 14 lines

When kernel is configured with CONFIG_MODULE_COMPRESS, host-kmod must be built with zlib / xz support to be able to properly generate dependency files for target.

---
 package/kmod/kmod.mk | 4 ++++
 1 file changed, 4 insertions(+)
package/kmod/kmod.mk +0 -0
index e2dfea5c7b..7149658ac7 100644

package/kmod/kmod.mk endif · line 39

Quoted 5 lines
package/kmod/kmod.mk +2 -0
@@ -35,11 +35,15 @@ endif
 ifeq ($(BR2_PACKAGE_ZLIB),y)
 KMOD_DEPENDENCIES += zlib
 KMOD_CONF_OPTS += --with-zlib
+HOST_KMOD_DEPENDENCIES += host-zlib
+HOST_KMOD_CONF_OPTS += --with-zlib

At first, I was going to say that this was incorrect, because support for gz/xz on the target should not mandate whether the host variant has such support of not.

However, it is my understanding that the uncompressing of modules is done by the modprobe/insmod, and the uncompressed blurb is handled to the kernel.

If that is true, then indeed, this patch makes sense:

- if the target variant does not have compression support, then it will not be able to load compresed modules, so it would be useless or the host variant to have compression support,

- if the target variant has compression support, then it may have to handle compressed modules, and thus the host variant must have ompression support.

So this patch kinda makes sense in the end.

However, there is still a gotcha: the kernel can be configured to compress modules on installation, and that is (in Buildroot) orthogonal with the selection of BR2_PACKAGE_ZLIB or BR2_PACKAGE_XZ.

So, it would be invaliud, but possible to have a Buildroot configuration that contains:

# BR2_PACKAGE_ZLIB is not set
# BR2_PACKAGE_XZ is not set

while at the same time having a kernel configuration that contains:

CONFIG_MODULE_COMPRESS=y CONFIG_MODULE_COMPRESS_GZIP=y (or XZ, you get the point)

To address this problem, I did a 6-patch series two years ago:
https://git.buildroot.org/~ymorin/git/buildroot/log?h=yem/host-kmod

I just rebased it, but I did not pursue this endeavour much further than just posting it once...

I think your patch is good, but I'd like some feedback from Thomas or Peter. Or any other that can confirm who is responsible for uncompressing kernel modules on load...

Regards,
Yann E. MORIN.

Quoted 17 lines
package/kmod/kmod.mk +2 -0
 endif
 
 ifeq ($(BR2_PACKAGE_XZ),y)
 KMOD_DEPENDENCIES += xz
 KMOD_CONF_OPTS += --with-xz
+HOST_KMOD_DEPENDENCIES += host-xz
+HOST_KMOD_CONF_OPTS += --with-xz
 endif
 
 ifeq ($(BR2_PACKAGE_PYTHON)$(BR2_PACKAGE_PYTHON3),y)
Signature · 6 lines
2.17.1

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#2 lore raw

PATCH 1/1 package/kmod - fix host build dependecies

Yann, All,

To be more precise, the patch fixes the host depmod utility from kmod package.

Host depmod runs in target-finalize and in will create emty modules.dep if not buit with compression support.

So the issue is not insmod/modprobe on target.

Regards,
Lucian

Quoted 95 lines · Yann E. MORIN

?Lucian, All,

+Thomas +Peter: question for you.

On 2020-05-03 09:51 +0300, Lucian Buga spake thusly:

Quoted 15 lines

When kernel is configured with CONFIG_MODULE_COMPRESS, host-kmod must be built with zlib / xz support to be able to properly generate dependency files for target.

---
package/kmod/kmod.mk | 4 ++++

1 file changed, 4 insertions(+)
package/kmod/kmod.mk +0 -0
index e2dfea5c7b..7149658ac7 100644
@@ -35,11 +35,15 @@ endif
 ifeq ($(BR2_PACKAGE_ZLIB),y)

KMOD_DEPENDENCIES += zlib
KMOD_CONF_OPTS += --with-zlib

package/kmod/kmod.mk endif · line 39

Quoted 2 lines
package/kmod/kmod.mk +2 -0
+HOST_KMOD_DEPENDENCIES += host-zlib
+HOST_KMOD_CONF_OPTS += --with-zlib

At first, I was going to say that this was incorrect, because support for gz/xz on the target should not mandate whether the host variant has such support of not.

However, it is my understanding that the uncompressing of modules is done by the modprobe/insmod, and the uncompressed blurb is handled to the kernel.

If that is true, then indeed, this patch makes sense:

- if the target variant does not have compression support, then it will not be able to load compresed modules, so it would be useless or the host variant to have compression support,

- if the target variant has compression support, then it may have to handle compressed modules, and thus the host variant must have ompression support.

So this patch kinda makes sense in the end.

However, there is still a gotcha: the kernel can be configured to compress modules on installation, and that is (in Buildroot) orthogonal with the selection of BR2_PACKAGE_ZLIB or BR2_PACKAGE_XZ.

So, it would be invaliud, but possible to have a Buildroot configuration that contains:

# BR2_PACKAGE_ZLIB is not set
# BR2_PACKAGE_XZ is not set

while at the same time having a kernel configuration that contains:

CONFIG_MODULE_COMPRESS=y CONFIG_MODULE_COMPRESS_GZIP=y (or XZ, you get the point)

To address this problem, I did a 6-patch series two years ago:
https://git.buildroot.org/~ymorin/git/buildroot/log?h=yem/host-kmod

I just rebased it, but I did not pursue this endeavour much further than just posting it once...

I think your patch is good, but I'd like some feedback from Thomas or Peter. Or any other that can confirm who is responsible for uncompressing kernel modules on load...

Regards,
Yann E. MORIN.

Quoted 14 lines

endif
ifeq ($(BR2_PACKAGE_XZ),y)
KMOD_DEPENDENCIES += xz
KMOD_CONF_OPTS += --with-xz

package/kmod/kmod.mk +2 -0
+HOST_KMOD_DEPENDENCIES += host-xz
+HOST_KMOD_CONF_OPTS += --with-xz

endif
ifeq ($(BR2_PACKAGE_PYTHON)$(BR2_PACKAGE_PYTHON3),y) --
2.17.1
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#3 lore raw

PATCH 1/1 package/kmod - fix host build dependecies

Lucian, All,

On 2020-05-03 12:13 +0300, Lucian Buga spake thusly:

Quoted 2 lines

To be more precise, the patch fixes the host depmod utility from kmod package.

Exactly.

Quoted 4 lines

Host depmod runs in target-finalize and in will create emty modules.dep if not buit with compression support.

So the issue is not insmod/modprobe on target.

But the two are linked, in fact: if insmod/modprobe on target do not have compression support, they would not be able to load compresed modules. As a consequence, we don't care that the host variant was able to generate a modules.dep or not.

But really, the problem is even deeper than that: the target kmod package may very well be missing, if busybox' implementation is used. Adn that too supports compressed modules. So if the kernel compresses its installed modules, and busybox is used (instead of kmod) on the target, then the host-kmod, which is still built, will not generate a proper modules.dep.

So in the end, we _do_ have to decorelate the host compression support from the target ones.

And we're back to the point we were two years ago when I initially proposed that: if we unconditionally build host-kmod with compression support, this is an overhead that is paid by everyone, for the benefit of the few who wants compressed modules.

So, I'm marling your patch as changes-requested because it indeed does not work.

Please, could you have a look at the branch I linked to earlier, to see if it would fir your needs:
https://git.buildroot.org/~ymorin/git/buildroot/log?h=yem/host-kmod

Regards,
Yann E. MORIN.

Quoted 98 lines

Regards,
Lucian

Quoted 95 lines · Yann E. MORIN

?Lucian, All,

+Thomas +Peter: question for you.

On 2020-05-03 09:51 +0300, Lucian Buga spake thusly:

Quoted 15 lines

When kernel is configured with CONFIG_MODULE_COMPRESS, host-kmod must be built with zlib / xz support to be able to properly generate dependency files for target.

---
package/kmod/kmod.mk | 4 ++++

1 file changed, 4 insertions(+)
package/kmod/kmod.mk +0 -0
index e2dfea5c7b..7149658ac7 100644
@@ -35,11 +35,15 @@ endif
 ifeq ($(BR2_PACKAGE_ZLIB),y)

KMOD_DEPENDENCIES += zlib
KMOD_CONF_OPTS += --with-zlib

package/kmod/kmod.mk endif · line 39

Quoted 2 lines
package/kmod/kmod.mk +2 -0
+HOST_KMOD_DEPENDENCIES += host-zlib
+HOST_KMOD_CONF_OPTS += --with-zlib

At first, I was going to say that this was incorrect, because support for gz/xz on the target should not mandate whether the host variant has such support of not.

However, it is my understanding that the uncompressing of modules is done by the modprobe/insmod, and the uncompressed blurb is handled to the kernel.

If that is true, then indeed, this patch makes sense:

- if the target variant does not have compression support, then it will not be able to load compresed modules, so it would be useless or the host variant to have compression support,

- if the target variant has compression support, then it may have to handle compressed modules, and thus the host variant must have ompression support.

So this patch kinda makes sense in the end.

However, there is still a gotcha: the kernel can be configured to compress modules on installation, and that is (in Buildroot) orthogonal with the selection of BR2_PACKAGE_ZLIB or BR2_PACKAGE_XZ.

So, it would be invaliud, but possible to have a Buildroot configuration that contains:

# BR2_PACKAGE_ZLIB is not set
# BR2_PACKAGE_XZ is not set

while at the same time having a kernel configuration that contains:

CONFIG_MODULE_COMPRESS=y CONFIG_MODULE_COMPRESS_GZIP=y (or XZ, you get the point)

To address this problem, I did a 6-patch series two years ago:
https://git.buildroot.org/~ymorin/git/buildroot/log?h=yem/host-kmod

I just rebased it, but I did not pursue this endeavour much further than just posting it once...

I think your patch is good, but I'd like some feedback from Thomas or Peter. Or any other that can confirm who is responsible for uncompressing kernel modules on load...

Regards,
Yann E. MORIN.

Quoted 14 lines

endif
ifeq ($(BR2_PACKAGE_XZ),y)
KMOD_DEPENDENCIES += xz
KMOD_CONF_OPTS += --with-xz

package/kmod/kmod.mk +2 -0
+HOST_KMOD_DEPENDENCIES += host-xz
+HOST_KMOD_CONF_OPTS += --with-xz

endif
ifeq ($(BR2_PACKAGE_PYTHON)$(BR2_PACKAGE_PYTHON3),y) --
2.17.1
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#4 lore raw

PATCH 1/1 package/kmod - fix host build dependecies

Lucian, All,

On 2020-05-03 14:11 +0200, Yann E. MORIN spake thusly:

Quoted 3 lines

On 2020-05-03 12:13 +0300, Lucian Buga spake thusly:

Quoted 2 lines

To be more precise, the patch fixes the host depmod utility from kmod package.

[--SNIP--]

Quoted 3 lines

Please, could you have a look at the branch I linked to earlier, to see if it would fir your needs:
https://git.buildroot.org/~ymorin/git/buildroot/log?h=yem/host-kmod

In fact I just thought of a much simpler solution. I'll send a patch series shortly...

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#5 lore raw

PATCH 1/1 package/kmod - fix host build dependecies

Lucian, All,

On 2020-05-03 14:20 +0200, Yann E. MORIN spake thusly:

Quoted 11 lines

On 2020-05-03 14:11 +0200, Yann E. MORIN spake thusly:

Quoted 3 lines

On 2020-05-03 12:13 +0300, Lucian Buga spake thusly:

Quoted 2 lines

To be more precise, the patch fixes the host depmod utility from kmod package.

[--SNIP--]

Quoted 3 lines

Please, could you have a look at the branch I linked to earlier, to see if it would fir your needs:
https://git.buildroot.org/~ymorin/git/buildroot/log?h=yem/host-kmod

In fact I just thought of a much simpler solution. I'll send a patch series shortly...

I'll have to test this a bit, but you can already have a peek at the 2-patch series:
https://git.buildroot.org/~ymorin/git/buildroot/log/?h=yem/host-kmod-z

I'll be doing better commit logs, and will push that later today after I test it further. But you may still have a look and provide some feedback (if you want to test on your side too).

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#6 lore raw

PATCH 1/1 package/kmod - fix host build dependecies

Yann, All,

Quoted 5 lines

I'll have to test this a bit, but you can already have a peek at the 2-patch series:
https://git.buildroot.org/~ymorin/git/buildroot/log/?h=yem/host-kmod-z

I think your suggested patches will fix the issue.

Thank you and regards,
Lucian Buga

Quoted 39 lines · Yann E. MORIN

Lucian, All,

On 2020-05-03 14:20 +0200, Yann E. MORIN spake thusly:

Quoted 8 lines

On 2020-05-03 14:11 +0200, Yann E. MORIN spake thusly:

Quoted 3 lines

On 2020-05-03 12:13 +0300, Lucian Buga spake thusly:

Quoted 2 lines

To be more precise, the patch fixes the host depmod utility from kmod package.

[--SNIP--]

Quoted 3 lines

Please, could you have a look at the branch I linked to earlier, to see if it would fir your needs:

https://git.buildroot.org/~ymorin/git/buildroot/log?h=yem/host-kmod

Quoted 3 lines

In fact I just thought of a much simpler solution. I'll send a patch series shortly...

I'll have to test this a bit, but you can already have a peek at the 2-patch series:
https://git.buildroot.org/~ymorin/git/buildroot/log/?h=yem/host-kmod-z

I'll be doing better commit logs, and will push that later today after I test it further. But you may still have a look and provide some feedback (if you want to test on your side too).

Regards,
Yann E. MORIN.

--

.-----------------.--------------------.------------------.--------------------.

|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics'
conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___
     |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is
no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v
 conspiracy.  |
Quoted 2 lines

'------------------------------^-------^------------------^--------------------'

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200503/40177d2e/attachment.html>