Skip to contentSkip to conversation

module: make modversion_info contain a pointer, not an array.

PATCH In review Rusty Russell 4 participants 10 messages Jan 28, 2009 – Feb 10, 2009 Module support

lore
Share & tools
lore
#0 lore raw

PATCH 5/6 module: make modversion_info contain a pointer, not an array.

With allmodconfig (minus non-building modules) on 32-bit x86:
Total size of modules before: 60009790 bytes
Total size of modules after: 55927866 bytes

Saving 7% of module size for CONFIG_MODVERSIONS=y; and these sections are kept resident as well.

---
 include/linux/module.h |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
include/linux/module.h +3 -1
@@ -37,10 +37,12 @@ struct kernel_symbol
 	const char *name;
 };
 
+/* This is put in the __versions section of a module to indicate the version
+ * it expects for unknown symbols. */
 struct modversion_info
 {
 	unsigned long crc;
-	char name[MODULE_NAME_LEN];
+	char *name;
 };
 
 struct module;
#1 lore raw
Quoted 8 lines · On Thu, 29 Jan 2009 00:05:52 +1030 Rusty Russell <[email protected]> wrote:

With allmodconfig (minus non-building modules) on 32-bit x86:
Total size of modules before: 60009790 bytes
Total size of modules after: 55927866 bytes

Saving 7% of module size for CONFIG_MODVERSIONS=y; and these sections are kept resident as well.

that reminds me.. can we just simplify MODVERSIONS to be a md5sum (or sha1 whatver) of the .config file in the VERMAGIC ?
it's a lot more reliable in detecting incompatibilities, and a lot less space consumed.

Signature · 3 lines
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
#2 lore raw
Quoted 8 lines · On Thursday 29 January 2009 01:22:31 Arjan van de Ven wrote:
Quoted 5 lines · On Thu, 29 Jan 2009 00:05:52 +1030 Rusty Russell <[email protected]> wrote:

With allmodconfig (minus non-building modules) on 32-bit x86:
Total size of modules before: 60009790 bytes
Total size of modules after: 55927866 bytes

Quoted 8 lines
Quoted 3 lines

Saving 7% of module size for CONFIG_MODVERSIONS=y; and these sections are kept resident as well.

that reminds me.. can we just simplify MODVERSIONS to be a md5sum (or sha1 whatver) of the .config file in the VERMAGIC ?
it's a lot more reliable in detecting incompatibilities, and a lot less space consumed.

Unfortunately people really seem to want the finer granularity that MODVERSIONS (sometimes) provides :( I've tried killing it off several times, and always failed.

We did discuss changing modversions not to descend more than one level deep into types (ie. MODVERSION(int fn(struct foo *)) would depend on the type of struct foo, but not the type of any struct pointers in struct foo) to reduce the problem where header changes cause type definitions to be exposed or hidden and thus change the modversion.

I asked someone to do some analysis on what effect this would have on signatures in real releases, but never heard back (it was in Austin last year, and there was beer...)

Cheers,
Rusty.

undefined types

Quoted 1 line
#3 lore raw
Quoted 13 lines · On Thu, 29 Jan 2009 08:59:40 +1030 Rusty Russell <[email protected]> wrote:
Quoted 12 lines · On Thursday 29 January 2009 01:22:31 Arjan van de Ven wrote:
Quoted 8 lines · On Thu, 29 Jan 2009 00:05:52 +1030 Rusty Russell <[email protected]> wrote:

With allmodconfig (minus non-building modules) on 32-bit x86:
Total size of modules before: 60009790 bytes
Total size of modules after: 55927866 bytes

Saving 7% of module size for CONFIG_MODVERSIONS=y; and these sections are kept resident as well.

Quoted 8 lines
Quoted 4 lines

that reminds me.. can we just simplify MODVERSIONS to be a md5sum (or sha1 whatver) of the .config file in the VERMAGIC ? it's a lot more reliable in detecting incompatibilities, and a lot less space consumed.

Unfortunately people really seem to want the finer granularity that MODVERSIONS (sometimes) provides :( I've tried killing it off several times, and always failed.

but we could just stick the result in VERMAGIC right? rather than tacking it to every symbol.

how you calculate the global checksum is almost a separate debate.

Signature · 3 lines
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
#4 lore raw
Quoted 21 lines · On Thursday 29 January 2009 09:11:14 Arjan van de Ven wrote:
Quoted 18 lines · On Thu, 29 Jan 2009 08:59:40 +1030 Rusty Russell <[email protected]> wrote:
Quoted 16 lines · On Thursday 29 January 2009 01:22:31 Arjan van de Ven wrote:
Quoted 8 lines · On Thu, 29 Jan 2009 00:05:52 +1030 Rusty Russell <[email protected]> wrote:

With allmodconfig (minus non-building modules) on 32-bit x86:
Total size of modules before: 60009790 bytes
Total size of modules after: 55927866 bytes

Saving 7% of module size for CONFIG_MODVERSIONS=y; and these sections are kept resident as well.

that reminds me.. can we just simplify MODVERSIONS to be a md5sum (or sha1 whatver) of the .config file in the VERMAGIC ? it's a lot more reliable in detecting incompatibilities, and a lot less space consumed.

Quoted 6 lines
Quoted 3 lines

Unfortunately people really seem to want the finer granularity that MODVERSIONS (sometimes) provides :( I've tried killing it off several times, and always failed.

but we could just stick the result in VERMAGIC right? rather than tacking it to every symbol.

Sure, but who would use it? If you add a driver I don't want my modules to fail.

Confused,
Rusty.

#5 lore raw
In reply to Rusty Russell
Quoted 6 lines · Rusty Russell

We did discuss changing modversions not to descend more than one level deep into types (ie. MODVERSION(int fn(struct foo *)) would depend on the type of struct foo, but not the type of any struct pointers in struct foo) to reduce the problem where header changes cause type definitions to be exposed or hidden and thus change the modversion.

Yeah. We looked at that for a certain Enterprise Linux distribution but various people suggested it was more hassle than it was worth to get it right. I did do some initial poking - I'll dig it up and post it here.

Jon.

#6 lore raw
In reply to Rusty Russell
Quoted 17 lines · Rusty Russell

With allmodconfig (minus non-building modules) on 32-bit x86:
Total size of modules before: 60009790 bytes
Total size of modules after: 55927866 bytes

Saving 7% of module size for CONFIG_MODVERSIONS=y; and these sections are kept resident as well.

---
 include/linux/module.h |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
include/linux/module.h +0 -0
@@ -37,10 +37,12 @@ struct kernel_symbol
 	const char *name;

include/linux/module.h struct kernel_symbol · line 46

Quoted 10 lines
include/linux/module.h +3 -1
 };
 
+/* This is put in the __versions section of a module to indicate the version
+ * it expects for unknown symbols. */
 struct modversion_info
 {
 	unsigned long crc;
-	char name[MODULE_NAME_LEN];
+	char *name;
 };

Hey Rusty thanks for the change, but I just got around to testing this and I have a few observations/questions. First this breaks:

modprobe --dump-modversions foo.ko

I actually don't care, but just happened to stumble upon it when I also noticed that your changes seemed to be working a little too well. I currently happen to be building my modules out of order for example module B depends on symbols from module A, but I build module B first. This means that the exported symbols are not in the Module.symvers file when module B is compiled. I would expect module B to fail to load yet with your patches it magically works and I don't see any errors in the logs. If I run:

objdump -s --section __versions B.ko

I only get something like:

Contents of section __versions:
0000 d782ec86 00000000

Of course if I build the modules in the correct order and dump the __versions section I see the correct number of crc/char* pairs.

--
Shawn

#7 lore raw
Quoted 32 lines · On Friday 06 February 2009 02:31:28 Shawn Bohrer wrote:
Quoted 17 lines · Rusty Russell

With allmodconfig (minus non-building modules) on 32-bit x86:
Total size of modules before: 60009790 bytes
Total size of modules after: 55927866 bytes

Saving 7% of module size for CONFIG_MODVERSIONS=y; and these sections are kept resident as well.

---
 include/linux/module.h |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
include/linux/module.h +0 -0
@@ -37,10 +37,12 @@ struct kernel_symbol
 	const char *name;

include/linux/module.h struct kernel_symbol · line 46

Quoted 10 lines
include/linux/module.h +3 -1
 };
 
+/* This is put in the __versions section of a module to indicate the version
+ * it expects for unknown symbols. */
 struct modversion_info
 {
 	unsigned long crc;
-	char name[MODULE_NAME_LEN];
+	char *name;
 };

Hey Rusty thanks for the change, but I just got around to testing this and I have a few observations/questions. First this breaks:

Quoted 1 line

modprobe --dump-modversions foo.ko

Hmm, that's an argument not to make this change. Really, other than disk space benefit we get all the benefits from just dropping the __versions section once loaded.

Quoted 8 lines

I actually don't care, but just happened to stumble upon it when I also noticed that your changes seemed to be working a little too well. I currently happen to be building my modules out of order for example module B depends on symbols from module A, but I build module B first. This means that the exported symbols are not in the Module.symvers file when module B is compiled. I would expect module B to fail to load yet with your patches it magically works and I don't see any errors in the logs.

Interesting. What is the value of /proc/sys/kernel/tainted?

And anyway, what was the symbol name which is over 56 characters long which started this?

Thanks,
Rusty.

#8 lore raw
Quoted 48 lines · Rusty Russell
Quoted 32 lines · On Friday 06 February 2009 02:31:28 Shawn Bohrer wrote:
Quoted 17 lines · Rusty Russell

With allmodconfig (minus non-building modules) on 32-bit x86:
Total size of modules before: 60009790 bytes
Total size of modules after: 55927866 bytes

Saving 7% of module size for CONFIG_MODVERSIONS=y; and these sections are kept resident as well.

---
 include/linux/module.h |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
include/linux/module.h +0 -0
@@ -37,10 +37,12 @@ struct kernel_symbol
 	const char *name;

include/linux/module.h struct kernel_symbol · line 46

Quoted 10 lines
include/linux/module.h +3 -1
 };
 
+/* This is put in the __versions section of a module to indicate the version
+ * it expects for unknown symbols. */
 struct modversion_info
 {
 	unsigned long crc;
-	char name[MODULE_NAME_LEN];
+	char *name;
 };

Hey Rusty thanks for the change, but I just got around to testing this and I have a few observations/questions. First this breaks:

Quoted 1 line

modprobe --dump-modversions foo.ko

Hmm, that's an argument not to make this change. Really, other than disk space benefit we get all the benefits from just dropping the __versions section once loaded.

Quoted 8 lines

I actually don't care, but just happened to stumble upon it when I also noticed that your changes seemed to be working a little too well. I currently happen to be building my modules out of order for example module B depends on symbols from module A, but I build module B first. This means that the exported symbols are not in the Module.symvers file when module B is compiled. I would expect module B to fail to load yet with your patches it magically works and I don't see any errors in the logs.

Quoted 1 line

Interesting. What is the value of /proc/sys/kernel/tainted?

It is 3 in both cases (built in the correct and incorrect order). If I'm reading this correctly that means my module is proprietary and was force loaded. It is proprietary, but was not literally force loaded. I'm guessing since the modversions information is wrong it sets the force loaded bit. I have no idea why the force loaded bit is set even when things are built in the correct order.

Of course this made me dig a little deeper, and I think I may have been fighting a regression all along, or possibly I'm missing a kernel config option. My real problem is that on openSUSE 11.1 (2.6.27) I was trying to load my module and modprobe returned:

FATAL: Error inserting foo: Invalid module format

running dmesg I saw the kernel print:

foo: no symbol version for struct_module

After some digging I realized I wasn't building with modversion support and that led to the long symbol name problem. I originally believed that openSUSE 11.1 turned on CONFIG_MODVERSIONS and previous versions did not have it enabled. After taking a closer look at openSUSE 10.3 (2.6.22) I found it did have CONFIG_MODVERSIONS=y and that modprobing my module succeeds but the kernel does print:

foo: no version for "stuct_module" found: kernel tainted

This also results in a taint flag of 3. So I'm thinking the regression here is that somewhere between 2.6.25 (I know openSUSE 11 worked too) and 2.6.27 the kernel stopped allowing modules to load if they didn't build with modversion support and the kernel was built with CONFIG_MODVERSIONS=y.

Quoted 2 lines

And anyway, what was the symbol name which is over 56 characters long which started this?

Yeah, that is an interesting question isn't it? Let me just say that namespaces and name mangling make it really easy to have names longer than 56 characters.

I actually don't care at all about modversions. I also don't care if it taints the kernel, I really just want the modules to load.

--
Shawn

#9 lore raw
Quoted 5 lines · On Tuesday 10 February 2009 04:20:50 Shawn Bohrer wrote:

This also results in a taint flag of 3. So I'm thinking the regression here is that somewhere between 2.6.25 (I know openSUSE 11 worked too) and 2.6.27 the kernel stopped allowing modules to load if they didn't build with modversion support and the kernel was built with CONFIG_MODVERSIONS=y.

Before 2.6.26, the kernel ignored the vermagic field when CONFIG_MODVERSIONS=y. I changed it to only ignore vermagic *if* there were CRCs.

If you use modprobe --force, you should be able to load a non-modversions module into a modversions kernel.

Quoted 6 lines
Quoted 2 lines

And anyway, what was the symbol name which is over 56 characters long which started this?

Yeah, that is an interesting question isn't it? Let me just say that namespaces and name mangling make it really easy to have names longer than 56 characters.

Yes, C++. This is one reason we don't support it.

Hope that helps,
Rusty.