memcmp in modules
A message that is not in the archive (sent off-list or not yet received): CAHp75VdDwh0wgCWrVDcPps1C2qfdDZra15y9BQvzfWHzSMqbWA@mail.gmail.com
HI.
I have an issue with memcmp, but it seems I didn't get what is happening there. Let's consider the details.
I have a module which has one call of memcmp and licensed as Dual BSD/GPL.
#include <linux/string.h>
...
if (memcmp(...))
...MODULE_LICENSE("Dual BSD/GPL");
I compile it for ARCH=i386 together with kernel and try to load it.
modprobe test-string_helpers
[ 25.134433] test_string_helpers: no symbol version for memcmp [ 25.140156] test_string_helpers: Unknown symbol memcmp (err -22)
Then I've changed string_32.h a bit (I think it's not a solution of the issue)
-#define memcmp __builtin_memcmp
+#define memcmp(a, b, n) __builtin_memcmp(a, b, n)
And in this case:
modprobe test_string_helpers
[ 15.391589] test_string_helpers: Running tests...
What did I miss?
--
With Best Regards,
Andy Shevchenko
Quoted 1 line · Andy Shevchenko
What did I miss?
Well, as of linux-next-20130422, memcmp() is not correctly exported to modules. Since linux-3.9-rc8 correctly exports memcmp(), this problem seems to be introduced in linux-next tree. Also, this problem seems to involve CONFIG_MODVERSIONS=y.
[root@localhost linux-next]# modprobe ipv6
FATAL: Error inserting ipv6 (/lib/modules/3.9.0-rc8-next-20130422/kernel/net/ipv6/ipv6.ko): Invalid argument [root@localhost linux-next]# dmesg
ipv6: no symbol version for memcmp
ipv6: Unknown symbol memcmp (err -22)
Since arch/x86/include/asm/string_64.h uses
int memcmp(const void *cs, const void *ct, size_t count);
while arch/x86/include/asm/string_32.h uses
#define memcmp __builtin_memcmp
changing to what you have tried
#define memcmp(a, b, n) __builtin_memcmp(a, b, n)
or changing to what x86_64 does
int memcmp(const void *cs, const void *ct, size_t count);
might solve this problem. But I don't know which one is correct solution...
PATCH 1/1 x86_32: Fix module version table mismatch.
Commit a4b6a77b "module: fix symbol versioning with symbol prefixes" broke loading of net/ipv6/ipv6.ko built with CONFIG_MODVERSIONS=y for x86_32.
# modprobe ipv6 FATAL: Error inserting ipv6 (/lib/modules/3.9.0-rc8-next-20130422/kernel/net/ipv6/ipv6.ko): Invalid argument # dmesg ipv6: no symbol version for memcmp ipv6: Unknown symbol memcmp (err -22)
The reason for breakage is that check_version() in kernel/module.c tries to find symname == "memcmp" but versions[i].name == "__builtin_memcmp".
The reason for versions[i].name == "__builtin_memcmp" is that memcmp() for x86_32 is defined as
#define memcmp __builtin_memcmp
in arch/x86/include/asm/string_32.h while memcmp() for x86_64 is defined as
int memcmp(const void *cs, const void *ct, size_t count);
in arch/x86/include/asm/string_64.h.
Since __builtin_memcmp is a gcc's built-in function which might emit a call to memcmp, __builtin_memcmp should not be used for versions[i].name field.
In order to make sure that versions[i].name == "memcmp", make the definition of memcmp() for x86_32 identical with that of x86_64.
--- arch/x86/include/asm/string_32.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/include/asm/string_32.h +1 -1
@@ -199,7 +199,7 @@ static inline void *__memcpy3d(void *to, const void *from, size_t len)#define __HAVE_ARCH_MEMMOVEvoid *memmove (void *dest,const void *src,size_t n); -#define memcmp __builtin_memcmp +int memcmp (const void *cs,const void *ct,size_t count);#define __HAVE_ARCH_MEMCHRextern void *memchr (const void *cs,int c,size_t count);
Signature · 1 line
1.7.1
Re: [PATCH] x86_32: Fix module version table mismatch.
Quoted 2 lines · Tetsuo Handa
Commit
a4b6a77b"module: fix symbol versioning with symbol prefixes" broke loading ofnet/ipv6/ipv6.ko
I would rephrase this to "any module uses memcmp, e.g. ipv6"
Otherwise: Reviewed-by: Andy Shevchenko <[email protected]> ...and
Quoted 49 lines
built with
CONFIG_MODVERSIONS=yforx86_32.# modprobe ipv6 FATAL: Error inserting ipv6 (/lib/modules/3.9.0-rc8-next-20130422/kernel/net/ipv6/ipv6.ko): Invalid argument # dmesg ipv6: no symbol version for memcmp ipv6: Unknown symbol memcmp (err -22)The reason for breakage is that
check_version()inkernel/module.ctries to find symname == "memcmp" but versions[i].name == "__builtin_memcmp".The reason for versions[i].name == "
__builtin_memcmp" is thatmemcmp()forx86_32is defined as#define memcmp __builtin_memcmpin
arch/x86/include/asm/string_32.hwhilememcmp()forx86_64is defined asint memcmp(const void *cs, const void *ct, size_t count);in
arch/x86/include/asm/string_64.h.Since
__builtin_memcmpis a gcc's built-in function which might emit a call to memcmp,__builtin_memcmpshould not be used for versions[i].name field.In order to make sure that versions[i].name == "memcmp", make the definition of
memcmp()forx86_32identical with that ofx86_64.Signed-off-by Tetsuo Handa <[email protected]>Cc James Hogan <[email protected]>Cc Rusty Russell <[email protected]>--- arch/x86/include/asm/string_32.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)arch/x86/include/asm/string_32.h +1
-2@@ -199,7 +199,7 @@ static inline void *__memcpy3d(void *to, const void *from, size_t len)#define __HAVE_ARCH_MEMMOVEvoid *memmove (void *dest,const void *src,size_t n); -#define memcmp __builtin_memcmp +int memcmp (const void *cs,const void *ct,size_t count);#define __HAVE_ARCH_MEMCHRextern void *memchr (const void *cs,int c,size_t count); --
Quoted 1 line
1.7.1
--
With Best Regards,
Andy Shevchenko
H. Peter Anvin
reviewerRe: [PATCH] x86_32: Fix module version table mismatch.
In reply to Tetsuo Handaarch/x86/include/asm/string_32.h static inline void *__memcpy3d(void *to, const void *from, size_t len) · line 202
Quoted 4 lines · Tetsuo Handa
arch/x86/include/asm/string_32.h +1
-1-#define memcmp __builtin_memcmp +int memcmp (const void *cs,const void *ct,size_t count);
Yuck. I really don't like this option unless it truly can't be avoided... it might be a fix for 3.9/stable but a better way to do this would be much more appreciated.
-hpa
Re: [PATCH] x86_32: Fix module version table mismatch.
Quoted 9 lines · H. Peter Anvin
arch/x86/include/asm/string_32.h static inline void *__memcpy3d(void *to, const void *from, size_t len) · line 202
Quoted 4 lines · Tetsuo Handa
arch/x86/include/asm/string_32.h +1
-1-#define memcmp __builtin_memcmp +int memcmp (const void *cs,const void *ct,size_t count);Yuck. I really don't like this option unless it truly can't be avoided... it might be a fix for 3.9/stable but a better way to do this would be much more appreciated.
What about my proposal to supply parameters for __builtin_memcmp in memcmp macro.
--
With Best Regards,
Andy Shevchenko
H. Peter Anvin
reviewerRe: [PATCH] x86_32: Fix module version table mismatch.
Quoted 11 lines · Andy Shevchenko
Quoted 9 lines · H. Peter Anvin
arch/x86/include/asm/string_32.h static inline void *__memcpy3d(void *to, const void *from, size_t len) · line 202
Quoted 4 lines · Tetsuo Handa
arch/x86/include/asm/string_32.h +1
-1-#define memcmp __builtin_memcmp +int memcmp (const void *cs,const void *ct,size_t count);Yuck. I really don't like this option unless it truly can't be avoided... it might be a fix for 3.9/stable but a better way to do this would be much more appreciated.
Quoted 3 lines
What about my proposal to supply parameters for
__builtin_memcmpin memcmp macro.
I don't see it anywhere.
-hpa
H. Peter Anvin
reviewerRe: [PATCH] x86_32: Fix module version table mismatch.
In reply to Tetsuo HandaQuoted 21 lines · Tetsuo Handa
Commit
a4b6a77b"module: fix symbol versioning with symbol prefixes" broke loading ofnet/ipv6/ipv6.kobuilt withCONFIG_MODVERSIONS=yforx86_32.# modprobe ipv6 FATAL: Error inserting ipv6 (/lib/modules/3.9.0-rc8-next-20130422/kernel/net/ipv6/ipv6.ko): Invalid argument # dmesg ipv6: no symbol version for memcmp ipv6: Unknown symbol memcmp (err -22)The reason for breakage is that
check_version()inkernel/module.ctries to find symname == "memcmp" but versions[i].name == "__builtin_memcmp".The reason for versions[i].name == "
__builtin_memcmp" is thatmemcmp()forx86_32is defined as#define memcmp __builtin_memcmpin
arch/x86/include/asm/string_32.hwhilememcmp()forx86_64is defined asint memcmp(const void *cs, const void *ct, size_t count);
Quoted 8 lines
in
arch/x86/include/asm/string_64.h.Since
__builtin_memcmpis a gcc's built-in function which might emit a call to memcmp,__builtin_memcmpshould not be used for versions[i].name field.In order to make sure that versions[i].name == "memcmp", make the definition of
memcmp()forx86_32identical with that ofx86_64.
I'm still confused by all of this. VMLINUX_SYMBOL_STR() ought to be a noop on x86, so how on Earth could a4b6a77b break anything?
Secondly, although memcmp is a macro, it is #undef'd before the definition in lib/string.c, which is the one that is exported.
I'm wondering if the real culprit isn't b92021b0, but I'm looking into it now.
-hpa
H. Peter Anvin
reviewerRe: [PATCH] x86_32: Fix module version table mismatch.
In reply to Tetsuo HandaQuoted 2 lines · Tetsuo Handa
Commit
a4b6a77b"module: fix symbol versioning with symbol prefixes" broke loading ofnet/ipv6/ipv6.kobuilt withCONFIG_MODVERSIONS=yforx86_32.
This really does seem to be the offending commit, although I'm still confused how the heck that is possible.
-hpa
H. Peter Anvin
reviewerRe: [PATCH] x86_32: Fix module version table mismatch.
Quoted 7 lines · H. Peter Anvin
Quoted 2 lines · Tetsuo Handa
Commit
a4b6a77b"module: fix symbol versioning with symbol prefixes" broke loading ofnet/ipv6/ipv6.kobuilt withCONFIG_MODVERSIONS=yforx86_32.This really does seem to be the offending commit, although I'm still confused how the heck that is possible.
OK, now I grok.
The bug is the use of VMLINUX_SYMBOL_STR(%s) which expands at the time the output of modpost is compiled. However, VMLINUX_SYMBOL_STR() unlike __VMLINUX_SYMBOL_STR() does macro expansion on its argument, which is actively wrong here.
I think the choice is either to change this to __VMLINUX_SYMBOL_STR() or re-introduce CONFIG_SYMBOL_PREFIX (or its equivalent) so that modprobe can emit it at compile time (assuming there even should *be* a prefix on the symbol here, i.e. that the compiler won't add it.)
Either way -- James, Rusty, this is in your court.
-hpa
Re: [PATCH] x86_32: Fix module version table mismatch.
Quoted 9 lines · H. Peter Anvin
Quoted 7 lines · H. Peter Anvin
Quoted 2 lines · Tetsuo Handa
Commit
a4b6a77b"module: fix symbol versioning with symbol prefixes" broke loading ofnet/ipv6/ipv6.kobuilt withCONFIG_MODVERSIONS=yforx86_32.This really does seem to be the offending commit, although I'm still confused how the heck that is possible.
Quoted 6 lines
OK, now I grok.
The bug is the use of
VMLINUX_SYMBOL_STR(%s)which expands at the time the output of modpost is compiled. However,VMLINUX_SYMBOL_STR()unlike__VMLINUX_SYMBOL_STR()does macro expansion on its argument, which is actively wrong here.
Yes, nasty bug there (sorry!)
Quoted 3 lines
I think the choice is either to change this to
__VMLINUX_SYMBOL_STR()or re-introduceCONFIG_SYMBOL_PREFIX(or its equivalent) so that modprobe can emit it at compile time
Using __VMLINUX_SYMBOL_STR looks like the correct solution to me.
Quoted 2 lines
(assuming there even should *be* a prefix on the symbol here, i.e. that the compiler won't add it.)
[__]VMLINUX_SYMBOL_STR expands to a string (e.g. "_" "memcmp" or just "memcmp") so the compiler won't touch it.
Quoted 2 lines
Either way -- James, Rusty, this is in your court.
How does the patch below look? I presume this is preferred over making VMLINUX_SYMBOL_STR non-argument-expanding?
Thanks
James
Subject: [PATCH 1/1] modpost: fix unwanted VMLINUX_SYMBOL_STR expansion
Commit a4b6a77b77ba4f526392612c2365797fab956014 ("module: fix symbol versioning with symbol prefixes") broke the MODVERSIONS loading of any module using memcmp (e.g. ipv6) on x86_32, as it's defined to __builtin_memcmp which is expanded by VMLINUX_SYMBOL_STR. Use __VMLINUX_SYMBOL_STR instead which doesn't expand the argument.
--- scripts/mod/modpost.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/mod/modpost.c +1 -1
@@ -1957,7 +1957,7 @@ static int add_versions(struct buffer *b, struct module *mod) s->name, mod->name);continue ; } -buf_printf (b,"\t{ %#8x, VMLINUX_SYMBOL_STR(%s) },\n" , +buf_printf (b,"\t{ %#8x, __VMLINUX_SYMBOL_STR(%s) },\n" , s->crc, s->name); }
Signature · 1 line
1.8.1.2
Re: [PATCH] x86_32: Fix module version table mismatch.
Quoted 7 lines · James Hogan
Quoted 3 lines · H. Peter Anvin
On 04/23/2013 05:52 PM, H. Peter Anvin wrote:
Either way -- James, Rusty, this is in your court.
How does the patch below look? I presume this is preferred over making
VMLINUX_SYMBOL_STRnon-argument-expanding?
I will test it soon.
Quoted 9 lines
Subject: [PATCH 1/1] modpost: fix unwanted
VMLINUX_SYMBOL_STRexpansionCommit
a4b6a77b77ba4f526392612c2365797fab956014("module: fix symbol versioning with symbol prefixes") broke the MODVERSIONS loading of any module using memcmp (e.g. ipv6) onx86_32, as it's defined to__builtin_memcmpwhich is expanded byVMLINUX_SYMBOL_STR. Use__VMLINUX_SYMBOL_STRinstead which doesn't expand the argument.Reported-by Tetsuo Handa <[email protected]>
To be strongly correct Tetsuo and me reported about it and Tetsuo did first analysis.
Quoted 3 lines
Signed-off-by James Hogan <[email protected]>Cc Rusty Russell <[email protected]>Cc Andy Shevchenko <[email protected]>
--
With Best Regards,
Andy Shevchenko
Re: [PATCH] x86_32: Fix module version table mismatch.
In reply to James HoganQuoted 6 lines · James Hogan
How does the patch below look? I presume this is preferred over making
VMLINUX_SYMBOL_STRnon-argument-expanding?Thanks
James
Quoted 8 lines
Subject: [PATCH 1/1] modpost: fix unwanted
VMLINUX_SYMBOL_STRexpansionCommit
a4b6a77b77ba4f526392612c2365797fab956014("module: fix symbol versioning with symbol prefixes") broke the MODVERSIONS loading of any module using memcmp (e.g. ipv6) onx86_32, as it's defined to__builtin_memcmpwhich is expanded byVMLINUX_SYMBOL_STR. Use__VMLINUX_SYMBOL_STRinstead which doesn't expand the argument.
The patch solves this problem. Thank you.
Andy Shevchenko
Tested-byRe: [PATCH] x86_32: Fix module version table mismatch.
In reply to James HoganIt works
Quoted 73 lines · James Hogan
Quoted 9 lines · H. Peter Anvin
Quoted 7 lines · H. Peter Anvin
Quoted 2 lines · Tetsuo Handa
Commit
a4b6a77b"module: fix symbol versioning with symbol prefixes" broke loading ofnet/ipv6/ipv6.kobuilt withCONFIG_MODVERSIONS=yforx86_32.This really does seem to be the offending commit, although I'm still confused how the heck that is possible.
Quoted 6 lines
OK, now I grok.
The bug is the use of
VMLINUX_SYMBOL_STR(%s)which expands at the time the output of modpost is compiled. However,VMLINUX_SYMBOL_STR()unlike__VMLINUX_SYMBOL_STR()does macro expansion on its argument, which is actively wrong here.Yes, nasty bug there (sorry!)
Quoted 3 lines
I think the choice is either to change this to
__VMLINUX_SYMBOL_STR()or re-introduceCONFIG_SYMBOL_PREFIX(or its equivalent) so that modprobe can emit it at compile timeUsing
__VMLINUX_SYMBOL_STRlooks like the correct solution to me.Quoted 2 lines
(assuming there even should *be* a prefix on the symbol here, i.e. that the compiler won't add it.)
[__]VMLINUX_SYMBOL_STR expands to a string (e.g. "_" "memcmp" or just "memcmp") so the compiler won't touch it.
Quoted 2 lines
Either way -- James, Rusty, this is in your court.
How does the patch below look? I presume this is preferred over making
VMLINUX_SYMBOL_STRnon-argument-expanding?Thanks
JamesSubject: [PATCH 1/1] modpost: fix unwanted
VMLINUX_SYMBOL_STRexpansionCommit
a4b6a77b77ba4f526392612c2365797fab956014("module: fix symbol versioning with symbol prefixes") broke the MODVERSIONS loading of any module using memcmp (e.g. ipv6) onx86_32, as it's defined to__builtin_memcmpwhich is expanded byVMLINUX_SYMBOL_STR. Use__VMLINUX_SYMBOL_STRinstead which doesn't expand the argument.Reported-by Tetsuo Handa <[email protected]>Signed-off-by James Hogan <[email protected]>Cc Rusty Russell <[email protected]>Cc Andy Shevchenko <[email protected]>Cc H. Peter Anvin <[email protected]>--- scripts/mod/modpost.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)scripts/mod/modpost.c +1
-2@@ -1957,7 +1957,7 @@ static int add_versions(struct buffer *b, struct module *mod) s->name, mod->name);continue ; } -buf_printf (b,"\t{ %#8x, VMLINUX_SYMBOL_STR(%s) },\n" , +buf_printf (b,"\t{ %#8x, __VMLINUX_SYMBOL_STR(%s) },\n" , s->crc, s->name); } --1.8.1.2
Signature · 2 lines
With Best Regards, Andy Shevchenko
H. Peter Anvin
reviewerRe: [PATCH] x86_32: Fix module version table mismatch.
In reply to James HoganOK, I assume Rusty will pick this up in his tree before pushing it to Linus.
-hpa
PATCH v2 1/1 modpost: fix unwanted VMLINUX_SYMBOL_STR expansion
Commit a4b6a77b77ba4f526392612c2365797fab956014 ("module: fix symbol versioning with symbol prefixes") broke the MODVERSIONS loading of any module using memcmp (e.g. ipv6) on x86_32, as it's defined to __builtin_memcmp which is expanded by VMLINUX_SYMBOL_STR. Use __VMLINUX_SYMBOL_STR instead which doesn't expand the argument.
---
I've corrected commit message tags for Rusty's convenience. Updated Reported-by to include Andy
Taken the liberty of adding Tetsuo's Tested-by
Cheers
James
scripts/mod/modpost.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/mod/modpost.c +1 -1
@@ -1957,7 +1957,7 @@ static int add_versions(struct buffer *b, struct module *mod) s->name, mod->name);continue ; } -buf_printf (b,"\t{ %#8x, VMLINUX_SYMBOL_STR(%s) },\n" , +buf_printf (b,"\t{ %#8x, __VMLINUX_SYMBOL_STR(%s) },\n" , s->crc, s->name); }
Signature · 1 line
1.8.1.2
Rusty Russell
AppliedRe: [PATCH v2] modpost: fix unwanted VMLINUX_SYMBOL_STR expansion
Quoted 13 lines · James Hogan
Commit
a4b6a77b77ba4f526392612c2365797fab956014("module: fix symbol versioning with symbol prefixes") broke the MODVERSIONS loading of any module using memcmp (e.g. ipv6) onx86_32, as it's defined to__builtin_memcmpwhich is expanded byVMLINUX_SYMBOL_STR. Use__VMLINUX_SYMBOL_STRinstead which doesn't expand the argument.Reported-by Tetsuo Handa <[email protected]>Reported-by Andy Shevchenko <[email protected]>Signed-off-by James Hogan <[email protected]>Cc Rusty Russell <[email protected]>Cc H. Peter Anvin <[email protected]>Tested-by Tetsuo Handa <[email protected]>Tested-by Andy Shevchenko <[email protected]>
Quoted 5 lines
---
I've corrected commit message tags for Rusty's convenience. Updated Reported-by to include Andy
Taken the liberty of adding Tetsuo's Tested-by
Applied.
Thanks especially for chasing this: I was away for a few days last week.
Cheers,
Rusty.
H. Peter Anvin
reviewerQuoted 23 lines · Tetsuo Handa
Quoted 1 line · Andy Shevchenko
What did I miss?
Well, as of linux-next-20130422,
memcmp()is not correctly exported to modules. Since linux-3.9-rc8 correctly exportsmemcmp(), this problem seems to be introduced in linux-next tree. Also, this problem seems to involveCONFIG_MODVERSIONS=y.[root@localhost linux-next]# modprobe ipv6
FATAL: Error inserting ipv6 (/lib/modules/3.9.0-rc8-next-20130422/kernel/net/ipv6/ipv6.ko): Invalid argument [root@localhost linux-next]# dmesg
ipv6: no symbol version for memcmp
ipv6: Unknown symbol memcmp (err -22)Since
arch/x86/include/asm/string_64.husesint memcmp(const void *cs, const void *ct, size_t count);while
arch/x86/include/asm/string_32.huses#define memcmp __builtin_memcmpchanging to what you have tried
Quoted 8 lines
#define memcmp(a, b, n) __builtin_memcmp(a, b, n)or changing to what
x86_64doesint memcmp(const void *cs, const void *ct, size_t count);might solve this problem. But I don't know which one is correct solution...
Rusty, this seems like a problem with your changes to the prefix handling. Somehow memcmp being a macro gets picked up by some, but not all, of the module-metadata generation tools.
Changing memcmp to a macro with arguments (see above) seems to paper over the problem for this one, but there seems to be something much more sinister going on and I would really like a good explanation as I fear this can crop up in other places.
-hpa