Skip to contentSkip to conversation

Kmod optimization

PATCH In review [email protected] 6 participants 12 messages Apr 13, 2006 – Apr 14, 2006 Module support

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Kmod optimization

Hi,

the request_mod functions try to load automatically a module by running a user mode process helper (modprobe).

The user process is launched even if the module is already loaded. I think it would be better to test if the module is already loaded.

Signature · 2 lines
tyler
[email protected]
include/linux/module.h +1 -0
diff -uprN -X linux-2.6.16.5/Documentation/dontdiff linux-2.6.16.5/include/linux/module.h linux-2.6.16.5-tyler/include/linux/module.h
@@ -338,6 +338,7 @@ struct module *module_get_kallsym(unsign
 unsigned long module_kallsyms_lookup_name(const char *name);
 
 int is_exported(const char *name, const struct module *mod);
+int is_loaded(const char *module_name);
 
 extern void __module_put_and_exit(struct module *mod, long code)
 	__attribute__((noreturn));
kernel/kmod.c +12 -0
diff -uprN -X linux-2.6.16.5/Documentation/dontdiff linux-2.6.16.5/kernel/kmod.c linux-2.6.16.5-tyler/kernel/kmod.c
@@ -37,6 +37,7 @@
 #include <linux/kernel.h>
 #include <linux/init.h>
 #include <asm/uaccess.h>
+#include <linux/spinlock.h>
 
 extern int max_threads;
 
@@ -48,6 +49,7 @@ static struct workqueue_struct *khelper_
 	modprobe_path is set via /proc/sys.
 */
 char modprobe_path[KMOD_PATH_LEN] = "/sbin/modprobe";
+extern spinlock_t modlist_lock;
 
 /**
  * request_module - try to load a kernel module
@@ -77,6 +79,7 @@ int request_module(const char *fmt, ...)
 	static atomic_t kmod_concurrent = ATOMIC_INIT(0);
 #define MAX_KMOD_CONCURRENT 50	/* Completely arbitrary value - KAO */
 	static int kmod_loop_msg;
+	unsigned long flags;
 
 	va_start(args, fmt);
 	ret = vsnprintf(module_name, MODULE_NAME_LEN, fmt, args);
@@ -84,6 +87,15 @@ int request_module(const char *fmt, ...)
 	if (ret >= MODULE_NAME_LEN)
 		return -ENAMETOOLONG;
 
+	/* We don't to load the module if it's already loaded */
+	spin_lock_irqsave(&modlist_lock, flags);
+	if (is_loaded(module_name)) {
+		return -EEXIST;
+	}
+	spin_unlock_irqrestore(&modlist_lock, flags);
+
+
+
 	/* If modprobe needs a service that is in a module, we get a recursive
 	 * loop.  Limit the number of running kmod threads to max_threads/2 or
 	 * MAX_KMOD_CONCURRENT, whichever is the smaller.  A cleaner method
kernel/module.c +13 -0
diff -uprN -X linux-2.6.16.5/Documentation/dontdiff linux-2.6.16.5/kernel/module.c linux-2.6.16.5-tyler/kernel/module.c
@@ -212,6 +212,19 @@ static struct module *find_module(const 
 	return NULL;
 }
 
+/* Test if a module is loaded : must hold module_mutex */
+int is_loaded(const char *module_name);
+{
+	struct module *mod = find_module(module_name);
+
+	if (!mod) {
+		return 1;
+	}
+
+	return 0;
+}
+
+
 #ifdef CONFIG_SMP
 /* Number of blocks used and allocated. */
 static unsigned int pcpu_num_used, pcpu_num_allocated;
#1 lore raw
Quoted 7 lines · On Thu, 13 Apr 2006 20:03:45 +0200 [email protected] wrote:

Hi,

the request_mod functions try to load automatically a module by running a user mode process helper (modprobe).

The user process is launched even if the module is already loaded. I think it would be better to test if the module is already loaded.

Please try not to use attachments: it makes it more difficult to review and comment on code (so I'll paste it here).

+ /* We don't to load the module if it's already loaded */
+ spin_lock_irqsave(&modlist_lock, flags);
+ if (is_loaded(module_name)) {
+ return -EEXIST;
+ }
+ spin_unlock_irqrestore(&modlist_lock, flags);

Need to do spin_unlock_irqrestore() even if is_loaded() is true.

+/* Test if a module is loaded : must hold module_mutex */ +int is_loaded(const char *module_name);
+{
+ struct module *mod = find_module(module_name);
+
+ if (!mod) {
+ return 1;
+ }

Don't use braces when not needed.
Why not make this function inline and put it into a header file?

---
~Randy

#2 lore raw

+/* Test if a module is loaded : must hold module_mutex */ +int is_loaded(const char *module_name);
+{
+ struct module *mod = find_module(module_name);
+
+ if (!mod) {
+ return 1;
+ }
+
+ return 0;
+}
+
+

Quoted 2 lines

Don't use braces when not needed.
Why not make this function inline and put it into a header file?

static inline int is_loaded(const char *module_name) {
    return find_module(module_name) != NULL;
}

Cheers. In that case, I think it can even be "hand-inlined" into the callers.

Jan Engelhardt

#3 lore raw
In reply to [email protected]
Quoted 7 lines · On Thu, Apr 13, 2006 at 08:03:45PM +0200, [email protected] wrote:

Hi,

the request_mod functions try to load automatically a module by running a user mode process helper (modprobe).

The user process is launched even if the module is already loaded. I think it would be better to test if the module is already loaded.

Does this cause a problem somehow? request_mod is called _very_ infrequently from a normal kernel these days, so I really don't think this is necessary.

thanks,

greg k-h

#4 lore raw
Quoted 6 lines · Greg KH
Quoted 5 lines · On Thu, Apr 13, 2006 at 08:03:45PM +0200, [email protected] wrote:

Hi,

the request_mod functions try to load automatically a module by running a user mode process helper (modprobe).

Quoted 6 lines
Quoted 2 lines

The user process is launched even if the module is already loaded. I think it would be better to test if the module is already loaded.

Does this cause a problem somehow? request_mod is called _very_ infrequently from a normal kernel these days, so I really don't think this is necessary.

Yes I agree it _should_ be very infrequently called but it _will_ be very infrequently called just if the user space configuration is done properly.

I personnaly think we shouldn't trust the configuration and the way the different modules are loaded.

Well anyway, I've corrected the patch regarding the advices of randy.

include/linux/module.h +13 -0
diff -uprN -X linux-2.6.16.5/Documentation/dontdiff linux-2.6.16.5/include/linux/module.h linux-2.6.16.5-tyler/include/linux/module.h
@@ -339,6 +339,19 @@ unsigned long module_kallsyms_lookup_nam
 
 int is_exported(const char *name, const struct module *mod);
 
+/* Test if a module is loaded : must hold module_mutex */
+inline int is_loaded(const char *module_name)
+{
+	struct module *mod = find_module(module_name);
+
+	if (!mod)
+		return 1;
+
+	return 0;
+}
+
+
+
 extern void __module_put_and_exit(struct module *mod, long code)
 	__attribute__((noreturn));
 #define module_put_and_exit(code) __module_put_and_exit(THIS_MODULE, code);
kernel/kmod.c +13 -0
diff -uprN -X linux-2.6.16.5/Documentation/dontdiff linux-2.6.16.5/kernel/kmod.c linux-2.6.16.5-tyler/kernel/kmod.c
@@ -37,6 +37,7 @@
 #include <linux/kernel.h>
 #include <linux/init.h>
 #include <asm/uaccess.h>
+#include <linux/spinlock.h>
 
 extern int max_threads;
 
@@ -48,6 +49,7 @@ static struct workqueue_struct *khelper_
 	modprobe_path is set via /proc/sys.
 */
 char modprobe_path[KMOD_PATH_LEN] = "/sbin/modprobe";
+extern spinlock_t modlist_lock;
 
 /**
  * request_module - try to load a kernel module
@@ -77,6 +79,7 @@ int request_module(const char *fmt, ...)
 	static atomic_t kmod_concurrent = ATOMIC_INIT(0);
 #define MAX_KMOD_CONCURRENT 50	/* Completely arbitrary value - KAO */
 	static int kmod_loop_msg;
+	unsigned long flags;
 
 	va_start(args, fmt);
 	ret = vsnprintf(module_name, MODULE_NAME_LEN, fmt, args);
@@ -84,6 +87,16 @@ int request_module(const char *fmt, ...)
 	if (ret >= MODULE_NAME_LEN)
 		return -ENAMETOOLONG;
 
+	/* We don't to load the module if it's already loaded */
+	spin_lock_irqsave(&modlist_lock, flags);
+	if (is_loaded(module_name)) {
+		spin_unlock_irqrestore(&modlist_lock, flags);
+		return -EEXIST;
+	}
+	spin_unlock_irqrestore(&modlist_lock, flags);
+
+
+
 	/* If modprobe needs a service that is in a module, we get a recursive
 	 * loop.  Limit the number of running kmod threads to max_threads/2 or
 	 * MAX_KMOD_CONCURRENT, whichever is the smaller.  A cleaner method

___________________________________________________________________________
Faites de Yahoo! votre page d'accueil sur le web pour retrouver directement vos services préférés : vérifiez vos nouveaux mails, lancez vos recherches et suivez l'actualité en temps réel. Rendez-vous sur http://fr.yahoo.com/set

#5 lore raw
Quoted 10 lines · On Thu, Apr 13, 2006 at 08:36:17PM +0200, [email protected] wrote:
Quoted 9 lines · Greg KH
Quoted 7 lines · On Thu, Apr 13, 2006 at 08:03:45PM +0200, [email protected] wrote:

Hi,

the request_mod functions try to load automatically a module by running a user mode process helper (modprobe).

The user process is launched even if the module is already loaded. I think it would be better to test if the module is already loaded.

Quoted 6 lines
Quoted 3 lines

Does this cause a problem somehow? request_mod is called _very_ infrequently from a normal kernel these days, so I really don't think this is necessary.

Yes I agree it _should_ be very infrequently called but it _will_ be very infrequently called just if the user space configuration is done properly.

What do you mean by this? Almost all 2.6 distros use udev today, which prevents this code from ever getting called. So odds are, you are optimising something that no one will ever use :)

Quoted 2 lines

I personnaly think we shouldn't trust the configuration and the way the different modules are loaded.

What way different modules are loaded? What problem are you referring to here?

thanks,

greg k-h

#6 lore raw
Quoted 15 lines · Greg KH
Quoted 14 lines · On Thu, Apr 13, 2006 at 08:36:17PM +0200, [email protected] wrote:
Quoted 12 lines · Greg KH
Quoted 7 lines · On Thu, Apr 13, 2006 at 08:03:45PM +0200, [email protected] wrote:

Hi,

the request_mod functions try to load automatically a module by running a user mode process helper (modprobe).

The user process is launched even if the module is already loaded. I think it would be better to test if the module is already loaded.

Does this cause a problem somehow? request_mod is called _very_ infrequently from a normal kernel these days, so I really don't think this is necessary.

Quoted 6 lines
Quoted 2 lines

Yes I agree it _should_ be very infrequently called but it _will_ be very infrequently called just if the user space configuration is done properly.

What do you mean by this? Almost all 2.6 distros use udev today, which prevents this code from ever getting called. So odds are, you are optimising something that no one will ever use :)

Well perhaps I don't understand the mechanism :) But let's take an example.
On all kernels (even recent), if the module smbfs is loaded, it's not handled by udev and request_module could be called.

Let"s take another example to see to illustrate why I think it depends on the user configuration :
module A depends on module B

if we have a script which do "insmod moduleA.ko ; insmod moduleB.ko", there will be a call to request_module.

if the script is "insmod moduleB.ko ; insmod moduleA.ko", request_mode is not called.

I know the first script is really idiot :)

This is what I was thinking about in my previous mail.

Signature · 11 lines
tyler
[email protected]


	

	
		
___________________________________________________________________________ 
Faites de Yahoo! votre page d'accueil sur le web pour retrouver directement vos services préférés : vérifiez vos nouveaux mails, lancez vos recherches et suivez l'actualité en temps réel. 
Rendez-vous sur http://fr.yahoo.com/set
#7 lore raw

Hi,

Quoted 4 lines · On Thu, 13 Apr 2006, [email protected] wrote:

Well perhaps I don't understand the mechanism :) But let's take an example.
On all kernels (even recent), if the module smbfs is loaded, it's not handled by udev and request_module could be called.

No, it can't. If the smbfs is loaded, get_fs_type() will find it and won't even try to load it.
Do you a real example, where this is a problem?

bye, Roman

#8 lore raw
In reply to [email protected]
Quoted 27 lines · On Thu, Apr 13, 2006 at 09:04:12PM +0200, [email protected] wrote:
Quoted 15 lines · Greg KH
Quoted 14 lines · On Thu, Apr 13, 2006 at 08:36:17PM +0200, [email protected] wrote:
Quoted 12 lines · Greg KH
Quoted 7 lines · On Thu, Apr 13, 2006 at 08:03:45PM +0200, [email protected] wrote:

Hi,

the request_mod functions try to load automatically a module by running a user mode process helper (modprobe).

The user process is launched even if the module is already loaded. I think it would be better to test if the module is already loaded.

Does this cause a problem somehow? request_mod is called _very_ infrequently from a normal kernel these days, so I really don't think this is necessary.

Quoted 6 lines
Quoted 2 lines

Yes I agree it _should_ be very infrequently called but it _will_ be very infrequently called just if the user space configuration is done properly.

What do you mean by this? Almost all 2.6 distros use udev today, which prevents this code from ever getting called. So odds are, you are optimising something that no one will ever use :)

Well perhaps I don't understand the mechanism :) But let's take an example.
On all kernels (even recent), if the module smbfs is loaded, it's not handled by udev and request_module could be called.

Quoted 6 lines

Let"s take another example to see to illustrate why I think it depends on the user configuration :
module A depends on module B

if we have a script which do "insmod moduleA.ko ; insmod moduleB.ko", there will be a call to request_module.

No, that script will fail. Try it out :)

The kernel will not call out to try to resolve the symbols, that's up to userspace to handle. Hint, try running 'modprobe moduleA.ko' instead, it will handle the dependancies correctly.

I still don't see where this is really needed...

thanks,

greg k-h

#9 lore raw
Quoted 9 lines · Greg KH

The kernel will not call out to try to resolve the symbols, that's up to userspace to handle. Hint, try running 'modprobe moduleA.ko' instead, it will handle the dependancies correctly.

I still don't see where this is really needed...

thanks,

greg k-h

I meant, the purpose of request_module is to handle dependencies from the kernel : if we need a functionnality, we request it. It's the definition of dependency.

But I understood the mechanism by reading some request_module examples. In fact, we test the avalaibility of the functionnality before calling the request_module function.

So the case described in my previous message would normally never happen.

if (!functionnality_not_avalaible) {
	module_request(functionnality)

	if (!functionnality_not_avalaible)
	    goto error;
}

Sorry, so the discussion is closed : forget it :)

Signature · 11 lines
tyler
[email protected]


	

	
		
___________________________________________________________________________ 
Faites de Yahoo! votre page d'accueil sur le web pour retrouver directement vos services préférés : vérifiez vos nouveaux mails, lancez vos recherches et suivez l'actualité en temps réel. 
Rendez-vous sur http://fr.yahoo.com/set
#10 lore raw
In reply to [email protected]

include/linux/module.h unsigned long module_kallsyms_lookup_nam · line 348

Quoted 8 lines · [email protected] wrote:
include/linux/module.h +7 -0
+/* Test if a module is loaded : must hold module_mutex */
+inline int is_loaded(const char *module_name)
+{
+	struct module *mod = find_module(module_name);
+
+	if (!mod)
+		return 1;

FWIW, this logic seems reversed....

Signature · 4 lines
Paulo Marques - www.grupopie.com

Pointy-Haired Boss: I don't see anything that could stand in our way.
           Dilbert: Sanity? Reality? The laws of physics?
#11 lore raw
Quoted 9 lines · Paulo Marques

Review

Quoted 8 lines · [email protected] wrote:
(quoted diff) +7 -0
+/* Test if a module is loaded : must hold module_mutex */
+inline int is_loaded(const char *module_name)
+{
+	struct module *mod = find_module(module_name);
+
+	if (!mod)
+		return 1;

For sure it is. Sorry, my mistake :)

include/linux/module.h +13 -0
diff -uprN -X linux-2.6.16.5/Documentation/dontdiff linux-2.6.16.5/include/linux/module.h linux-2.6.16.5-tyler/include/linux/module.h
@@ -339,6 +339,19 @@ unsigned long module_kallsyms_lookup_nam
 
 int is_exported(const char *name, const struct module *mod);
 
+/* Test if a module is loaded : must hold module_mutex */
+inline int is_loaded(const char *module_name)
+{
+	struct module *mod = find_module(module_name);
+
+	if (!mod)
+		return 0;
+
+	return 1;
+}
+
+
+
 extern void __module_put_and_exit(struct module *mod, long code)
 	__attribute__((noreturn));
 #define module_put_and_exit(code) __module_put_and_exit(THIS_MODULE, code);
kernel/kmod.c +13 -0
diff -uprN -X linux-2.6.16.5/Documentation/dontdiff linux-2.6.16.5/kernel/kmod.c linux-2.6.16.5-tyler/kernel/kmod.c
@@ -37,6 +37,7 @@
 #include <linux/kernel.h>
 #include <linux/init.h>
 #include <asm/uaccess.h>
+#include <linux/spinlock.h>
 
 extern int max_threads;
 
@@ -48,6 +49,7 @@ static struct workqueue_struct *khelper_
 	modprobe_path is set via /proc/sys.
 */
 char modprobe_path[KMOD_PATH_LEN] = "/sbin/modprobe";
+extern spinlock_t modlist_lock;
 
 /**
  * request_module - try to load a kernel module
@@ -77,6 +79,7 @@ int request_module(const char *fmt, ...)
 	static atomic_t kmod_concurrent = ATOMIC_INIT(0);
 #define MAX_KMOD_CONCURRENT 50	/* Completely arbitrary value - KAO */
 	static int kmod_loop_msg;
+	unsigned long flags;
 
 	va_start(args, fmt);
 	ret = vsnprintf(module_name, MODULE_NAME_LEN, fmt, args);
@@ -84,6 +87,16 @@ int request_module(const char *fmt, ...)
 	if (ret >= MODULE_NAME_LEN)
 		return -ENAMETOOLONG;
 
+	/* We don't to load the module if it's already loaded */
+	spin_lock_irqsave(&modlist_lock, flags);
+	if (is_loaded(module_name)) {
+		spin_unlock_irqrestore(&modlist_lock, flags);
+		return -EEXIST;
+	}
+	spin_unlock_irqrestore(&modlist_lock, flags);
+
+
+
 	/* If modprobe needs a service that is in a module, we get a recursive
 	 * loop.  Limit the number of running kmod threads to max_threads/2 or
 	 * MAX_KMOD_CONCURRENT, whichever is the smaller.  A cleaner method

___________________________________________________________________________
Faites de Yahoo! votre page d'accueil sur le web pour retrouver directement vos services préférés : vérifiez vos nouveaux mails, lancez vos recherches et suivez l'actualité en temps réel. Rendez-vous sur http://fr.yahoo.com/set