add devname module aliases to allow module on-demand auto-loading
Subsystems (8)
From: Kay Sievers <[email protected]>
Subject: add devname module aliases to allow module on-demand auto-loading
This adds:
alias: devname:<name>
to some common kernel modules, which will allow the on-demand loading of the kernel module when the device node is accessed.
Ideally all these modules would be compiled-in, but distros seems too much in love with their modularization that we need to cover the common cases with this new facility. It will allow us to remove a bunch of pretty useless init scripts and modprobes from init scripts.
The static device node aliases will be carried in the module itself. The program depmod will extract this information to a file in the module directory:
$ cat /lib/modules/2.6.34-00650-g537b60d-dirty/modules.devname # Device nodes to trigger on-demand module loading. microcode cpu/microcode c10:184 fuse fuse c10:229 ppp_generic ppp c108:0 tun net/tun c10:200 dm_mod mapper/control c10:235
Udev will pick up the depmod created file on startup and create all the static device nodes which the kernel modules specify, so that these modules get automatically loaded when the device node is accessed:
$ /sbin/udevd --debug ... static_dev_create_from_modules: mknod '/dev/cpu/microcode' c10:184 static_dev_create_from_modules: mknod '/dev/fuse' c10:229 static_dev_create_from_modules: mknod '/dev/ppp' c108:0 static_dev_create_from_modules: mknod '/dev/net/tun' c10:200 static_dev_create_from_modules: mknod '/dev/mapper/control' c10:235 udev_rules_apply_static_dev_perms: chmod '/dev/net/tun' 0666 udev_rules_apply_static_dev_perms: chmod '/dev/fuse' 0666
A few device nodes are switched to statically allocated numbers, to allow the static nodes to work. This might also useful for systems which still run a plain static /dev, which is completely unsafe to use with any dynamic minor numbers.
Note:
The devname aliases must be limited to the *common* and *single*instance* device nodes, like the misc devices, and never be used for conceptually limited systems like the loop devices, which should rather get fixed properly and get a control node for losetup to talk to, instead of creating a random number of device nodes in advance, regardless if they are ever used.
This facility is to hide the mess distros are creating with too modualized kernels, and just to hide that these modules are not compiled-in, and not to paper-over broken concepts. Thanks! :)
---
arch/x86/kernel/microcode_core.c | 1 + drivers/md/dm-ioctl.c | 5 ++++- drivers/net/ppp_generic.c | 4 ++-- drivers/net/tun.c | 1 + fs/autofs4/dev-ioctl.c | 5 ++++- fs/btrfs/super.c | 5 ++++- fs/fuse/dev.c | 1 + include/linux/miscdevice.h | 3 +++ 8 files changed, 20 insertions(+), 5 deletions(-)
arch/x86/kernel/microcode_core.c +1 -0
@@ -260,6 +260,7 @@ static void microcode_dev_exit(void) }MODULE_ALIAS_MISCDEV (MICROCODE_MINOR ); +MODULE_ALIAS ("devname:cpu/microcode" );#else #define microcode_dev_init () 0#define microcode_dev_exit () do { } while (0)
drivers/md/dm-ioctl.c +4 -1
@@ -1599,12 +1599,15 @@ static const struct file_operations _ctl };static struct miscdevice _dm_misc = { - .minor =MISC_DYNAMIC_MINOR , + .minor =MAPPER_CTRL_MINOR , .name =DM_NAME , .nodename ="mapper/control" , .fops = &_ctl_fops }; +MODULE_ALIAS_MISCDEV (MAPPER_CTRL_MINOR ); +MODULE_ALIAS ("devname:mapper/control" ); +/* * Create misc character device and link to DM_DIR/control. */
drivers/net/ppp_generic.c +2 -2
@@ -2907,5 +2907,5 @@ EXPORT_SYMBOL(ppp_output_wakeup);EXPORT_SYMBOL (ppp_register_compressor);EXPORT_SYMBOL (ppp_unregister_compressor);MODULE_LICENSE ("GPL" ); -MODULE_ALIAS_CHARDEV_MAJOR (PPP_MAJOR ); -MODULE_ALIAS ("/dev/ppp" ); +MODULE_ALIAS_CHARDEV (PPP_MAJOR ,0 ); +MODULE_ALIAS ("devname:ppp" );
drivers/net/tun.c +1 -0
@@ -1624,3 +1624,4 @@ MODULE_DESCRIPTION(DRV_DESCRIPTION);MODULE_AUTHOR (DRV_COPYRIGHT );MODULE_LICENSE ("GPL" );MODULE_ALIAS_MISCDEV (TUN_MINOR ); +MODULE_ALIAS ("devname:net/tun" );
fs/autofs4/dev-ioctl.c +4 -1
@@ -736,11 +736,14 @@ static const struct file_operations _dev };static struct miscdevice _autofs_dev_ioctl_misc = { - .minor =MISC_DYNAMIC_MINOR , + .minor =AUTOFS_MINOR , .name =AUTOFS_DEVICE_NAME , .fops = &_dev_ioctl_fops }; +MODULE_ALIAS_MISCDEV (BTRFS_MINOR ); +MODULE_ALIAS ("devname:autofs" ); +/* Register/deregister misc character device */ int autofs_dev_ioctl_init (void ) {
fs/btrfs/super.c +4 -1
@@ -832,11 +832,14 @@ static const struct file_operations btrf };static struct miscdevice btrfs_misc = { - .minor =MISC_DYNAMIC_MINOR , + .minor =BTRFS_MINOR , .name ="btrfs-control" , .fops = &btrfs_ctl_fops }; +MODULE_ALIAS_MISCDEV (BTRFS_MINOR ); +MODULE_ALIAS ("devname:btrfs-control" ); +static int btrfs_interface_init (void ) {return misc_register (&btrfs_misc);
fs/fuse/dev.c +1 -0
@@ -18,6 +18,7 @@#include <linux/slab.h> MODULE_ALIAS_MISCDEV (FUSE_MINOR ); +MODULE_ALIAS ("devname:fuse" );static struct kmem_cache *fuse_req_cachep;
include/linux/miscdevice.h +3 -0
@@ -31,6 +31,9 @@#define FUSE_MINOR 229#define KVM_MINOR 232#define VHOST_NET_MINOR 233 +#define BTRFS_MINOR 234 +#define MAPPER_CTRL_MINOR 235 +#define AUTOFS_MINOR 236#define MISC_DYNAMIC_MINOR 255struct device ;
Quoted 8 lines · Kay Sievers
From: Kay Sievers <[email protected]>
Subject: add devname module aliases to allow module on-demand auto-loadingThis adds:
alias: devname:<name>
to some common kernel modules, which will allow the on-demand loading of the kernel module when the device node is accessed.
snip ...
fs/autofs4/dev-ioctl.c static const struct file_operations _dev · line 744
Quoted 13 lines
fs/autofs4/dev-ioctl.c +2
-1@@ -736,11 +736,14 @@ static const struct file_operations _dev };static struct miscdevice _autofs_dev_ioctl_misc = { - .minor =MISC_DYNAMIC_MINOR , + .minor =AUTOFS_MINOR , .name =AUTOFS_DEVICE_NAME , .fops = &_dev_ioctl_fops }; +MODULE_ALIAS_MISCDEV (BTRFS_MINOR );
Is this a typo or am I missing something?
Shouldn't it be:
+MODULE_ALIAS_MISCDEV(AUTOFS_MINOR);
Quoted 47 lines
fs/autofs4/dev-ioctl.c +2
-0+MODULE_ALIAS ("devname:autofs" ); +/* Register/deregister misc character device */ int autofs_dev_ioctl_init (void ) {fs/btrfs/super.c +4
-1@@ -832,11 +832,14 @@ static const struct file_operations btrf };static struct miscdevice btrfs_misc = { - .minor =MISC_DYNAMIC_MINOR , + .minor =BTRFS_MINOR , .name ="btrfs-control" , .fops = &btrfs_ctl_fops }; +MODULE_ALIAS_MISCDEV (BTRFS_MINOR ); +MODULE_ALIAS ("devname:btrfs-control" ); +static int btrfs_interface_init (void ) {return misc_register (&btrfs_misc);fs/fuse/dev.c +1
-0@@ -18,6 +18,7 @@#include <linux/slab.h> MODULE_ALIAS_MISCDEV (FUSE_MINOR ); +MODULE_ALIAS ("devname:fuse" );static struct kmem_cache *fuse_req_cachep;include/linux/miscdevice.h +3
-0@@ -31,6 +31,9 @@#define FUSE_MINOR 229#define KVM_MINOR 232#define VHOST_NET_MINOR 233 +#define BTRFS_MINOR 234 +#define MAPPER_CTRL_MINOR 235 +#define AUTOFS_MINOR 236#define MISC_DYNAMIC_MINOR 255struct device ;
Quoted 6 lines · Ian Kent
Quoted 2 lines · Kay Sievers
---
a/fs/autofs4/dev-ioctl.c
+MODULE_ALIAS_MISCDEV(BTRFS_MINOR);Is this a typo or am I missing something?
Yikes, sorry, that's just broken. New patch. Thanks a lot for checking.
Kay
From: Kay Sievers <[email protected]>
Subject: add devname module aliases to allow module on-demand auto-loading
This adds:
alias: devname:<name>
to some common kernel modules, which will allow the on-demand loading of the kernel module when the device node is accessed.
Ideally all these modules would be compiled-in, but distros seems too much in love with their modularization that we need to cover the common cases with this new facility. It will allow us to remove a bunch of pretty useless init scripts and modprobes from init scripts.
The static device node aliases will be carried in the module itself. The program depmod will extract this information to a file in the module directory:
$ cat /lib/modules/2.6.34-00650-g537b60d-dirty/modules.devname # Device nodes to trigger on-demand module loading. microcode cpu/microcode c10:184 fuse fuse c10:229 ppp_generic ppp c108:0 tun net/tun c10:200 dm_mod mapper/control c10:235
Udev will pick up the depmod created file on startup and create all the static device nodes which the kernel modules specify, so that these modules get automatically loaded when the device node is accessed:
$ /sbin/udevd --debug ... static_dev_create_from_modules: mknod '/dev/cpu/microcode' c10:184 static_dev_create_from_modules: mknod '/dev/fuse' c10:229 static_dev_create_from_modules: mknod '/dev/ppp' c108:0 static_dev_create_from_modules: mknod '/dev/net/tun' c10:200 static_dev_create_from_modules: mknod '/dev/mapper/control' c10:235 udev_rules_apply_static_dev_perms: chmod '/dev/net/tun' 0666 udev_rules_apply_static_dev_perms: chmod '/dev/fuse' 0666
A few device nodes are switched to statically allocated numbers, to allow the static nodes to work. This might also useful for systems which still run a plain static /dev, which is completely unsafe to use with any dynamic minor numbers.
Note:
The devname aliases must be limited to the *common* and *single*instance* device nodes, like the misc devices, and never be used for conceptually limited systems like the loop devices, which should rather get fixed properly and get a control node for losetup to talk to, instead of creating a random number of device nodes in advance, regardless if they are ever used.
This facility is to hide the mess distros are creating with too modualized kernels, and just to hide that these modules are not compiled-in, and not to paper-over broken concepts. Thanks! :)
---
Documentation/devices.txt | 3 +++ arch/x86/kernel/microcode_core.c | 1 + drivers/md/dm-ioctl.c | 5 ++++- drivers/net/ppp_generic.c | 4 ++-- drivers/net/tun.c | 1 + fs/autofs4/dev-ioctl.c | 5 ++++- fs/btrfs/super.c | 5 ++++- fs/fuse/dev.c | 1 + include/linux/miscdevice.h | 3 +++ 9 files changed, 23 insertions(+), 5 deletions(-)
Documentation/devices.txt +3 -0
@@ -443,6 +443,9 @@ Your cooperation is appreciated. 231 = /dev/snapshot System memory snapshot device 232 = /dev/kvm Kernel-based virtual machine (hardware virtualization extensions) 233 = /dev/kmview View-OS A process with a view + 234 = /dev/btrfs-control Btrfs control device + 235 = /dev/mapper/control Device-Mapper control device + 236 = /dev/autofs Autofs control device 240-254 Reserved for local use 255 Reserved for MISC_DYNAMIC_MINOR
arch/x86/kernel/microcode_core.c +1 -0
@@ -260,6 +260,7 @@ static void microcode_dev_exit(void) }MODULE_ALIAS_MISCDEV (MICROCODE_MINOR ); +MODULE_ALIAS ("devname:cpu/microcode" );#else #define microcode_dev_init () 0#define microcode_dev_exit () do { } while (0)
drivers/md/dm-ioctl.c +4 -1
@@ -1599,12 +1599,15 @@ static const struct file_operations _ctl };static struct miscdevice _dm_misc = { - .minor =MISC_DYNAMIC_MINOR , + .minor =MAPPER_CTRL_MINOR , .name =DM_NAME , .nodename ="mapper/control" , .fops = &_ctl_fops }; +MODULE_ALIAS_MISCDEV (MAPPER_CTRL_MINOR ); +MODULE_ALIAS ("devname:mapper/control" ); +/* * Create misc character device and link to DM_DIR/control. */
drivers/net/ppp_generic.c +2 -2
@@ -2907,5 +2907,5 @@ EXPORT_SYMBOL(ppp_output_wakeup);EXPORT_SYMBOL (ppp_register_compressor);EXPORT_SYMBOL (ppp_unregister_compressor);MODULE_LICENSE ("GPL" ); -MODULE_ALIAS_CHARDEV_MAJOR (PPP_MAJOR ); -MODULE_ALIAS ("/dev/ppp" ); +MODULE_ALIAS_CHARDEV (PPP_MAJOR ,0 ); +MODULE_ALIAS ("devname:ppp" );
drivers/net/tun.c +1 -0
@@ -1624,3 +1624,4 @@ MODULE_DESCRIPTION(DRV_DESCRIPTION);MODULE_AUTHOR (DRV_COPYRIGHT );MODULE_LICENSE ("GPL" );MODULE_ALIAS_MISCDEV (TUN_MINOR ); +MODULE_ALIAS ("devname:net/tun" );
fs/autofs4/dev-ioctl.c +4 -1
@@ -736,11 +736,14 @@ static const struct file_operations _dev };static struct miscdevice _autofs_dev_ioctl_misc = { - .minor =MISC_DYNAMIC_MINOR , + .minor =AUTOFS_MINOR , .name =AUTOFS_DEVICE_NAME , .fops = &_dev_ioctl_fops }; +MODULE_ALIAS_MISCDEV (AUTOFS_MINOR ); +MODULE_ALIAS ("devname:autofs" ); +/* Register/deregister misc character device */ int autofs_dev_ioctl_init (void ) {
fs/btrfs/super.c +4 -1
@@ -832,11 +832,14 @@ static const struct file_operations btrf };static struct miscdevice btrfs_misc = { - .minor =MISC_DYNAMIC_MINOR , + .minor =BTRFS_MINOR , .name ="btrfs-control" , .fops = &btrfs_ctl_fops }; +MODULE_ALIAS_MISCDEV (BTRFS_MINOR ); +MODULE_ALIAS ("devname:btrfs-control" ); +static int btrfs_interface_init (void ) {return misc_register (&btrfs_misc);
fs/fuse/dev.c +1 -0
@@ -18,6 +18,7 @@#include <linux/slab.h> MODULE_ALIAS_MISCDEV (FUSE_MINOR ); +MODULE_ALIAS ("devname:fuse" );static struct kmem_cache *fuse_req_cachep;
include/linux/miscdevice.h +3 -0
@@ -31,6 +31,9 @@#define FUSE_MINOR 229#define KVM_MINOR 232#define VHOST_NET_MINOR 233 +#define BTRFS_MINOR 234 +#define MAPPER_CTRL_MINOR 235 +#define AUTOFS_MINOR 236#define MISC_DYNAMIC_MINOR 255struct device ;
Quoted 9 lines · Kay Sievers
Quoted 6 lines · Ian Kent
Quoted 2 lines · Kay Sievers
---
a/fs/autofs4/dev-ioctl.c
+MODULE_ALIAS_MISCDEV(BTRFS_MINOR);Is this a typo or am I missing something?
Yikes, sorry, that's just broken. New patch. Thanks a lot for checking.
Thanks, I've updated the patch in my tree to this version.
greg k-h
Alasdair G Kergon
maintainerQuoted 4 lines · Kay Sievers
This adds:
alias: devname:<name>
to some common kernel modules, which will allow the on-demand loading of the kernel module when the device node is accessed.
I don't see any need for this for device-mapper: please leave dm out of this.
Quoted 1 line
Ideally all these modules would be compiled-in,
Why do you think that? Currently that's a matter for the user/distro to decide! IMHO It's really not for the kernel to force a policy like this on its users. If that's what you think, why does your patch instead not go the whole way and refuse to allow these items even to be compiled as modules?
Quoted 4 lines
but distros seems too
much in love with their modularization that we need to cover the common cases with this new facility. It will allow us to remove a bunch of pretty useless init scripts and modprobes from init scripts.
Again, I don't see why this needs any kernel changes. If this was important, any distro could deal with it itself trivially without needing a kernel patch.
Nack for the dm part of this from my point of view as it removes flexibility with a 'one size fits all' approach that introduces a fixed minor number into a dynamic world.
Alasdair
Quoted 6 lines · Alasdair G Kergon
Quoted 4 lines · Kay Sievers
This adds:
alias: devname:<name>
to some common kernel modules, which will allow the on-demand loading of the kernel module when the device node is accessed.
Quoted 8 lines
I don't see any need for this for device-mapper: please leave dm out of this.
Quoted 1 line
Ideally all these modules would be compiled-in,
Why do you think that? Currently that's a matter for the user/distro to decide! IMHO It's really not for the kernel to force a policy like this on its users. If that's what you think, why does your patch instead not go the whole way and refuse to allow these items even to be compiled as modules?
Well, they will work fine as modules, and we need them rto work as such. It just does not make much sense if you are not a developer, and distros should not do what they do, but that's a different story, I don't want to get into. As a developer modules are more than useful, they make kernel development possible without constantly rebooting.
This patch just brings the both needlessly different cases closer to each other, and does not require special init scripts anymore, to activate a specific sybsystem prior to using it.
Quoted 8 lines
Quoted 4 lines
but distros seems too
much in love with their modularization that we need to cover the common cases with this new facility. It will allow us to remove a bunch of pretty useless init scripts and modprobes from init scripts.Again, I don't see why this needs any kernel changes. If this was important, any distro could deal with it itself trivially without needing a kernel patch.
That is actually to make systemd work on Fedora. And that's driven by the company you work for. You might just not work in the area where people fight against such problems, and don't know about them now.
Quoted 3 lines
Nack for the dm part of this from my point of view as it removes flexibility with a 'one size fits all' approach that introduces a fixed minor number into a dynamic world.
There is no harm to make a well-know device node static, it just solves a lot of problems, and also makes it possible to work off of a static /dev. It's nothing different from the statically allocated numbers for /dev/dm-*
Thanks for considering,
Kay
Quoted 5 lines · Kay Sievers
On Fri, May 21, 2010 at 13:34, Alasdair G Kergon <[email protected]> wrote:
There is no harm to make a well-know device node static, it just solves a lot of problems, and also makes it possible to work off of a static /dev.
To illustrate:
On my box without this patch:
dmsetup version
Library version: 1.02.42 (2010-01-14)
/proc/misc: No entry for device-mapper found
Is device-mapper driver missing from kernel? Failure to communicate with kernel device-mapper driver.
And the same box just with this patch, nothing else changed:
dmsetup version
Library version: 1.02.42 (2010-01-14)
Driver version: 4.17.0
But its up to you to care if device-mapper just works, or if there is stuff like an init script with modprobe needed to load stuff that might never be needed. :)
This is surely not about dynamic vs. static, it is about race-free on-demand activation of services and subsystems.
Thanks,
Kay
Quoted 16 lines · On Friday 21 May 2010 18:41:38 Kay Sievers wrote:
Quoted 5 lines · Kay Sievers
On Fri, May 21, 2010 at 13:34, Alasdair G Kergon <[email protected]> wrote:
There is no harm to make a well-know device node static, it just solves a lot of problems, and also makes it possible to work off of a static /dev.
To illustrate:
On my box without this patch:
dmsetup version
Library version: 1.02.42 (2010-01-14)
/proc/misc: No entry for device-mapper found
Is device-mapper driver missing from kernel? Failure to communicate with kernel device-mapper driver.
Quoted 9 lines
And the same box just with this patch, nothing else changed:
dmsetup version
Library version: 1.02.42 (2010-01-14)
Driver version: 4.17.0But its up to you to care if device-mapper just works, or if there is stuff like an init script with modprobe needed to load stuff that might never be needed. :)
If this is needed, the dmsetup itself can do a modprobe dm instead of printing the message, "Is device-mapper driver missing from kernel?"?
Quoted 3 lines
This is surely not about dynamic vs. static, it is about race-free on-demand activation of services and subsystems.
Loading dm-mod alone is enough for dmsetup version. But for other operations dm-mod may not be enough, as various other modules like dm-crypt, dm- mirror,... would also be required, depending on the dm table, which may or may not be installed.
Thanks
Nikanth
Quoted 8 lines
Thanks,
Kay
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 22 lines · Nikanth Karthikesan
Quoted 21 lines · On Friday 21 May 2010 18:41:38 Kay Sievers wrote:
Quoted 5 lines · Kay Sievers
On Fri, May 21, 2010 at 13:34, Alasdair G Kergon <[email protected]> wrote:
There is no harm to make a well-know device node static, it just solves a lot of problems, and also makes it possible to work off of a static /dev.
To illustrate:
On my box without this patch:
dmsetup version
Library version: 1.02.42 (2010-01-14)/proc/misc: No entry for device-mapper found
Is device-mapper driver missing from kernel? Failure to communicate with kernel device-mapper driver.And the same box just with this patch, nothing else changed:
dmsetup version
Library version: 1.02.42 (2010-01-14)
Driver version: 4.17.0
Quoted 7 lines
Quoted 4 lines
But its up to you to care if device-mapper just works, or if there is stuff like an init script with modprobe needed to load stuff that might never be needed. :)
If this is needed, the dmsetup itself can do a
modprobe dminstead of printing the message, "Is device-mapper driver missing from kernel?"?
No, it can't. There is a delay until the node appears, unless you use devtmpfs. This can not be handled reliably by usual non-hotplug-aware tools.
Also it could be any user of libdevmapper doing this. And running processes from libraries without the user of the lib knowing all the problems sub-processes create is not recommended at all.
There is a good reason to let the kernel load the module on-demand, and not to code all sorts of racy stuff into tools. It's just open(), nothing else, and it is guaranteed by the kenrel to work, without any further synchronization.
Quoted 7 lines
Quoted 3 lines
This is surely not about dynamic vs. static, it is about race-free on-demand activation of services and subsystems.
Loading dm-mod alone is enough for
dmsetup version. But for other operations dm-mod may not be enough, as various other modules like dm-crypt, dm- mirror,... would also be required, depending on the dm table, which may or may not be installed.
Sure, but that is a detail, that can be solved by kernel-driven on-demand loading as well.
Kay
Alasdair G Kergon
maintainerQuoted 8 lines · Kay Sievers
Quoted 7 lines · Nikanth Karthikesan
Quoted 6 lines · On Friday 21 May 2010 18:41:38 Kay Sievers wrote:
Quoted 5 lines · Kay Sievers
On Fri, May 21, 2010 at 13:34, Alasdair G Kergon <[email protected]> wrote:
There is no harm to make a well-know device node static, it just solves a lot of problems, and also makes it possible to work off of a static /dev.
Well until now we've tried to make everything possible treat device numbers as dynamic, so I'd like to see some compelling logic behind the change.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
To illustrate:
On my box without this patch:
dmsetup version
Library version: 1.02.42 (2010-01-14)/proc/misc: No entry for device-mapper found
Is device-mapper driver missing from kernel? Failure to communicate with kernel device-mapper driver.
(Curiously this is not an issue I remember anyone raising with me as a problem before.)
So what concepts are at stake here.
The module is available to the kernel but has not been loaded. Some users do not need the module, so it should not be loaded by default. Users who *do* need it would like it to get loaded automatically.
The matter under discussion is what mechanism to use to load it automatically.
The unique information is the module name so the mechanism should ideally be tied directly to that. Anything wanting to use dm already knows that name. The character device only becomes available later, after the module is loaded, and userspace obtains it from /proc/misc.
The modprobe mechanism is tied to the name, so we should really look for a solution based on that in the first instance.
A related matter - not yet mentioned or discussed between us - is how the /dev/mapper/control node gets created and what it should be called, as that name obviously goes against the udev standard of placing everything in a flat /dev namespace with kernel-based names (so something like '/dev/miscNNN' in this case) and creating symlinks from the traditional filesystem locations like /dev/mapper/control.
Currently libdevmapper creates /dev/mapper/control as required based on /proc/misc. Presumably we should now hand that over to udev as a special case of the way we transferred the entries for the actual devices with similar waiting & notification.
Currently tools (not libdevmapper) take responsibility for checking and autoloading. E.g. lvm issues modprobes to autoload dm target modules. (The kernel does also handle this but lvm doesn't use it because it wants to know earlier that the operation would fail to avoid more complex error-handling code.)
So I think we should first try to make an extension of the existing mechanisms work. Everything is keyed off a single piece of information, the module name, shared between userspace and kernel.
Alasdair
Quoted 4 lines · Alasdair G Kergon
Quoted 3 lines · Kay Sievers
Quoted 2 lines · Nikanth Karthikesan
Quoted 1 line · On Friday 21 May 2010 18:41:38 Kay Sievers wrote:
On Fri, May 21, 2010 at 13:51, Kay Sievers <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines · Alasdair G Kergon
There is no harm to make a well-know device node static, it just solves a lot of problems, and also makes it possible to work off of a static /dev.
Well until now we've tried to make everything possible treat device numbers as dynamic, so I'd like to see some compelling logic behind the change.
It's a well known single-instance device, and the static assignment solves a few problems with on-demand subsystem activation (which is the new fashion :)) and makes it possible to hide the compiled-in/modular issues we see, and have no proper solution for it.
Quoted 11 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
To illustrate:
On my box without this patch:
dmsetup version
Library version: 1.02.42 (2010-01-14)/proc/misc: No entry for device-mapper found
Is device-mapper driver missing from kernel? Failure to communicate with kernel device-mapper driver.(Curiously this is not an issue I remember anyone raising with me as a problem before.)
Because distros go the easy and stupid way and just load everything they configured as a module. :) That is something to solve, or at least avoid and hide a bit.
Quoted 4 lines
So what concepts are at stake here.
The module is available to the kernel but has not been loaded. Some users do not need the module, so it should not be loaded by default. Users who *do* need it would like it to get loaded automatically.
Yes. Like many other things too. Ideally we just hide as much as we can that stuff is a module and not compiled-in.
Quoted 10 lines
The matter under discussion is what mechanism to use to load it automatically.
The unique information is the module name so the mechanism should ideally be tied directly to that. Anything wanting to use dm already knows that name. The character device only becomes available later, after the module is loaded, and userspace obtains it from
/proc/misc.The modprobe mechanism is tied to the name, so we should really look for a solution based on that in the first instance.
Quoted 6 lines
A related matter - not yet mentioned or discussed between us - is how the
/dev/mapper/controlnode gets created and what it should be called, as that name obviously goes against the udev standard of placing everything in a flat /dev namespace with kernel-based names (so something like '/dev/miscNNN' in this case) and creating symlinks from the traditional filesystem locations like/dev/mapper/control.
The kernel itself creates /dev/mapper/control today. And if devtmpfs is not used (all recent released or upcoming distro releases depend on it) the kernel will supply this name to udev, and udev will create it.
Quoted 4 lines
Currently libdevmapper creates
/dev/mapper/controlas required based on/proc/misc. Presumably we should now hand that over to udev as a special case of the way we transferred the entries for the actual devices with similar waiting & notification.
It like this already for a while. But it's all in the kernel, and is race free regarding the module loading, At the moment modprobe comes back, we are sure the node exists.
Quoted 5 lines
Currently tools (not libdevmapper) take responsibility for checking and autoloading. E.g. lvm issues modprobes to autoload dm target modules. (The kernel does also handle this but lvm doesn't use it because it wants to know earlier that the operation would fail to avoid more complex error-handling code.)
That's what this is about. The kernel will block the process that accesses /dev/mapper/control until it is loaded. No need to sync anything or then, or handle complex error cases.
Quoted 3 lines
So I think we should first try to make an extension of the existing mechanisms work. Everything is keyed off a single piece of information, the module name, shared between userspace and kernel.
I think its all covered with this patch. :) But if yo have any better idea, let us know.
Thanks,
Kay
--
dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel
Quoted 4 lines · Alasdair G Kergon
Quoted 3 lines · Kay Sievers
Quoted 2 lines · Nikanth Karthikesan
Quoted 1 line · On Friday 21 May 2010 18:41:38 Kay Sievers wrote:
On Fri, May 21, 2010 at 13:51, Kay Sievers <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines · Alasdair G Kergon
There is no harm to make a well-know device node static, it just solves a lot of problems, and also makes it possible to work off of a static /dev.
Well until now we've tried to make everything possible treat device numbers as dynamic, so I'd like to see some compelling logic behind the change.
It's a well known single-instance device, and the static assignment solves a few problems with on-demand subsystem activation (which is the new fashion :)) and makes it possible to hide the compiled-in/modular issues we see, and have no proper solution for it.
Quoted 11 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
To illustrate:
On my box without this patch:
dmsetup version
Library version: 1.02.42 (2010-01-14)/proc/misc: No entry for device-mapper found
Is device-mapper driver missing from kernel? Failure to communicate with kernel device-mapper driver.(Curiously this is not an issue I remember anyone raising with me as a problem before.)
Because distros go the easy and stupid way and just load everything they configured as a module. :) That is something to solve, or at least avoid and hide a bit.
Quoted 4 lines
So what concepts are at stake here.
The module is available to the kernel but has not been loaded. Some users do not need the module, so it should not be loaded by default. Users who *do* need it would like it to get loaded automatically.
Yes. Like many other things too. Ideally we just hide as much as we can that stuff is a module and not compiled-in.
Quoted 10 lines
The matter under discussion is what mechanism to use to load it automatically.
The unique information is the module name so the mechanism should ideally be tied directly to that. Anything wanting to use dm already knows that name. The character device only becomes available later, after the module is loaded, and userspace obtains it from
/proc/misc.The modprobe mechanism is tied to the name, so we should really look for a solution based on that in the first instance.
Quoted 6 lines
A related matter - not yet mentioned or discussed between us - is how the
/dev/mapper/controlnode gets created and what it should be called, as that name obviously goes against the udev standard of placing everything in a flat /dev namespace with kernel-based names (so something like '/dev/miscNNN' in this case) and creating symlinks from the traditional filesystem locations like/dev/mapper/control.
The kernel itself creates /dev/mapper/control today. And if devtmpfs is not used (all recent released or upcoming distro releases depend on it) the kernel will supply this name to udev, and udev will create it.
Quoted 4 lines
Currently libdevmapper creates
/dev/mapper/controlas required based on/proc/misc. Presumably we should now hand that over to udev as a special case of the way we transferred the entries for the actual devices with similar waiting & notification.
It like this already for a while. But it's all in the kernel, and is race free regarding the module loading, At the moment modprobe comes back, we are sure the node exists.
Quoted 5 lines
Currently tools (not libdevmapper) take responsibility for checking and autoloading. E.g. lvm issues modprobes to autoload dm target modules. (The kernel does also handle this but lvm doesn't use it because it wants to know earlier that the operation would fail to avoid more complex error-handling code.)
That's what this is about. The kernel will block the process that accesses /dev/mapper/control until it is loaded. No need to sync anything or then, or handle complex error cases.
Quoted 3 lines
So I think we should first try to make an extension of the existing mechanisms work. Everything is keyed off a single piece of information, the module name, shared between userspace and kernel.
I think its all covered with this patch. :) But if yo have any better idea, let us know.
Thanks,
Kay
Alasdair G Kergon
maintainerQuoted 1 line · Kay Sievers
The kernel itself creates
/dev/mapper/controltoday.
Must have missed that patch:)
Userspace lvm and dm code is certainly not tied to that today.
The device node that userspace lvm and dm tools use is under the complete control of this userspace code: it chooses the name, and creates or fixes it if it doesn't already exist with the required
properties: anything devtmpfs is relevant only if it guessed the right name that this *completely independent* userspace code chooses to use and already created it exactly as required.
If we were to move to a fixed name for the control device and hand control of that name to the kernel, then obviously this userspace code would need adjusting to respect that.
Alasdair
Quoted 13 lines · Alasdair G Kergon
Quoted 1 line · Kay Sievers
The kernel itself creates
/dev/mapper/controltoday.Must have missed that patch:)
Userspace lvm and dm code is certainly not tied to that today.The device node that userspace lvm and dm tools use is under the complete control of this userspace code: it chooses the name, and creates or fixes it if it doesn't already exist with the required
properties: anything devtmpfs is relevant only if it guessed the right name that this *completely independent* userspace code chooses to use and already created it exactly as required.
Quoted 3 lines
If we were to move to a fixed name for the control device and hand control of that name to the kernel, then obviously this userspace code would need adjusting to respect that.
Upstream udev has no primary device node naming anymore, it's all controlled by the kernel. In fact, recent udev versions log errors if userspace defines names which disagree with the kernel-provided names, or the kernel-created nodes.
There is no "abstraction" anymore, the kernel defines the API today, and device naming is 100% inside the kernel.
Udev's job is manage permissions, and to create meaningful symlinks which may change during lifetime of a kernel device.
The primary kernel device name is always in sync with "dmesg", "proc/sys" and "/dev" these days. Only the symlinks may change at any time, reflecting the current state of the device.
Kay
Quoted 13 lines · Alasdair G Kergon
Quoted 1 line · Kay Sievers
The kernel itself creates
/dev/mapper/controltoday.Must have missed that patch:)
Userspace lvm and dm code is certainly not tied to that today.The device node that userspace lvm and dm tools use is under the complete control of this userspace code: it chooses the name, and creates or fixes it if it doesn't already exist with the required
properties: anything devtmpfs is relevant only if it guessed the right name that this *completely independent* userspace code chooses to use and already created it exactly as required.
Quoted 3 lines
If we were to move to a fixed name for the control device and hand control of that name to the kernel, then obviously this userspace code would need adjusting to respect that.
Upstream udev has no primary device node naming anymore, it's all controlled by the kernel. In fact, recent udev versions log errors if userspace defines names which disagree with the kernel-provided names, or the kernel-created nodes.
There is no "abstraction" anymore, the kernel defines the API today, and device naming is 100% inside the kernel.
Udev's job is manage permissions, and to create meaningful symlinks which may change during lifetime of a kernel device.
The primary kernel device name is always in sync with "dmesg", "proc/sys" and "/dev" these days. Only the symlinks may change at any time, reflecting the current state of the device.
Kay
Quoted 11 lines · Kay Sievers
Quoted 10 lines · Alasdair G Kergon
Quoted 6 lines · Kay Sievers
Quoted 6 lines
Quoted 6 lines
On my box without this patch:
dmsetup version
Library version: 1.02.42 (2010-01-14)/proc/misc: No entry for device-mapper found
Is device-mapper driver missing from kernel? Failure to communicate with kernel device-mapper driver.(Curiously this is not an issue I remember anyone raising with me as a problem before.)
I am fairly sure i did report that, i even tried moving the module loading framework from lvm2 to device mapper, but the resulting code sucked too much to submit, the idea was to load device-mapper module, and all target modules on first use.
Quoted 3 lines
Because distros go the easy and stupid way and just load everything they configured as a module. :) That is something to solve, or at least avoid and hide a bit.
In this case we were forced to preload dm-mod to work around the issue ;)
Quoted 3 lines
I think its all covered with this patch. :) But if yo have any better idea, let us know.
what patch?
L.
Signature · 6 lines
Luca Berra -- [email protected] Communication Media & Services S.r.l. /"\ \ / ASCII RIBBON CAMPAIGN X AGAINST HTML MAIL / \
Alasdair G Kergon
maintainerQuoted 1 line · Luca Berra
what patch?
Peter Rajnoha is working on a final version of this, which includes tweaking the userspace logic to give supremacy to devtmpfs.
Alasdair
Alasdair G Kergon
maintainerQuoted 1 line · Luca Berra
what patch?
Peter Rajnoha is working on a final version of this, which includes tweaking the userspace logic to give supremacy to devtmpfs.
Alasdair
Quoted 5 lines · Alasdair G Kergon
Quoted 1 line · Luca Berra
what patch?
Peter Rajnoha is working on a final version of this, which includes tweaking the userspace logic to give supremacy to devtmpfs.
Note, that this will also work without devtmpfs, it will be just udev pre-creating the nodes on startup, even when the modules aren't loaded. Any access to the "dead" device nodes will cause the kernel to load the corresponding module.
We need this facility mainly for other subsystems, which depend on this behavior for some time already.
Thanks,
Kay
Quoted 5 lines · Alasdair G Kergon
Quoted 1 line · Luca Berra
what patch?
Peter Rajnoha is working on a final version of this, which includes tweaking the userspace logic to give supremacy to devtmpfs.
Note, that this will also work without devtmpfs, it will be just udev pre-creating the nodes on startup, even when the modules aren't loaded. Any access to the "dead" device nodes will cause the kernel to load the corresponding module.
We need this facility mainly for other subsystems, which depend on this behavior for some time already.
Thanks,
Kay