ANNOUNCE: User-space System Device Enumation (uSDE)
Initial availability of User-Space System Device Enumeration (uSDE) package, version 0.74, can be found at http://sourceforge.net/projects/usde
The uSDE provides an open framework for the enumeration (specification) of system devices in a dynamic environment. Device handling is implemented via plug-in programs known as policy methods. Policy methods are free to handle their devices in any way, from trivial to complex - anything from providing LSB device nodes to persistent device name handling with replacement and relocation strategies.
The uSDE depends on /sbin/hotplug (for dynamic insertions and removals), sysfs (for device information) and /proc (various pieces of information). It is not dependent on initrd - it explicitly scans sysfs upon system startup to determine the initial device ensemble.
Part of the uSDE release is a collection of sample polices:
disk-ide-policy - handles IDE, EIDE, SATA and USB-EIDE disks. Implements persistent device naming, automatic device replacement and automatic device relocation features.
disk-scsi-policy - handles SCSI, IEEE-1394, FibreChannel and USB-SCSI disks including multiported devices. Iplements persistent device naming, automatic device replacement and automatic device relocation features.
multipath-policy - handles the automatic provisioning of multipathing for multiported storage devices.
ethernet-policy - handles ethernet interefaces. Implements persistent interface naming, interface anchoring, automatic device replacement and automatic device relocation features.
floppy-policy - handles internal floppy disks.
simple-device-policy - a "catch all" policy for block and character devices.
devfs-policy - provides devfs device names.
lsb-policy - provides LSB device names.
Location: http://sourceforge.net/projects/usde Mailing list: [email protected]
mark
Quoted 2 lines · Mark Bellon
Initial availability of User-Space System Device Enumeration (uSDE) package, version 0.74, can be found at http://sourceforge.net/projects/usde
How does it relate to udev?
Quoted 2 lines
multipath-policy - handles the automatic provisioning of multipathing for multiported storage devices.
How does this integrate with DM, md, EVMS, LVM...?
Thanks,
Lars Marowsky-Brée <[email protected]>
Signature · 8 lines
High Availability & Clustering \ ever tried. ever failed. no matter. SUSE Labs | try again. fail again. fail better. Research & Development, SUSE LINUX AG \ -- Samuel Beckett - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 4 lines
How does it relate to udev?
The uSDE and udev are simlar in some respects. The uSDE allows for complete control of the policy handling a device - not just its naming. It can perform complex policies on behalf of its devices including multiple concurrent policies for each device. It is also very aggressive about handing the system start up issues when many devices must be dealt with.
Quoted 2 lines
How does this integrate with DM, md, EVMS, LVM...?
As devices appear in sysfs the uSDE reacts to them via their hotplug events. The policy for each device handles any device issues including dealing with any device nodes. It is possible to track and maintain multiported devices and automatically provide multipath devices nodes for instance.
mark
Quoted 2 lines · Mark Bellon
The uSDE and udev are simlar in some respects. The uSDE allows for complete control of the policy handling a device - not just its naming.
Well, so could udev in theory, and I had this plan to enhance it to do so for the specific case of multipathing one day in the not too distant future (ie, before q1/04).
In as far as I can see, udev and uSDE really do not have too different goals. Competition is good, but only if they explore distinct approaches ;-)
Quoted 6 lines
Quoted 1 line
How does this integrate with DM, md, EVMS, LVM...?
As devices appear in sysfs the uSDE reacts to them via their hotplug events. The policy for each device handles any device issues including dealing with any device nodes. It is possible to track and maintain multiported devices and automatically provide multipath devices nodes for instance.
Yes, I know that, I was asking whether you had done any discussion with the EVMS2 folks for example to have a policy plugin to interact with EVMS2 accordingly and do the magic.
Sincerely,
Lars Marowsky-Brée <[email protected]>
Signature · 3 lines
High Availability & Clustering \ ever tried. ever failed. no matter. SUSE Labs | try again. fail again. fail better. Research & Development, SUSE LINUX AG \ -- Samuel Beckett
Quoted 6 lines · Lars Marowsky-Bree
Quoted 2 lines · Mark Bellon
The uSDE and udev are simlar in some respects. The uSDE allows for complete control of the policy handling a device - not just its naming.
Quoted 8 lines
Well, so could udev in theory, and I had this plan to enhance it to do so for the specific case of multipathing one day in the not too distant future (ie, before q1/04).
In as far as I can see, udev and uSDE really do not have too different goals. Competition is good, but only if they explore distinct approaches ;-)
There are several distinct approaches which have been enumerated in other mails.
Since this point has not been addressed, I'd like to focus on the major difference in philosophy.
SDE places all policy in the hands of the policy developer in a seperate policy program. udev places the policies in the main processing loop of the system, effectively implementing whatever policy is desired by the udev maintainers.
Without seperating policies from the core executive of device naming system, the core of udev suffers from the same issues as placing policy
in the kernel suffers. Lack of maintainability, lack of user-defined
functionality, bloat, etc.
Quoted 7 lines
Quoted 6 lines
Quoted 1 line
How does this integrate with DM, md, EVMS, LVM...?
As devices appear in sysfs the uSDE reacts to them via their hotplug events. The policy for each device handles any device issues including dealing with any device nodes. It is possible to track and maintain multiported devices and automatically provide multipath devices nodes for instance.
Quoted 5 lines
Yes, I know that, I was asking whether you had done any discussion with the EVMS2 folks for example to have a policy plugin to interact with EVMS2 accordingly and do the magic.
No but this is definately a good idea.
Thanks!
-steve
Quoted 3 lines
Sincerely,
Lars Marowsky-Brée <[email protected]>
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 18 lines · Steven Dake
Quoted 6 lines · Lars Marowsky-Bree
Quoted 2 lines · Mark Bellon
The uSDE and udev are simlar in some respects. The uSDE allows for complete control of the policy handling a device - not just its naming.
Quoted 8 lines
Well, so could udev in theory, and I had this plan to enhance it to do so for the specific case of multipathing one day in the not too distant future (ie, before q1/04).
In as far as I can see, udev and uSDE really do not have too different goals. Competition is good, but only if they explore distinct approaches ;-)
There are several distinct approaches which have been enumerated in other mails.
Quoted 7 lines
Since this point has not been addressed, I'd like to focus on the major difference in philosophy.
SDE places all policy in the hands of the policy developer in a seperate policy program. udev places the policies in the main processing loop of the system, effectively implementing whatever policy is desired by the udev maintainers.
What do you mean by "policy"?
If you are saying that SDE allows programmers the ability to write new programs to plug into your framework to create new types of policies, then I understand that. Your loadable plugins look like they support that. But they require a dynamic loader :)
What udev is doing is trying to provide a flexible policy that will work for everyone, with a heirchy of rules that can be easily controlled and changed by anyone who can operate a text editor (and soon to be changed by GUI applications, like the HAL project).
I have not heard of any situation in which the current udev set of rules do not work out for them. And if you can think of one, can't it be covered by the CALLOUT rule? For example, someone has sent me a small userspace program that works with the CALLOUT rule that handles multipath devices by talking to the dm code. Now that's pretty flexible.
If you (or anyone else) thinks of something that the existing udev rules do not handle, please let me know. If it's too complex, then yes, the user should use write their own SDE plugin. But remember, 99.9% of the people out there just want the LSB device names, with possibly a persistent entry for their digital camera and USB joystick, which udev handles just fine today.
For people with 4000 real scsi disks, udev also works well. That seems to cover the wide range of users.
Quoted 4 lines
Without seperating policies from the core executive of device naming system, the core of udev suffers from the same issues as placing policy
in the kernel suffers. Lack of maintainability, lack of user-defined
functionality, bloat, etc.
Easy Separation? Hm, in looking at udev, if you just replace 2 .c files with your new naming scheme, everything works just fine.
And if you want to go down the path of accusations about lack of maintainability, bloat, etc. I will be glad to point people at your tree and then they can see these kinds of numbers:
For SDE:
$ find . -type f | egrep '[.c|.h]$' | xargs wc -l | tail -1 57328 total
Wow, you build 18 shared libraries:
$ find . -type f | grep '\.so' | wc
18 18 625For udev:
$ find . -type f | egrep '[.c|.h]$' | xargs wc -l | tail -1 17632 total
And that includes all of klibc, which really is not fair for udev to calculate. So let's just look at the udev code size:
$ ls | egrep '[.c|.h]$' | xargs wc -l | tail -1 2613 total
And to be complete, let's add the totals of libsysfs and tdb, but to be fair any udev developer never has to look into those files:
libsysfs is this big:
3798 total
And tdb is this big:
2679 total
So adding those numbers up we get these kinds of numbers for size of the .c and .h files in the different projects:
SDE: 57328 lines udev: 9090 lines
That makes SDE over 6 times bigger in source code alone than all of udev (including tdb and libsysfs).
I can compare executable size too, if you really want to still claim that udev is suffering from "lack of maintainability and bloat" if you really want :)
Oh, any reason you all haven't shown a working uSDE system in public anywhere?
thanks,
greg k-h
p.s. yes, I know lines of code is a horrible metric, and doesn't really mean squat. I just want to point out the huge size difference between the current state of udev and SDE, with pretty much identical functionality from what I can tell.
Quoted 3 lines · Greg KH
For SDE:
$ find . -type f | egrep '[.c|.h]$' | xargs wc -l | tail -1 57328 total
...
Quoted 3 lines
For udev:
$ find . -type f | egrep '[.c|.h]$' | xargs wc -l | tail -1 17632 total
Here is a config file and a 41 lines shell script, that will populate /udev with all devices found via /sys. A version to add or remove only one file should be about the same size.
---cat /etc/makedev.conf---
ttyS0 root dialout 0660
zero root root 0666
null root root 0666
---cat makedev---
#!/bin/sh
set -e
DEV=/udev
CONFIG=/etc/makedev.conf
cd $DEV
find /sys/class -name dev |while read A;
do
B=`dirname $A` B=`basename $B` MM=`cat $A|tr ":" " "` mknod --mode=0600 $B c $MM if grep -q "$B " $CONFIG then USER=`grep "$B " $CONFIG|cut -d" " -f2` GROUP=`grep "$B " $CONFIG|cut -d" " -f3` MODE=`grep "$B " $CONFIG|cut -d" " -f4` chown $USER.$GROUP $B chmod $MODE $B fi
done
find /sys/block -name dev |while read A;
do
B=`dirname $A` B=`basename $B` MM=`cat $A|tr ":" " "` mknod --mode=0600 $B b $MM if grep -q "$B " $CONFIG then USER=`grep "$B " $CONFIG|cut -d" " -f2` GROUP=`grep "$B " $CONFIG|cut -d" " -f3` MODE=`grep "$B " $CONFIG|cut -d" " -f4` chown $USER.$GROUP $B chmod $MODE $B fi
done
---cut---
So udev is 99% overhead?
sure, it's fast and small, and has lots of features that I don't need.
Quoted 2 lines
SDE: 57328 lines udev: 9090 lines
shell script: 41 lines
Quoted 2 lines
that udev is suffering from "lack of maintainability and bloat" if you really want :)
bloat. lots of bloat. what is that tdb database for? filesystems are persistent. if you want to save space, create a tar file :-)
Quoted 4 lines
p.s. yes, I know lines of code is a horrible metric, and doesn't really mean squat. I just want to point out the huge size difference between the current state of udev and SDE, with pretty much identical functionality from what I can tell.
I agree. lines of codes is a horrible metric, and comparing a shell script that uses many external commands to a c application with everything build is makes absolutely no sense. but I wonder why the off the shelf machine needs a c applications, if all those external commands are installed anyway.
Regards, Andreas
Quoted 1 line · Andreas Jellinghaus
On Tue, 28 Oct 2003 22:52:33 +0000, Greg KH wrote:
[snip]
Quoted 6 lines
Quoted 2 lines
that udev is suffering from "lack of maintainability and bloat" if you really want :)
bloat. lots of bloat. what is that tdb database for? filesystems are persistent. if you want to save space, create a tar file :-)
[snip]
The tdb database is for storing current device information, udev needs to reference names to devices. The database also enables an api for applications to query what devices are on the system, their names, and their nodes.
Using tdb has its advantages too; it's small, it's flexible, it's fast, it can be in memory or on disk, and it has locking for multiple accesses.
IMVHO - tdb isn't bloat.
Thanks,
Dan
Quoted 8 lines · Daniel Stekloff
The tdb database is for storing current device information, udev needs to reference names to devices. The database also enables an api for applications to query what devices are on the system, their names, and their nodes.
Using tdb has its advantages too; it's small, it's flexible, it's fast, it can be in memory or on disk, and it has locking for multiple accesses.
IMVHO - tdb isn't bloat.
Hi Dan,
thanks for your email.
I took a look at tdb. Upon adding devices, the DEVPATH is resolved via config files etc. to a final /dev filename. That combination is stored in tdb, and when the device is remove, the same resolution process is not done, but the tdb is looked up to find the filename, and remove the device. Is that right?
So the advantage would be resistance against config file changes - if the nameing scheme is changed while a devices is added, the remove would get the new name, and that way try to remove the wrong device.
Also this mechanism could be used to implement counting device names like "disc/0", where the final name depends on the devices currently available, so there is no static translation from devpath to the filename.
I'd prefer the kernel giving up the old device names, and migrating to counter names i.e. disc/0, cdrom/0, printer/0, etc. Those who still want the old names could use /sys/ to determine the details on the device, and that way create devices per the old naming schema. That way tdb wouldn't be necessary for counting device names, at least if sysfs still has the full information on the device while the hotplug event runs. I guess that is not the case or not guaranteed?
Also I have to admit, if symlinks like "hpdeskjet" to some usb device are configured in the config, the device is attached, and the config is changed, then a remove event will not find the old symlink and cannot remove it, without tdb.
But maybe like a coldplug / fulling an empty /dev, there should be rerun command? I.e. like coldplug determine what device and symlinks shold be in /dev, and the remove unnecessary, add missing, and modify outdated entries (devices,files)? If that existed, configuration changes wouldn't be a reason for udev to use tdb?
So why is tdb currently required? I only see the possibility to use naming schemes like disc/0 as a reason, but that isn't implemented in udev so far...
other than a theological discussion about needed or not, I guess nobody will complain about it - even people with /dev on tmpfs and a readonly / will have a writable or tmpfs /var so they can live with it anyway. but I'm still not sure, if it isn't unnecessary.
Regards, Andreas
Quoted 6 lines · Andreas Jellinghaus
Quoted 5 lines · Daniel Stekloff
The tdb database is for storing current device information, udev needs to reference names to devices. The database also enables an api for applications to query what devices are on the system, their names, and their nodes.
Quoted 6 lines
Quoted 4 lines
Using tdb has its advantages too; it's small, it's flexible, it's fast, it can be in memory or on disk, and it has locking for multiple accesses.
IMVHO - tdb isn't bloat.
Hi Dan,
Hi Andreas,
Sorry for my late reply.
Quoted 6 lines
thanks for your email.
I took a look at tdb. Upon adding devices, the DEVPATH is resolved via config files etc. to a final /dev filename. That combination is stored in tdb, and when the device is remove, the same resolution process is not done, but the tdb is looked up to find the filename, and remove the device. Is that right?
The device path and device name are stored in the tdb database. The path is used as the primary key, the unique identifier to locate the specific device. A device is added by path and then removed using the path.
Quoted 3 lines
So the advantage would be resistance against config file changes - if the nameing scheme is changed while a devices is added, the remove would get the new name, and that way try to remove the wrong device.
Adding and removing device from the tdb database is done based on the device's path and not the generated name. Changing the config file shouldn't impact currently loaded and configured devices.
Quoted 24 lines
Also this mechanism could be used to implement counting device names like "disc/0", where the final name depends on the devices currently available, so there is no static translation from devpath to the filename.
I'd prefer the kernel giving up the old device names, and migrating to counter names i.e. disc/0, cdrom/0, printer/0, etc. Those who still want the old names could use
/sys/to determine the details on the device, and that way create devices per the old naming schema. That way tdb wouldn't be necessary for counting device names, at least if sysfs still has the full information on the device while the hotplug event runs. I guess that is not the case or not guaranteed?Also I have to admit, if symlinks like "hpdeskjet" to some usb device are configured in the config, the device is attached, and the config is changed, then a remove event will not find the old symlink and cannot remove it, without tdb.
But maybe like a coldplug / fulling an empty /dev, there should be rerun command? I.e. like coldplug determine what device and symlinks shold be in /dev, and the remove unnecessary, add missing, and modify outdated entries (devices,files)? If that existed, configuration changes wouldn't be a reason for udev to use tdb?
Quoted 8 lines
So why is tdb currently required? I only see the possibility to use naming schemes like disc/0 as a reason, but that isn't implemented in udev so far...
other than a theological discussion about needed or not, I guess nobody will complain about it - even people with /dev on tmpfs and a readonly / will have a writable or tmpfs /var so they can live with it anyway. but I'm still not sure, if it isn't unnecessary.
Honestly, I'm not exactly sure what you're questioning - is tdb specifically unnecessary or is having a backing store unnecessary.
It's true, you could do without a backing store. You could build the reference between device and name every time you wish to work with the device. In some cases that would be easy, such as simply using the kernel naming system. In other cases it might not be so easy, especially if you use a specific and complicated naming scheme. You'd have to pay the processing hit it would take to calculate the names every time you access a device. You'd also have to deal with issues like current devices and adjusting the config files, which you've already brought up.
I believe having a backing store - whether tdb, gdbm, a table in memory, or a flat file - is useful for storing the configured devices and their names. It's cheap to submit a small query to find the necessary device, rather than having to go through the naming process every time a device is added, removed, or queried. Having the backing store removes the problem you mentioned about changing config files on the fly, there's no loose ends or missing or changed device names - they are stored in the database. I believe the trade off between backing store complexity and storage versus on the fly calculation is worth it. I can understand if you feel differently.
As for tdb specifically being unnecessary, you could most certainly use something else to store device information. You could use a flat file. You could use gdbm. You could use a simple table in memory. I chose tdb for the following reasons:
- it is a proven mechanism.
- it could handle thousands of devices.
- it is fast to query.
- it has a small footprint.
It seemed to me like a good solution for udev's backing store need.
If you believe udev doesn't need the backing store and/or tdb, let's see your solution and we can decide on its technical merit by looking it over. I'm certainly open to new ideas and solutions. I'm sorry if you felt this was a "theological" discussion, I certainly don't feel the same way.
Cheers,
Dan
Quoted 1 line
Regards, Andreas
Quoted 1 line · Andreas Jellinghaus
---cut---
Sweet shell script, nice job.
Quoted 1 line
So udev is 99% overhead?
To you, sure, it might be. Don't use it then, I'm not forcing anyone.
Quoted 3 lines
Quoted 2 lines
SDE: 57328 lines udev: 9090 linesshell script: 41 lines
Hm, how about the size of bash?
Quoted 6 lines
Quoted 2 lines
that udev is suffering from "lack of maintainability and bloat" if you really want :)
bloat. lots of bloat. what is that tdb database for? filesystems are persistent. if you want to save space, create a tar file :-)
Sweet, and then run everything on a in-ram compress filesystem just to save that precious disk space.
Quoted 10 lines
Quoted 4 lines
p.s. yes, I know lines of code is a horrible metric, and doesn't really mean squat. I just want to point out the huge size difference between the current state of udev and SDE, with pretty much identical functionality from what I can tell.
I agree. lines of codes is a horrible metric, and comparing a shell script that uses many external commands to a c application with everything build is makes absolutely no sense. but I wonder why the off the shelf machine needs a c applications, if all those external commands are installed anyway.
Remember, userspace is just a load test for the kernel, who really needs applications anyway.
</sarcasm>
greg k-h
Hi Greg!
Quoted 4 lines · Greg KH
Sweet shell script, nice job.
Quoted 1 line
So udev is 99% overhead?
To you, sure, it might be. Don't use it then, I'm not forcing anyone.
Thats right! So I don't understand all those discussions why some people created an alternative to udev, as everyone is free to. All implementations seem to be quit young, and I enjoy many different approaches to solve these related problems. a bit of diversity or evolution and natural selection doesn't hurt.
But, all these different user space tools share one thing: the same kernel. the linux kernel has to work for all of us, and I even think it is very important to have many different user space tools at the current time, so everyone can voice their opinions and suggest how the kernel should be changed, so it will work for him, and preferable for all of us.
Now, I'd love if the kernel would keep the permissions that devfs has. you don't. nobody else wrote something about it? but that issue isn't that important.
I'd like to see a different namespace. I need some info on block devices. currently there is no way to see if hdc is a cdrom or a disc in sysfs, so that should be added.
how do you call that distinction cdrom/disc? type? class? I don't now. I was under the impression, that this should be part of the path to the dev file, i.e. cdroms should be in a cdrom/ directory and discs in a disc/ directory. would that be a major change?
the alternative would be an additional attribute to the kobject with that information.
I'm no kernel hacker, I can bareley understand some parts of the code and do trivial changes. But I will try to learn more and maybe be of some help.
Andreas
Quoted 5 lines · Andreas Jellinghaus
Hi Greg!
Quoted 2 lines · Greg KH
Sweet shell script, nice job.
Quoted 8 lines
Quoted 2 lines
Quoted 1 line
So udev is 99% overhead?
To you, sure, it might be. Don't use it then, I'm not forcing anyone.
Thats right! So I don't understand all those discussions why some people created an alternative to udev, as everyone is free to. All implementations seem to be quit young, and I enjoy many different approaches to solve these related problems. a bit of diversity or evolution and natural selection doesn't hurt.
I agree, evolution is good in this case. I'm just really curious as to why Monta Vista and Intel decided that udev wasn't an approiate project to work with, and decided to spend time and resources to develop an alternative one. Just a bit frustrated, as udev really could have used a few people working on it full time as SDE had.
But hey, that's all past, if those companies like to live with the NIH symptom, I can't change that. And udev development will go on regardless.
Quoted 6 lines
But, all these different user space tools share one thing: the same kernel. the linux kernel has to work for all of us, and I even think it is very important to have many different user space tools at the current time, so everyone can voice their opinions and suggest how the kernel should be changed, so it will work for him, and preferable for all of us.
Agreed.
Quoted 1 line
I'd like to see a different namespace.
Different namespace where? In /sys? Or somewhere else?
Quoted 2 lines
I need some info on block devices. currently there is no way to see if hdc is a cdrom or a disc in sysfs, so that should be added.
Not true at all. Just look at the device symlink, and the type file in that directory. Shows it quite well for all of my scsi devices. Now it looks like IDE needs to add that same type of file, but that's an IDE issue. Actually IDE's sysfs code probably could use some good cleanups, now that I look at it some more. Want to work on that?
thanks,
greg k-h
Quoted 6 lines · Lars Marowsky-Bree
Quoted 2 lines · Mark Bellon
Initial availability of User-Space System Device Enumeration (uSDE) package, version 0.74, can be found at http://sourceforge.net/projects/usde
Quoted 7 lines
How does it relate to udev?
Quoted 2 lines
multipath-policy - handles the automatic provisioning of multipathing for multiported storage devices.
How does this integrate with DM, md, EVMS, LVM...?
Device mapper and the md layer generates hotplug events which are managed by the framework. There is no specific policy for these subsystems, but they could be implemented... With today's code, they fall through and are exported as disks. One good policy method that could be implemented is to scan the lvm lv list and map the major/minor to the real lv, and use that to name the device (instead of lvm naming the device). This hasn't been done but if your up for it, Mark can probably help you.
One of the cool features that integrates with MD is that multiple paths are automatically configured via the MD driver (similiar to the automatic multipathing program I have written and posted to this list). Basically that program was turned into a policy for the uSDE.
The code your looking for if you want to see how the multipath policy works is in:
sde/programs/policy-methods/mass-storage/multipath
Thanks!
-steve
Quoted 2 lines
Thanks,
Lars Marowsky-Brée <[email protected]>
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 11 lines
Device mapper and the md layer generates hotplug events which are managed by the framework. There is no specific policy for these subsystems, but they could be implemented... With today's code, they fall through and are exported as disks. One good policy method that could be implemented is to scan the lvm lv list and map the major/minor to the real lv, and use that to name the device (instead of lvm naming the device). This hasn't been done but if your up for it, Mark can probably help you.
Cool idea! I would be most open to having someone contribute this or, if they could help work out a set of simple requirements I could womp up an initial policy for this in a short time.
mark