driverfs: [PATCH] remove bus and improve driver management (2.5.30)

This patch removes bus.c from the Driver Model and replaces it with
a more advanced and scaleable driver subsystem.

The following diagram illustrates the current Driver Model implementation for drivers:
bus-> pci-> parport_pc

                          agpgart
                          cardbus
             usb->    
                          hid

Lets say that the usb bus is connected to pci0. Notice how the usb
driver has no linkage to the pci driver. It should in order to take full advatage of the power management and other features of the driver model. Of course the usb bus could create another driver and register it to pci but that would be wasteful and overly complicated. Also notice that parport_pc could have a printer attached to it. The lp driver would control this device but there is no way to represent the lp driver in the current model since only buses can have child drivers.

The following diagram illustrates the driver implementation after this patch: (this assumes that the listed driver have been converted to the Driver Model)
driver-> pci-> usb-> hid

                             parport_pc->    lp
                             agpgart
                             cardbus

Notice that this implementation not only solves the problems listed
earlier but also it is more scalable because it lists the drivers by thier true relationship to one another. These changes actually reduce the total amount of code as well as the complexity of the entire system resulting in better efficiency and perhaps even less memory consumption.

`This patch also provides user level driver management support
through the advanced features of the new interface. It creates a file entry named "driver" for each device. To attach a driver simply echo the name of the driver you want to attach. For example:
#cd ./root/pci0/00:00.0
#echo "agpgart" > driver

To remove a driver simply echo remove to the driver file while a driver is loaded. For example:
#echo "remove" > driver

If you read the driver file you will get the name of the loaded driver if a driver is loaded. For example:
#cat driver
output: agpgart

This patch is against 2.5.30. The following still needs to be done:
1.) port to 2.5.31
2.) update bus drivers, currently only pci has been ported and tested, use the other buses at your own risk
3.) explore the possibility of linking the driver and root trees. I have included an unused and untested function that will provide at least a framework for this. See the code and comments in fs.c. The function is called "device_driver_link".

Sincerely,

Adam Belay

*because the patch is so large I have gzipped it and attached it. It is available in other formats upon request.

Hi,

Hm, I'm a bit slow, but even with your second drawings, I still don't understand this. Can you add | and - lines to the drawing, it would help me understand what you are trying to show here.

But there is a pci driver that implements the USB "bridge" from PCI to usb. It's a pci driver, and it shows up in the full device tree as joining PCI to USB. It's called uhci-hcd, ohci-hcd, or ehci-hcd depending on the type of PCI USB controller you have.

Um, I don't understand, this is already there.

Have you looked at the /root directory in the driverfs mount? It shows all the devices hooked together like you are talking about.

Yes, the bus subdirectories are separate, but that's because they focus on bus specific stuff, not the interlinking between the busses. That's shown in the /root tree.

<plug>
My latest patches to convert the USB code to use the driver model are at:

	bk://linuxusb.bkbits.net/usb-2.5-driver

It's working pretty well now, thanks to the patches from Pat that are in Linus's latest bk tree.
</plug>

Again, I don't quite understand the drawing, sorry.

But what's the problem you are trying to solve? How the devices interact in the global device tree? That's already shown properly. Or am I missing something here?

Hm, how does this bind the driver to the device? What if the driver doesn't want to bind to this device? And how does userspace know what pci device to bind to what driver? Does it use the information in the modules.*map file? If so, that comes directly from the drivers themselves, which are the ones knowing what devices they _should_ be bound to, so why not let the driver themselves do the work (well actually the very small function at drivers/pci/pci-driver.c:pci_match_device() does the work).

Again, why? And does this force the ->remove() function to be called?

Now this is a nice idea. But I was thinking of moving the symlink from the bus/pci/devices to be under the specific driver in bus/pci/drivers/foo_driver. That would show you at a simple glance which driver is bound to which device. But putting the name in the device entry in the /root tree would be good enough too.

Watch out, there are lots of driver model changes already in Linus's tree (post 2.5.31.)

thanks,

greg k-h

Sorry about the confusing diagram, I'll just write out paths. 3rd time is a charm.

my old driver tree:
/driverfs/bus
/driverfs/bus/pci
/driverfs/bus/pci/drivers/agpgart
/driverfs/bus/pci/drivers/cardbus
/driverfs/bus/pci/drivers/parport_pc
/driverfs/bus/usb
my new tree now:
/driverfs/driver/
/driverfs/driver/pci
/driverfs/driver/pci/agpgart
/driverfs/driver/pci/cardbus
/driverfs/driver/pci/parport_pc

a possibility for my tree when drivers that need it convert to my changes and the driver model in general:
/driverfs/driver/
/driverfs/driver/pci
/driverfs/driver/pci/agpgart
/driverfs/driver/pci/cardbus
/driverfs/driver/pci/parport_pc
/driverfs/driver/pci/parport_pc/lp
/driverfs/driver/pci/usb
/driverfs/driver/pci/usb/hid

* I am not saying that usb has to be done this way but if you notice in the first diagram (the old implementation) usb does not have any driver representation with pci or if it does it is not displayed. The lp driver is significant because it is imposible to cleanly implement 3 levels of drivers in the current Driver Model.

As you can now see in my new diagrams the problem I am trying to
solve is not how devices are represented but rather how drivers are represented. Sorry I wasn't clear. I have completely removed bus and replaced it with a driver interface. This is better because a bus does not deserve special implementations as it is a driver too. My more scaleable interface can handle both buses and drivers.

This feature is well designed, if a driver doesn't want to bind to the device the request will fail. Check the code yourself. This is simply a userspace override for the Driver Model. This is useful because it provides the user with more control over driver management. If two drivers can support the same device and the driver model selects one the user doesn't want to use the user can change it. This could even happen now with ALSA and Open Sound. Since it calls both probe and match it is also a great debugging tool. I'm currently working on a user level utility that will take full advantage of this and other driver management features.

Yes this does call ->remove() among other things. In fact it calls do_driver_detach. Why is above.

There are many ways to go about it. I even left in a few extra fucntions in fs.c for other options. The question is what's the best way to do it?

:-(

If you have any questions feel free to ask.

Thanks,
Adam

Ah that makes more sense now, thanks.

But the problem is that the USB hid driver does NOT have a relationship with a pci driver. The USB drivers don't care about what kind of host controller driver they talk to, _and_ that relationship isn't present. There are quite a few USB host drivers that do not sit on the PCI bus, but talk directly over a "system" bus to the controller (embedded devices mostly.)

But a PCI bus could also be present, with a USB controller, and the hid driver would be able to handle devices on it too. So how would you show this "dual" relationship then?

In short, I don't see the value in showing relationships between different driver types. But I see the value in relationships between devices, as is shown in the root/ tree.

A bus is different than a driver, as drivers bind to the bus type. In a way, you can say a bus is nothing more than a "class", and class support _is_ coming soon. I think class support is what you are really looking for, as it will be able to show you the relationship between devices much better.

Again, I don't see the advantage here. Since you say the kernel driver knows best, and that the kernel driver gets a say in if it really binds to the device or not, why not just let the kernel driver do all the work then? I don't see how an extra step from the user will help any. 'insmod' works quite well for picking which driver you want loaded for the devices (ALSA vs. OSS.) Yes, it doesn't handle mixing lots of the same device with different drivers, but I don't think that is a very common thing (otherwise people would have complained about it by now.)

Again, 'rmmod' does the same thing :)

thanks,

greg k-h

BTW, for the restless and curious, there is a BK tree available that has preliminary class and interface support in it. I'm relatively satisfied with the way things are starting to fall out in it, so if anyone wants to start taking a look and give me some feedback, that'd be great.

Note that it's still very immature, and lacking documentation. But, for people that are already playing with it, have at it:

	bk://ldm.bkbits.net/linux-2.5-devclass


	-pat

Good point.

Check this out. Rather than explaining it, I've attached it to this email. It might solve this problem. It's based on an idea I stated earlier. I haven't quite worked out the details yet and I'm not really even sure if it's the best thing to do. I created a sample interface comprised of folders and links and then tarred and gzipped it. I'm looking forward to some reactions on it. (look in ./driver)

Also I have two questions:

1.) Is it worth it to remove the bus interface?
my answer: I think it is because an interface in which drivers can have children is far more flexable and scaleable. Also it would result in less code. I'm looking for some feedback so I can revise my current efforts.

2.) Should driver management occur through driver model interfaces?
my answer: I already have the code to do this, it's just a matter of what's the best way to manage drivers. I feel that the driver model is the best place because it offers the most flexability and it allows for control over all drivers, not just modules.

Thanks,
Adam

PS: I'm currently working on a patch that just implements the read for "driver" as discussed earlier.

Hm, I think you're missing the whole point about classes. You are trying to do to the driver code, what will be done with the class code.

Here's the interaction:
- devices have a driver bound to them
- devices live in the /root tree, showing how they are interconnected.
- drivers register with a bus (possibly more than one.)
- drivers bind to a device present on a bus, and a class (some drivers bind to many classes)
- classes interact with userspace somehow, providing the interface between the device and the user.

As an example:
- A USB keyboard lives in the device tree.
- The USB HID driver binds to the device, and the input class (both the keyboard and generic subclasses of the input code.)
- the user types keys, and that data gets sent from the USB code, to the HID driver, to the input core, which then translates them and sends the info out the /dev node.

Within your model, you are not accounting for the different classes (input, serial, usb-serial, tty, disk, video, etc.). Take a look at the documentation for all of this for more information.

No.

No. Let's leave that the way things are today in this regards.

thanks,

greg k-h

You're missing the point of the bus driver structure. It is simply there to group all the devices on all the instances of that particular bus in the system. It doesn't care about hierarchy, and what you're doing makes absolutely no sense in anything but a typical PC:

- Not all PCI bridges are Host/PCI bridges. They may exist on some other proprietary bus.

- In some embedded systems, USB controllers are sometimes root devices

- There are such things as PCI->Cardbus->USB->PCI bridge device pathologies

AFAICT, your model assumes that there is only one bus of each type in the system, and the hierarchical relationship always holds. It doesn't. And, in order to get the relationship right, it seems way more complicated than what exists now.

You're also missing the distinction between bus drivers and device drivers. Device drivers control a particular device. This includes bridge devices from one bus type to another, like USB host controllers. These are currently represented in the current model:

[mochel@cherise mochel]$ tree -d /sys/bus/pci/
/sys/bus/pci/

|-- devices
|   |-- 00:00.0 -> ../../../root/pci0/00:00.0
|   |-- 00:01.0 -> ../../../root/pci0/00:01.0
|   |-- 00:02.0 -> ../../../root/pci0/00:02.0
|   |-- 00:1e.0 -> ../../../root/pci0/00:1e.0
|   |-- 00:1f.0 -> ../../../root/pci0/00:1f.0
|   |-- 00:1f.1 -> ../../../root/pci0/00:1f.1
|   |-- 00:1f.2 -> ../../../root/pci0/00:1f.2
|   |-- 00:1f.3 -> ../../../root/pci0/00:1f.3
|   |-- 00:1f.5 -> ../../../root/pci0/00:1f.5
|   |-- 01:00.0 -> ../../../root/pci0/00:01.0/01:00.0
|   |-- 02:1f.0 -> ../../../root/pci0/00:02.0/02:1f.0
|   |-- 03:00.0 -> ../../../root/pci0/00:02.0/02:1f.0/03:00.0
|   `-- 04:04.0 -> ../../../root/pci0/00:1e.0/04:04.0
`-- drivers
    |-- Intel ICH
    |-- Intel ICH Joystick
    |-- agpgart
    |-- e100
    |-- matroxfb
    |-- serial
    `-- uhci-hcd
        ^^^^^^^^

Note also the above. The symlinks have every PCI device on every PCI bus (all 3 of them) in my system, in a flat namespace. It also has every PCI driver in a flat namespace. As you can see in device_attach() and driver_attach(), we exploit these flat namespaces to bind drivers to devices, regardless of wherever the device resides.

That's it. Nothing fancy, and completely agnostic of where we found a particular bus.

As far as the lp driver goes: fix it. It's been on my list for sometime, but I havne't had a chance to get to it. But, I'm not going to change the driver model to special case legacy devices.

But, that's a _bad_ thing. Bus drivers and device drivers are completely separate entities, with completely separate semantics. Why munge them together? That only convolutes the model and the code.

man 8 modprobe

man 5 modules.conf

man 8 rmmod

I've been meaning to do both for sometime. I'll consider a patch that does that (and just that).

-pat

This is not a problem. My code does not force pci to be the root driver.

Once again not a problem.

This causes me to change my position. Drivers should be aranged in the following manner while using my patch. /driverfs
/driverfs/driver
/driverfs/driver/pci
/driverfs/driver/pci/agpgart
/driverfs/driver/pci/parport
/driverfs/driver/pci/parport/lp
/driverfs/driver/pci/parport/parport-ide
/driverfs/driver/pci/serial
/driverfs/driver/pci/cardbus
/driverfs/driver/usb
/driverfs/driver/usb/usb-storage
/driverfs/driver/usb/hid

The code in my patch can do this now. the only problem is parport could belong to a bus other than pci. The current model (without my patch) isn't even capable of presenting parport properly. Parport has IEEE 1284 transfers and therefore has device ids. The truth is parport could almost be considered a bus. It already allows for drivers to be attached to it. And parport is just one example. These drivers of course can not be represented in the current model.

Here's a more experimental way to go about it that would solve the problem listed above: (> = link) /driverfs
/driverfs/driver
/driverfs/driver/pci
/driverfs/driver/sys
/driverfs/driver/usb
/driverfs/driver/ide
/driverfs/driver/agpgart
/driverfs/driver/parport
/driverfs/driver/serial
/driverfs/driver/lp
/driverfs/driver/hid
/driverfs/driver/pci/agpgart>
/driverfs/driver/pci/parport>
/driverfs/driver/pci/serial>
/driverfs/driver/pci/usb>
/driverfs/driver/pci/ide>
/driverfs/driver/sys/parport>
/driverfs/driver/sys/serial>
/driverfs/driver/usb/hid>
/driverfs/driver/parport/ide>
/driverfs/driver/parport/lp>
/driverfs/driver/serial/lp>

This is just something to think about. Drivers could attach to multiple drivers and create links for each driver that they're attached to. The links would create an illusion to the user that there was a complex driver tree. Therefore this idea is simple yet ellegant. I just came up with this right now and actually I like it quite a bit. Links like this would be easy as pie to make. All I'd have to do is change the make_dir code as well as a couple things in adm.c.

lp isn't the only parport child driver. parport-ide also comes to mind. There probably are more and there could be more added in the future.

Look at the code for yourself, I've attached it to the end of the email. It's not munged or convoluted at all. If anything it's more efficient than the previous code.

I am fully aware of modutils. The problem is modutils should not manage drivers. I feel its most important function is to load code into a running kernel. The driver model, as it already does manage drivers including those loaded as modules, should do the rest. If a user wants more control over the driver attaching process he or she can use "driver". I may even add more functionality to it later. When contrasting it's similair to the relationship between a manual and automatic. More user control is always better. In this case you get an automatic with a manual override. When designing an interface that is going to unite all the existing methods it has to be scaleable. 3rd party drivers will come in to play for some users and users may want to choose which drivers attach to which devices. This provides the foundation for such a system.

The ability to remove drivers is also important becuase if you rmmod a module then it will detach from all devices it controls. What if the user wanted it to be removed only from one particular device?

Ok.

Thanks,
Adam
/*from device.h */
struct device_driver {
char * name;
struct device_driver * parent;

rwlock_t lock;
atomic_t refcount;

   list_t            node;         /* node in parent driver's list */
   list_t            children;
   list_t            attached_devices;/* devices the driver is attached 

to */
list_t child_devices; /* devices attached to the
devices in
in "attached_devices" */

struct driver_dir_entry dir;

   int    (*probe)    (struct device * dev);
   int     (*remove)    (struct device * dev);

int (*suspend) (struct device * dev, u32 state, u32 level);
int (*resume) (struct device * dev, u32 level);

void (*release) (struct device_driver * drv);
int (*match) (struct device * dev, struct device_driver * drv);

};

/*
* driver.c - centralized device driver management
*
*/

#define DEBUG 0

#include <linux/device.h>
#include <linux/module.h>
#include <linux/errno.h>
#include <linux/init.h>
#include "base.h"

static LIST_HEAD(global_driver_list);

static struct driver_dir_entry driver_root_dir = {
name: "driver",
mode: (S_IFDIR| S_IRWXU | S_IRUGO | S_IXUGO),
};

int driver_for_each_attached_dev(struct device_driver * drv, void * data, int (*callback)(struct device *, void * )) {
struct device * next;
struct device * dev = NULL;
struct list_head * node;
int error = 0;

   get_driver(drv);
   read_lock(&drv->lock);
   node = drv->attached_devices.next;
   while (node != &drv->attached_devices) {
       next = list_entry(node,struct device,driver_list);
       get_device(next);
       read_unlock(&drv->lock);

       if (dev)
           put_device(dev);
       dev = next;
       if ((error = callback(dev,data))) {
           put_device(dev);
           break;
       }
       read_lock(&drv->lock);
       node = dev->driver_list.next;
   }
   read_unlock(&drv->lock);
   if (dev)
       put_device(dev);

put_driver(drv);
return error;
}

int driver_for_each_child_dev(struct device_driver * bus, void * data,
int (*callback)(struct device * dev, void * data))
{
struct device * next;
struct device * dev = NULL;
struct list_head * node;
int error = 0;

   get_driver(bus);
   read_lock(&bus->lock);
   node = bus->child_devices.next;
   while (node != &bus->child_devices) {
       next = list_entry(node,struct device,bus_list);
       get_device(next);
       read_unlock(&bus->lock);

       if (dev)
           put_device(dev);
       dev = next;
       if ((error = callback(dev,data))) {
           put_device(dev);
           break;
       }
       read_lock(&bus->lock);
       node = dev->bus_list.next;
   }
   read_unlock(&bus->lock);
   if (dev)
       put_device(dev);

put_driver(bus);
return error;
}

int driver_for_each_drv(struct device_driver * bus, void * data,
int (*callback)(struct device_driver * drv, void * data))
{
struct device_driver * next;
struct device_driver * drv = NULL;
struct list_head * driver;
int error = 0;

   /* pin bus in memory */
   get_driver(bus);

   read_lock(&bus->lock);
   driver = bus->children.next;
   while (driver != &bus->children) {
       next = list_entry(driver,struct device_driver,node);
       get_driver(next);
       read_unlock(&bus->lock);

       if (drv)
           put_driver(drv);
       drv = next;
       if ((error = callback(drv,data))) {
           put_driver(drv);
           break;
       }
       read_lock(&bus->lock);
       driver = drv->node.next;
   }
   read_unlock(&bus->lock);
   if (drv)
       put_driver(drv);

put_driver(bus);
return error;
}

/**
* drv_add_device - add device to driver
* @dev: device being added
*
* Add the device to its driver's list of child devices.
* Try and bind the device to a driver.
*/
int driver_add_device(struct device * dev)
{
if (dev->bus) {
pr_debug("registering %s with bus

'%s'\n",dev->bus_id,dev->bus->name);
       get_driver(dev->bus);
       write_lock(&dev->bus->lock);
       list_add_tail(&dev->bus_list,&dev->bus->child_devices);
       write_unlock(&dev->bus->lock);
}

return 0;
}

/**
* drv_remove_device - remove device from driver
* @dev: device to be removed
*
* Delete device from drivers's list.
*/
void driver_remove_device(struct device * dev)
{

   if (dev->bus) {
       write_lock(&dev->bus->lock);
       list_del_init(&dev->bus_list);
       write_unlock(&dev->bus->lock);
       put_driver(dev->bus);

}
}

/**
* driver_make_dir - create a driverfs directory for a driver
* @drv: driver in question
*/

static int driver_make_dir(struct device_driver * drv) {
int error;
struct driver_dir_entry * parent = NULL; drv->dir.name = drv->name;
if (drv->parent)

       parent = &drv->parent->dir;
   else
       parent = &driver_root_dir;

error = device_create_dir(&drv->dir,parent);
return error;
}

/**
* driver_register - register driver with bus driver
* @drv: driver to register
*
* Add to parent driver's list of drivers
*/
int driver_register(struct device_driver * drv) {
atomic_set(&drv->refcount,2); rwlock_init(&drv->lock); INIT_LIST_HEAD(&drv->attached_devices); INIT_LIST_HEAD(&drv->child_devices); INIT_LIST_HEAD(&drv->children); INIT_LIST_HEAD(&drv->node);
if (drv->parent){
pr_debug("Registering driver '%s' with bus

'%s'\n",drv->name,drv->parent->name);
       get_driver(drv->parent);
       list_add_tail(&drv->node,&drv->parent->children);
   }else
       list_add_tail(&drv->node,&global_driver_list);
   driver_make_dir(drv);
   if (drv->parent)
       driver_attach(drv);

put_driver(drv);
return 0;
}

static void __remove_driver(struct device_driver * drv) {
pr_debug("Unregistering driver '%s' from bus
'%s'\n",drv->name,drv->parent->name);
driver_detach(drv); driverfs_remove_dir(&drv->dir);
if (drv->release)

       drv->release(drv);

put_driver(drv->parent);
}

void remove_driver(struct device_driver * drv)
{
write_lock(&drv->parent->lock);
atomic_set(&drv->refcount,0);
list_del_init(&drv->node);
write_unlock(&drv->parent->lock);
__remove_driver(drv);
}

/**
* put_driver - decrement driver's refcount and clean up if necessary
* @drv: driver in question
*/
void put_driver(struct device_driver * drv)
{
if (!atomic_dec_and_lock(&drv->refcount,&device_lock))

       return;

list_del_init(&drv->node);
spin_unlock(&device_lock);
__remove_driver(drv);
}

static int __init drv_init(void)
{
return driverfs_create_dir(&driver_root_dir,NULL);
}

core_initcall(drv_init);

EXPORT_SYMBOL(driver_for_each_dev);
EXPORT_SYMBOL(driver_for_each_child_dev);
EXPORT_SYMBOL(driver_for_each_drv);
EXPORT_SYMBOL(driver_register);
EXPORT_SYMBOL(put_driver);
EXPORT_SYMBOL(remove_driver);