Device Resource Management
RFC 1/1 Device Resource Management
Hi All,
This patch is a draft for the core infustructure of driver model based resource management. Buses capable of dynamically allocating resources (such as PCI or pcmcia), and recent developments with linux ACPI support have incited the need for the kernel to keep track of system resources, and assist in the allocation process. Eventually I'd like to move on to more complex problems like resource rebalancing. This patch simply lays out some foundation.
We currently have some infustructure, but it is more or less incomplete. For example, it doesn't show how resources and the devices that consume them are related. Also it isn't flexable enough to track all types of bus resources. ACPI, PnP, and PCI have to allocate interrupts somewhat blindly because request_irq happens too late in the game, and really is used to register interrupt handlers, not track interrupt usage.
My implementation is not based on the existing "struct resource" stuff because quite frankly, even making small changes would break a large number of drivers. Instead it uses "struct iores" My hope is that we can gradually phase the old implementation out.
With that in mind, I'd like to work out a solid new API, and would appreciate any comments or suggestions, both on the code in this patch and the overall design of the API.
A few issues need to be discussed. The most important is probably how sysfs will display resource usage? "struct kobject" is actually larger then "struct iores" so I don't feel very comfortable embedding it. But, if a kobject based sysfs resource tree would be needed, then I'd be happy to implement it. If not what would be a good alternative? Does the user have to read resources from each device like one would with this current patch?
Thanks,
Adam
This patch was only tested for compilation so it may have a few bugs/typos. Once again I'd really appreciate any comments or suggestions.
drivers/base/Makefile +1 -1
@@ -3,7 +3,7 @@ obj-y := core.o sys.o interface.o bus.o \ driver.o class.o class_simple.o platform.o \ cpu.o firmware.o init.o map.o dmapool.o -obj-y += power/ +obj-y += power/ resource/ obj-$(CONFIG_FW_LOADER) += firmware_class.o obj-$(CONFIG_NUMA) += node.o
drivers/base/resource/bars.c +42 -0
42 lines not on this page · raw message
drivers/base/resource/base.h +19 -0
@@ -0,0 +1,19 @@ +/* +* base.h - internal resource management header +* +* Copyright (c) 2004 Adam Belay +* +* This file is released under the GPLv2 +*/ + +struct iores_type { +char *name; +spinlock_t lock; +struct iores *default_root; + +struct iores * (*conflict) (struct iores *new,struct iores *root, +struct iores *from); +}; + +extern struct iores_type *iores_get_type (struct iores *res); +
drivers/base/resource/device.c +69 -0
69 lines not on this page · raw message
drivers/base/resource/iores.c +306 -0
306 lines not on this page · raw message
drivers/base/resource/Makefile +9 -0
@@ -0,0 +1,9 @@ +# +# Makefile for Linux Resource Management. +# + +obj-y := bars.o device.o iores.o sysfs.o + +ifeq ($(CONFIG_DEBUG_DRIVER),y) +EXTRA_CFLAGS += -DDEBUG +endif
drivers/base/resource/sysfs.c +145 -0
145 lines not on this page · raw message
include/linux/device.h +15 -0
@@ -13,6 +13,7 @@#include <linux/config.h> #include <linux/ioport.h> +#include <linux/iores.h> #include <linux/kobject.h> #include <linux/list.h> #include <linux/spinlock.h>
@@ -41,6 +42,11 @@ RESUME_ENABLE, }; +enum { +DEVICE_DISABLED , +DEVICE_ENABLED , +}; +struct device ;struct device_driver ;struct class ;
@@ -63,6 +69,10 @@ int num_envp, char *buffer, int buffer_size);int (*suspend)(struct device * dev,u32 state);int (*resume)(struct device * dev); + +int (*enable)(struct device *dev); +int (*disable)(struct device *dev); +int (*assign)(struct device *dev); };extern int bus_register (struct bus_type * bus);
@@ -282,6 +292,11 @@64 bit addresses for consistent allocations such descriptors. */ +int resource_state;/* specifies whether the device is +decoding resources */ +unsigned int num_resources;/* number of resources in res */ +struct iores *res;/* a table of resources */ +struct list_head dma_pools;/* dma pools (if dma'ble) */ struct dma_coherent_mem *dma_mem;/* internal for coherent mem
include/linux/iores.h +148 -0
148 lines not on this page · raw message
Quoted 11 lines · Adam Belay
Hi All,
This patch is a draft for the core infustructure of driver model based resource management. Buses capable of dynamically allocating resources (such as PCI or pcmcia), and recent developments with linux ACPI support have incited the need for the kernel to keep track of system resources, and assist in the allocation process. Eventually I'd like to move on to more complex problems like resource rebalancing. This patch simply lays out some foundation.
Quoted 6 lines
We currently have some infustructure, but it is more or less incomplete. For example, it doesn't show how resources and the devices that consume them are related. Also it isn't flexable enough to track all types of bus resources. ACPI, PnP, and PCI have to allocate interrupts somewhat blindly because
request_irqhappens too late in the game, and really is used to register interrupt handlers, not track interrupt usage.
But the interrupts are assigned early in the process, right? We still need to do that.
Quoted 4 lines
My implementation is not based on the existing "struct resource" stuff because quite frankly, even making small changes would break a large number of drivers. Instead it uses "struct iores" My hope is that we can gradually phase the old implementation out.
Ah, what's wrong with a little breakage :)
Quoted 10 lines
With that in mind, I'd like to work out a solid new API, and would appreciate any comments or suggestions, both on the code in this patch and the overall design of the API.
A few issues need to be discussed. The most important is probably how sysfs will display resource usage? "struct kobject" is actually larger then "struct iores" so I don't feel very comfortable embedding it. But, if a kobject based sysfs resource tree would be needed, then I'd be happy to implement it. If not what would be a good alternative? Does the user have to read resources from each device like one would with this current patch?
Why would it matter if we export this info to userspace? Do any userspace programs care about this information?
Quoted 2 lines
This patch was only tested for compilation so it may have a few bugs/typos. Once again I'd really appreciate any comments or suggestions.
It looks good. What's the main goals of this redesign (for those of us who are old and forgot what you said at the last kernel summit...)
thanks,
greg k-h
Quoted 19 lines · Greg KH
Quoted 11 lines · Adam Belay
Hi All,
This patch is a draft for the core infustructure of driver model based resource management. Buses capable of dynamically allocating resources (such as PCI or pcmcia), and recent developments with linux ACPI support have incited the need for the kernel to keep track of system resources, and assist in the allocation process. Eventually I'd like to move on to more complex problems like resource rebalancing. This patch simply lays out some foundation.
Quoted 6 lines
We currently have some infustructure, but it is more or less incomplete. For example, it doesn't show how resources and the devices that consume them are related. Also it isn't flexable enough to track all types of bus resources. ACPI, PnP, and PCI have to allocate interrupts somewhat blindly because
request_irqhappens too late in the game, and really is used to register interrupt handlers, not track interrupt usage.
Quoted 2 lines
But the interrupts are assigned early in the process, right? We still need to do that.
Yes, interrupts are usually assigned much earlier than they are serviced by device drivers.
Quoted 7 lines
Quoted 5 lines
My implementation is not based on the existing "struct resource" stuff because quite frankly, even making small changes would break a large number of drivers. Instead it uses "struct iores" My hope is that we can gradually phase the old implementation out.
Ah, what's wrong with a little breakage :)
I can work with that approach too :) I was just concerned with some of the not so testable legacy drivers.
Quoted 12 lines
Quoted 11 lines
With that in mind, I'd like to work out a solid new API, and would appreciate any comments or suggestions, both on the code in this patch and the overall design of the API.
A few issues need to be discussed. The most important is probably how sysfs will display resource usage? "struct kobject" is actually larger then "struct iores" so I don't feel very comfortable embedding it. But, if a kobject based sysfs resource tree would be needed, then I'd be happy to implement it. If not what would be a good alternative? Does the user have to read resources from each device like one would with this current patch?
Quoted 2 lines
Why would it matter if we export this info to userspace? Do any userspace programs care about this information?
It depends if we want userspace to handle any resource management policy. If it doesn't, then more complex PnP algorithms will have to be implemented in kernel. Userspace control is more important for old technology where resource assignment can be less flexable. Examples would include ISAPnP and PCMCIA. The majority of systems could probably get by without userspace control if we are able to map out tricky things like the 10-bit decode VGA I/O in kernel.
Pat suggested a resource tree in sysfs with each item in the tree linking back to the device that consumes it. Kobject size, and strange locking/lifetime challenges would be the major disadvantages. Also a method for making the actual assignments would have to be worked out.
Another reason userspace would care about this information is that it could be really useful for debugging hardware problems.
Quoted 6 lines
Quoted 3 lines
This patch was only tested for compilation so it may have a few bugs/typos. Once again I'd really appreciate any comments or suggestions.
It looks good. What's the main goals of this redesign (for those of us who are old and forgot what you said at the last kernel summit...)
Basically there are three goals:
1.) To solve the resource allocation problems we currently see with ACPI and other bus/firmware technologies and fullfill their requirements.
2.) To integrate the tracking of bus resources into the driver model-- This will allow us to more accurately track device dependencies. Just like there are power parents/domains in power management, there are resource producers in resource management. Also it should allow us to remove a bunch of duplicate code between bus drivers.
3.) To support resource rebalancing-- This will likely become necessary as PCI express gains wider usage because it will allow for more complex PCI bridge hierarchies. A device could be paused, and it's resources could be adjusted to accommodate a newly hotplugged device etc.
I appreciate the comments.
Thanks,
Adam
Quoted 11 lines · Adam Belay
Quoted 7 lines · Greg KH
Quoted 4 lines · Adam Belay
My implementation is not based on the existing "struct resource" stuff because quite frankly, even making small changes would break a large number of drivers. Instead it uses "struct iores" My hope is that we can gradually phase the old implementation out.
Ah, what's wrong with a little breakage :)
I can work with that approach too :) I was just concerned with some of the not so testable legacy drivers.
Well, if they build properly, that's good enough to start with... You know the drill, you did it before...
Quoted 22 lines
Quoted 11 lines
Quoted 10 lines
With that in mind, I'd like to work out a solid new API, and would appreciate any comments or suggestions, both on the code in this patch and the overall design of the API.
A few issues need to be discussed. The most important is probably how sysfs will display resource usage? "struct kobject" is actually larger then "struct iores" so I don't feel very comfortable embedding it. But, if a kobject based sysfs resource tree would be needed, then I'd be happy to implement it. If not what would be a good alternative? Does the user have to read resources from each device like one would with this current patch?
Quoted 2 lines
Why would it matter if we export this info to userspace? Do any userspace programs care about this information?
It depends if we want userspace to handle any resource management policy. If it doesn't, then more complex PnP algorithms will have to be implemented in kernel. Userspace control is more important for old technology where resource assignment can be less flexable. Examples would include ISAPnP and PCMCIA. The majority of systems could probably get by without userspace control if we are able to map out tricky things like the 10-bit decode VGA I/O in kernel.
Quoted 4 lines
Pat suggested a resource tree in sysfs with each item in the tree linking back to the device that consumes it. Kobject size, and strange locking/lifetime challenges would be the major disadvantages. Also a method for making the actual assignments would have to be worked out.
Ok, that makes sense. I wouldn't worry about the kobject size. I don't think that we will run into a lot of different resources on a lot of different devices. Today, the only kobject size issue is for thousands of storage devices, and those issues are already taken care of.
We can always work on shrinking the kobject size (it's on my todo list) if it gets too big and becomes an issue here.
So in short, add it to your structure and we can go from there.
Quoted 17 lines
Quoted 5 lines
Quoted 2 lines
This patch was only tested for compilation so it may have a few bugs/typos. Once again I'd really appreciate any comments or suggestions.
It looks good. What's the main goals of this redesign (for those of us who are old and forgot what you said at the last kernel summit...)
Basically there are three goals:
1.) To solve the resource allocation problems we currently see with ACPI and other bus/firmware technologies and fullfill their requirements.
2.) To integrate the tracking of bus resources into the driver model-- This will allow us to more accurately track device dependencies. Just like there are power parents/domains in power management, there are resource producers in resource management. Also it should allow us to remove a bunch of duplicate code between bus drivers.
Quoted 4 lines
3.) To support resource rebalancing-- This will likely become necessary as PCI express gains wider usage because it will allow for more complex PCI bridge hierarchies. A device could be paused, and it's resources could be adjusted to accommodate a newly hotplugged device etc.
Thanks for reminding me. All good goals to have.
There is a PCI Express patch on lkml that you might want to take a look at if you are concerned as to how that might play out.
thanks,
greg k-h
Quoted 2 lines · Greg KH
Why would it matter if we export this info to userspace? Do any userspace programs care about this information?
We'd love to be able to view and manipulate resource information from HAL. As HAL replaces vendor-specific solutions such as, say, kudzu, it will need to make device/driver decisions and implement work arounds, so this information is incredibly invaluable, let alone just neat to have.
Note _requiring_ user-space to implement the resource allocation strategy is an entirely different path, and I am not arguing that. We'd like to read it, if nothing else.
Robert Love
Two issues that we have discussed for embedded usage would be making resources 64-bit always. We are starting to see more and more systems with greater than 32-bits of physical address space while still having 32-bit effective. The second is sharing resources. On several devices the IO memory region for one device may reside inside another. One example of this is the registers used to control ethernet MII PHYs exist inside of an ethernet controllers register block.
Also, I liked having a name in resource. It allows us to find the specific resource we need if we have multiple resources of a given type without assuming order. And thus adding a device_find_iores_bynames().
- kumar
Quoted 9 lines · Robert Love
Quoted 2 lines · Greg KH
Why would it matter if we export this info to userspace? Do any userspace programs care about this information?
We'd love to be able to view and manipulate resource information from HAL. As HAL replaces vendor-specific solutions such as, say, kudzu, it will need to make device/driver decisions and implement work arounds, so this information is incredibly invaluable, let alone just neat to have.
Ok, fair enough. I keep forgetting about the PnP resource mess, I guess I just want to block that nastiness out of my brain...
thanks,
greg k-h