Hardware Abstraction Layer

DISCUSSION David Zeuthen 4 participants 19 messages Sep 13, 2003 – Oct 1, 2003

lore

Share & tools

lore

David Zeuthen

author

2003-09-13 13:30 UTC #0 lore raw

Hi,

I'm currently working on a hardware abstraction layer, see

http://pdx.freedesktop.org/~hal/

for use in desktop environments such as GNOME or KDE. The HAL project is in it's early stages - there is a draft spec planned and I got an implementation running as well.

I want to hear how you think we can cooperate, by e.g. having /sbin/hotplug send events to hald. Note that the HAL aims at a range of UNIX-like systems, but I think we can handle that requirement... I use GNU/Linux myself.

There is also a discussion going over at the xdg-list at freedesktop.org, you might want to out check the archives.

Let me know what you think. Comments are appreciated. Note that I'm out traveling until late Tuesday with no access to e-mail so I won't be able to reply for while.

Thank you for your time!

Regards,
David

-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- David Zeuthen pgp key: http://fubar.dk/pgpkey.asc
- david(at)fubar.dk pgp key id: b89bab82

-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Joerg Sommer

2003-09-14 22:34 UTC #1 lore raw

Quoted 3 lines · begin David Zeuthen

I'm currently working on a hardware abstraction layer, see
http://pdx.freedesktop.org/~hal/

Nice.

Quoted 4 lines

I want to hear how you think we can cooperate, by e.g. having /sbin/hotplug send events to hald. Note that the HAL aims at a range of UNIX-like systems, but I think we can handle that requirement... I use GNU/Linux myself.

if you take a look at /sbin/hotplug, you see it calls all files in /etc/hotplug.d/default/*.hotplug. So if you place their a script you receive all hotplug events and it then up to you to react on them.

Jörg.

Greg KH

2003-09-16 16:39 UTC #2 lore raw

In reply to David Zeuthen

Quoted 9 lines · David Zeuthen

Hi,
I'm currently working on a hardware abstraction layer, see
http://pdx.freedesktop.org/~hal/
for use in desktop environments such as GNOME or KDE. The HAL project is in it's early stages - there is a draft spec planned and I got an implementation running as well.

How does this differ from D-BUS? Hm, in looking at your page, it says you are using D-BUS messages. Oh, have you looked at udev too? It will handle naming the devices for you in /dev.

Quoted 4 lines

I want to hear how you think we can cooperate, by e.g. having /sbin/hotplug send events to hald. Note that the HAL aims at a range of UNIX-like systems, but I think we can handle that requirement... I use GNU/Linux myself.

As was already posted, anyone can get /sbin/hotplug events by just dropping a link into the proper directory. Also, D-BUS messages will be created for all /sbin/hotplug events too. You might want to work with that if it's easier for you.

Hope this helps,

greg k-h

David Zeuthen

author

2003-09-17 00:40 UTC #3 lore raw

In reply to David Zeuthen

Quoted 5 lines · Greg KH

Quoted 4 lines · David Zeuthen
Hi,
I'm currently working on a hardware abstraction layer, see

Quoted 8 lines

Quoted 5 lines
http://pdx.freedesktop.org/~hal/
for use in desktop environments such as GNOME or KDE. The HAL project is in it's early stages - there is a draft spec planned and I got an implementation running as well.
How does this differ from D-BUS? Hm, in looking at your page, it says you are using D-BUS messages.

I use D-BUS as the IPC mechanism between hotplug agents and applications wanting to query a device database. In the middle there is a daemon.

In a sense I want the entire HAL not to care about specific hardware issues at all, but build on existing software like linux-hotplug ;-) (see my recent post to the xdg-list for detail)

Quoted 3 lines

Oh, have you looked at udev too? It will
handle naming the devices for you in /dev.

Excellent - will look into this later. Will this help me, e.g., getting the information that my CF card reader is at /dev/sda1 when using the kernel module usb-storage? Or have I misunderstood?

Quoted 7 lines

Quoted 4 lines
I want to hear how you think we can cooperate, by e.g. having /sbin/hotplug send events to hald. Note that the HAL aims at a range of UNIX-like systems, but I think we can handle that requirement... I use GNU/Linux myself.
As was already posted, anyone can get /sbin/hotplug events by just dropping a link into the proper directory.

Cool.

Quoted 4 lines

Also, D-BUS messages will be
created for all /sbin/hotplug events too. You might want to work with that if it's easier for you.

I'll be interested in this, definately. Are you using the service org.freedesktop.DBus.Broadcast for this?

I'll also need a unique ID of each device being hotplugged, and I have some problems seeing this is even possible on some busses like USB. The unique ID must be the same over plugs/unplugs.

Say, that I plug two identical cameras into the system with no device-instance specific information. Corner case, but nasty one...

In the event this is not possible the hotplug agent must somehow convey this information so I can append a counter to make the ID unique in that way..

Any suggestions / comments ?

Quoted 2 lines

Hope this helps,

Very much, Thanks,
David

Greg KH

2003-09-18 00:29 UTC #4 lore raw

In reply to David Zeuthen

Quoted 15 lines · David Zeuthen

Quoted 5 lines · Greg KH
Quoted 4 lines · David Zeuthen
Hi,
I'm currently working on a hardware abstraction layer, see
Quoted 8 lines
Quoted 5 lines
http://pdx.freedesktop.org/~hal/
for use in desktop environments such as GNOME or KDE. The HAL project is in it's early stages - there is a draft spec planned and I got an implementation running as well.
How does this differ from D-BUS? Hm, in looking at your page, it says you are using D-BUS messages.

Quoted 6 lines

I use D-BUS as the IPC mechanism between hotplug agents and applications wanting to query a device database. In the middle there is a daemon.
In a sense I want the entire HAL not to care about specific hardware issues at all, but build on existing software like linux-hotplug ;-) (see my recent post to the xdg-list for detail)

Hm, in looking over your post I think you are going to be duplicating a lot of existing information that is dynamically available to you from the kernel.

For example, the kernel already knows what USB devices are supported by what drivers. It exports that info to userspace in the modules.*map files in /lib/modules/<KERNEL_VERSION>/

The existing hotplug scripts already handle the mapping from device to driver, so you don't need to duplicate that functionaity.

Quoted 7 lines

Quoted 3 lines
Oh, have you looked at udev too? It will
handle naming the devices for you in /dev.
Excellent - will look into this later. Will this help me, e.g., getting the information that my CF card reader is at /dev/sda1 when using the kernel module usb-storage? Or have I misunderstood?

Yes it will. See my OLS paper from this year for lots more information about it.

Quoted 7 lines

Quoted 4 lines
Also, D-BUS messages will be
created for all /sbin/hotplug events too. You might want to work with that if it's easier for you.
I'll be interested in this, definately. Are you using the service org.freedesktop.DBus.Broadcast for this?

I'm not using anything :)

There was a simple command line program that generated dbus messages. Someone needs to take that and have it convert hotplug messages into dbus messages. I think it's going to use the org.kernel namespace.

Quoted 3 lines

I'll also need a unique ID of each device being hotplugged, and I have some problems seeing this is even possible on some busses like USB. The unique ID must be the same over plugs/unplugs.

What "unique ID"? In short, you are never guaranteed such a thing, but you can get close. See my udev paper for more info on this.

Quoted 2 lines

Say, that I plug two identical cameras into the system with no device-instance specific information. Corner case, but nasty one...

Use the bus topology to distinguish between those two devices.

Quoted 3 lines

In the event this is not possible the hotplug agent must somehow convey this information so I can append a counter to make the ID unique in that way..

The kernel will tell you that this is a new device. It's up to userspace to do something with that info. udev will determine the /dev name for the device. It's up to other programs to do other things with this info (which is what I'm thinking your proposal does, correct?)

thanks,

greg k-h

David Zeuthen

author

2003-09-18 11:24 UTC #5 lore raw

In reply to David Zeuthen

Quoted 19 lines · Greg KH

Quoted 18 lines · David Zeuthen
Quoted 5 lines · Greg KH
Quoted 4 lines · David Zeuthen
Hi,
I'm currently working on a hardware abstraction layer, see
Quoted 8 lines
Quoted 5 lines
http://pdx.freedesktop.org/~hal/
for use in desktop environments such as GNOME or KDE. The HAL project is in it's early stages - there is a draft spec planned and I got an implementation running as well.
How does this differ from D-BUS? Hm, in looking at your page, it says you are using D-BUS messages.
I use D-BUS as the IPC mechanism between hotplug agents and applications wanting to query a device database. In the middle there is a daemon.

Quoted 8 lines

Quoted 3 lines
In a sense I want the entire HAL not to care about specific hardware issues at all, but build on existing software like linux-hotplug ;-) (see my recent post to the xdg-list for detail)
Hm, in looking over your post I think you are going to be duplicating a lot of existing information that is dynamically available to you from the kernel.

I see your point. You specifically mention the word ''post'', so I'll assume that you are talking about my initial message to the xdg-list. This post mentions both ''drivers'' and ''kernel modules'', which I admit is confusing.

In the latest version, available at the link above, I specifically avoid using these words and only talk about ''device info files'' and ''boot programs''.

The contention is that properties for a device should contain a) bus-device-specific properties (such as usb.idVendor, usb.bcdDevice, usb.linux_devfs etc.) and category-specific properties (such as Storage.Device=/dev/sda1 and Storage.MountPoint=/mnt/flash).

Applications of the HAL (e.g. desktop environments) will then use standard kernel and OS interfaces to e.g. display the amount of free space on a Storage device.

I hope this clear matters up?

Quoted 7 lines

For example, the kernel already knows what USB devices are supported by what drivers. It exports that info to userspace in the modules.*map files in /lib/modules/<KERNEL_VERSION>/
The existing hotplug scripts already handle the mapping from device to driver, so you don't need to duplicate that functionaity.

You are right. Note that the HAL needs to be kernel / OS agnostic but the idea is to handle this at the device-info level. So, for Linux the BootProgram for a storage-device simply mounts the device somewhere chosen by the user or the desktop environment. On other OS'es the bootprogram may need to insert modules or whatever into a running kernel.

This implies that, for instance, /etc/hotplug/usb.hotplug will need to send a DBUS message *after* the kernel modules have been successfully loaded - see proposal below.

Side question: If a device vendor ships a (kernel) driver for his USB device will modules.*map get updated?

Quoted 11 lines

Quoted 7 lines
Quoted 3 lines
Oh, have you looked at udev too? It will
handle naming the devices for you in /dev.
Excellent - will look into this later. Will this help me, e.g., getting the information that my CF card reader is at /dev/sda1 when using the kernel module usb-storage? Or have I misunderstood?
Yes it will. See my OLS paper from this year for lots more information about it.

Great! I'll look into this soon.

Quoted 5 lines

Quoted 5 lines
Quoted 4 lines
Also, D-BUS messages will be
created for all /sbin/hotplug events too. You might want to work with that if it's easier for you.

Quoted 9 lines

Quoted 2 lines
I'll be interested in this, definately. Are you using the service org.freedesktop.DBus.Broadcast for this?
I'm not using anything :)
There was a simple command line program that generated dbus messages. Someone needs to take that and have it convert hotplug messages into dbus messages. I think it's going to use the org.kernel namespace.

My initial experiments was actually to patch /etc/hotplug/usb.hotplug and used the same command-line tool. The tool is a bit weak still (but DBUS is not at 1.0 yet) in so far that obvious data-types like dictionaries are not available yet. Now I just have a hackish C program using libdbus to parse the environment given from the kernel and send it off to the HAL daemon.

IMHO, the best possible scenario for HAL (wrt. hotplugging) would be to have a well-defined DBUS message scheme for each bus type and linux-hotplug to implement this broadcasting it through the service org.freedesktop.DBus.Broadcast.

Data in these messages would contain the usb.* fields and information about what (linux) kernel modules was loaded if any (cannot find device driver etc.).

If there is not a well-defined scheme yet, I'll volunteer to write a spec and implement it as part of linux-hotplug. We can put this spec at either the HAL site or the linux-hotplug site. The former would IMHO be better since we want the scheme to be OS agnostic, but I'm flexible. I'm quite certain the scheme, to an extent, can be OS agnostic.

What is your take on this?

Quoted 12 lines

Quoted 3 lines
I'll also need a unique ID of each device being hotplugged, and I have some problems seeing this is even possible on some busses like USB. The unique ID must be the same over plugs/unplugs.
What "unique ID"? In short, you are never guaranteed such a thing, but you can get close. See my udev paper for more info on this.
Quoted 2 lines
Say, that I plug two identical cameras into the system with no device-instance specific information. Corner case, but nasty one...
Use the bus topology to distinguish between those two devices.

Quoted 9 lines

Quoted 3 lines
In the event this is not possible the hotplug agent must somehow convey this information so I can append a counter to make the ID unique in that way..
The kernel will tell you that this is a new device. It's up to userspace to do something with that info. udev will determine the /dev name for the device. It's up to other programs to do other things with this info (which is what I'm thinking your proposal does, correct?)

Ok, the HAL daemon can take care of this as long as we agree on the DBUS message format. HAL is strictly user-space.

Thanks,
David

Greg KH

2003-09-18 17:36 UTC #6 lore raw

In reply to David Zeuthen

Quoted 29 lines · David Zeuthen

Quoted 19 lines · Greg KH
Quoted 18 lines · David Zeuthen
Quoted 5 lines · Greg KH
Quoted 4 lines · David Zeuthen
Hi,
I'm currently working on a hardware abstraction layer, see
Quoted 8 lines
Quoted 5 lines
http://pdx.freedesktop.org/~hal/
for use in desktop environments such as GNOME or KDE. The HAL project is in it's early stages - there is a draft spec planned and I got an implementation running as well.
How does this differ from D-BUS? Hm, in looking at your page, it says you are using D-BUS messages.
I use D-BUS as the IPC mechanism between hotplug agents and applications wanting to query a device database. In the middle there is a daemon.
Quoted 8 lines
Quoted 3 lines
In a sense I want the entire HAL not to care about specific hardware issues at all, but build on existing software like linux-hotplug ;-) (see my recent post to the xdg-list for detail)
Hm, in looking over your post I think you are going to be duplicating a lot of existing information that is dynamically available to you from the kernel.

Quoted 4 lines

I see your point. You specifically mention the word ''post'', so I'll assume that you are talking about my initial message to the xdg-list. This post mentions both ''drivers'' and ''kernel modules'', which I admit is confusing.

Yes, that is what I was referring to.

Quoted 8 lines

In the latest version, available at the link above, I specifically avoid using these words and only talk about ''device info files'' and ''boot programs''.
The contention is that properties for a device should contain a) bus-device-specific properties (such as usb.idVendor, usb.bcdDevice, usb.linux_devfs etc.) and category-specific properties (such as Storage.Device=/dev/sda1 and Storage.MountPoint=/mnt/flash).

But my main point is, why store this info anywhere, when it is already stored on your machine today? And it is automatically already handled properly today, without any need for "yet another abstraction layer" to handle it?

Yes, I know this only works on Linux, and you want to be "cross-platform". Might I suggest you get the other OSs that you want to support to also support what Linux already does, instead of trying to duplicate work?

Oh, what oses are you wanting to support? I think FreeBSD already handles much of what Linux does, with it's devfs implementation, but I haven't looked at that in a long time to verify this.

Quoted 5 lines

Applications of the HAL (e.g. desktop environments) will then use standard kernel and OS interfaces to e.g. display the amount of free space on a Storage device.
I hope this clear matters up?

Not really. I'm getting the impression that you don't realize the current state of Linux and how hotplugging and driver loading and device naming currently works. A lot of what you are saying you want to do, already works today.

The only "major" piece that is missing, is what udev is working on solving (the persistant device naming issue.) Now I agree that udev and the hotplug scripts, and the raw hotplug events should get sent using dbus events, but I was thinking that userspace programs (like GNOME and KDE) that cared about those events would be listening for them. I didn't think there needed to be a whole intermediate layer for these messages to have to go through. But hey, I'm probably missing something big here :)

Quoted 14 lines

Quoted 6 lines
For example, the kernel already knows what USB devices are supported by what drivers. It exports that info to userspace in the modules.*map files in /lib/modules/<KERNEL_VERSION>/
The existing hotplug scripts already handle the mapping from device to driver, so you don't need to duplicate that functionaity.
You are right. Note that the HAL needs to be kernel / OS agnostic but the idea is to handle this at the device-info level. So, for Linux the BootProgram for a storage-device simply mounts the device somewhere chosen by the user or the desktop environment. On other OS'es the bootprogram may need to insert modules or whatever into a running kernel.

Quoted 3 lines

This implies that, for instance, /etc/hotplug/usb.hotplug will need to send a DBUS message *after* the kernel modules have been successfully loaded - see proposal below.

That's fine.

Quoted 2 lines

Side question: If a device vendor ships a (kernel) driver for his USB device will modules.*map get updated?

Side question to your side question: What device vendor ships kernel drivers for USB devices? :)

Anyway, yes, it will get updated, next boot time when depmod is run. Actually in 2.6, those files are not needed at all, and people activly want to get rid of them. And I agree with them, it's just that the current hotplug scripts rely too much on them at this time... They should be rewritten as to not need them at all.

So be aware that they might go away in the future.

Quoted 15 lines

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Also, D-BUS messages will be
created for all /sbin/hotplug events too. You might want to work with that if it's easier for you.
Quoted 9 lines
Quoted 2 lines
I'll be interested in this, definately. Are you using the service org.freedesktop.DBus.Broadcast for this?
I'm not using anything :)
There was a simple command line program that generated dbus messages. Someone needs to take that and have it convert hotplug messages into dbus messages. I think it's going to use the org.kernel namespace.

Quoted 6 lines

My initial experiments was actually to patch /etc/hotplug/usb.hotplug and used the same command-line tool. The tool is a bit weak still (but DBUS is not at 1.0 yet) in so far that obvious data-types like dictionaries are not available yet. Now I just have a hackish C program using libdbus to parse the environment given from the kernel and send it off to the HAL daemon.

Nice.

Quoted 4 lines

IMHO, the best possible scenario for HAL (wrt. hotplugging) would be to have a well-defined DBUS message scheme for each bus type and linux-hotplug to implement this broadcasting it through the service org.freedesktop.DBus.Broadcast.

Hm, why that address? I thought the org.kernel namespace was going to be reserved for these types of messages. Ah, but the org.kernel name should be used for "raw" hotplug events (which some people might want to listen for). Ok, I understand now.

Quoted 3 lines

Data in these messages would contain the usb.* fields and information about what (linux) kernel modules was loaded if any (cannot find device driver etc.).

Why would loading a new driver really be of any interest to anyone? Userspace can't do anything with that. Userspace can only interact with /dev nodes.

But I can understand that if no driver is found for a device, that would be useful. Also remember that not all USB devices use kernel drivers (like all of the cameras that use libusb and gphoto2 to talk to devices, there is no kernel driver in use there.)

Quoted 7 lines

If there is not a well-defined scheme yet, I'll volunteer to write a spec and implement it as part of linux-hotplug. We can put this spec at either the HAL site or the linux-hotplug site. The former would IMHO be better since we want the scheme to be OS agnostic, but I'm flexible. I'm quite certain the scheme, to an extent, can be OS agnostic.
What is your take on this?

I think I need to see your spec first :)

Quoted 4 lines

Quoted 4 lines
Quoted 3 lines
In the event this is not possible the hotplug agent must somehow convey this information so I can append a counter to make the ID unique in that way..

Quoted 8 lines

Quoted 5 lines
The kernel will tell you that this is a new device. It's up to userspace to do something with that info. udev will determine the /dev name for the device. It's up to other programs to do other things with this info (which is what I'm thinking your proposal does, correct?)
Ok, the HAL daemon can take care of this as long as we agree on the DBUS message format. HAL is strictly user-space.

So is udev. And yes, I have no problem agreeing on the dbus message format.

thanks,

greg k-h

Havoc Pennington

2003-09-18 18:30 UTC #7 lore raw

In reply to David Zeuthen

Quoted 2 lines · Greg KH

But my main point is, why store this info anywhere, when it is already stored on your machine today?

Keep in mind the original proposal at http://ometer.com/hardware.html
which is that the HAL _merges_ information from various sources into a single list/tree of devices, and property set for each device.

So we may read some stuff from the USB hardware itself, then add some info provided by the kernel, then add some metadata from some systemwide files, then add some data that has been obtained by the desktop and stored per-user, then look at some blacklist, and finally we have a complete picture of everything known about that particular device.

But the exact set of things merged varies by device. The suggestion someone else posted about having "interfaces" is interesting in this respect.

One point of the HAL is to abstract where this information is coming from. Each info source is part of the "backend" not the application-visible API.

Quoted 3 lines

And it is automatically already handled
properly today, without any need for "yet another abstraction layer" to handle it?

Things aren't handled properly today in the sense that there's no UI on it, there's no integration with applications. To add that on the desktop/app level, we first have to have a single API that will work with different kinds of device and on different operating systems.

One principle to keep in mind is: application code should not know the details of specific devices. If I add a new device model, apps should automatically use it. Ideally, I can even add a new kind of bus and apps can use devices on that bus; e.g. an app doesn't care if a scanner is SCSI or USB.

From an OEM standpoint, if I submit a patch to one or a small well-defined set of projects, then my device should get used by all the interesting apps.

You may be lumping all userspace code together. Remember that the main desktop apps are built on dozens of libraries, all with opaque long-term-frozen ABIs that create additional abstraction layers.

Quoted 4 lines

Yes, I know this only works on Linux, and you want to be "cross-platform". Might I suggest you get the other OSs that you want to support to also support what Linux already does, instead of trying to duplicate work?

That doesn't help, unless the other OSs have _exactly_ the same filenames, interfaces, executables, and all other details. If there's _any_ difference between any interesting versions of Linux and any interesting versions of any other kernel, then we have to have the abstraction barrier. Apps can't contain #ifdef hell, it's not practical. App authors know _nothing_ about any specific hardware models or buses, and understand a single OS if that. They are on the level of "list the cameras, get the pictures from each"

Interesting kernels at the moment seem to be Solaris, Linux, and *BSD including Darwin primarily. I don't see coordinating exact interfaces between those as practical.

If it is practical, it definitely involves writing a spec for what the interfaces are, and a conformance suite, and committing to maintaining spec conformance over time. I haven't seen anything like this for the Linux stuff yet. ;-)

Quoted 3 lines

Oh, what oses are you wanting to support? I think FreeBSD already handles much of what Linux does, with it's devfs implementation, but I haven't looked at that in a long time to verify this.

Isn't devfs a device naming solution? HAL really has little or nothing to do with device naming; from an app standpoint I don't think we care at all what's in /dev other than being able to find out what to use in there, see the "implementation goals" point #2 in
http://ometer.com/hardware.html

Think of it this way: it's a bug if we have a field in the UI like:
Device Node: [ /dev/foo ]
that the user has to fill in. Sometimes we might have to do it, but it's a UI bug for the OS/desktop combo as a whole system. Instead the UI should be:
Cameras: [x] Minolta XYZ [ ] Canon
where those may be on different buses/kernel subsystems.

Quoted 7 lines

Now I agree that udev and
the hotplug scripts, and the raw hotplug events should get sent using dbus events, but I was thinking that userspace programs (like GNOME and KDE) that cared about those events would be listening for them. I didn't think there needed to be a whole intermediate layer for these messages to have to go through. But hey, I'm probably missing something big here :)

Possibly you're expecting apps to understand the specifics of each kernel subsystem (USB, etc.) and have significant Linux-specific code. That's not feasible from the GNOME/KDE/OpenOffice.org/Mozilla/etc. standpoint.
To the user we display "here are your cameras"; and at some level there has to be an abstraction layer that puts things on the level of "cameras" not "USB devices" - that layer needs to be below the application, so that all apps have the same list of cameras.

(Cameras are just an example of course.)

The motivation is partly portability, and partly simply shielding app developers from having to have a deep understanding of hardware in order to write the UI for it. App developers _won't_ have such an understanding, so if they have to, not much UI will happen.

Another angle on the motivation is the OEM one, allowing hardware to just drop in and work after I bought it at the store, at least when possible (e.g. the kernel driver is a generic one).

Maybe the most important motivation is getting the UI right, which requires us to know as much as we can about each device.

Quoted 3 lines

Why would loading a new driver really be of any interest to anyone? Userspace can't do anything with that. Userspace can only interact with /dev nodes.

If you have a device and you don't know which driver to use, the desktop could ask the user which driver to use. Also, if I'm an OEM and I know my device works with a standard Linux driver, but that some deployed kernels won't know that, I could ideally ship the information in a little file of some kind for the user to install.

Quoted 2 lines

But I can understand that if no driver is found for a device, that would be useful.

Right.

Quoted 3 lines

Also remember that not all USB devices use kernel drivers
(like all of the cameras that use libusb and gphoto2 to talk to devices, there is no kernel driver in use there.)

Right, this is another reason to have the HAL. Apps do not want to sort out all this which-device-uses-what stuff. They want to get a list of cameras, then pass a camera in to a camera API (e.g. gphoto) that magically gets the photos. And nobody wants to know what kind of plug is on the camera or who implemented the driver where. ;-)

Hopefully this mail helps clarify some of the goals. To use hardware effectively, apps need to have a hard dependency on an ABI for doing so. Hard deps must be portable and well-encapsulated/well-defined in terms of the semantics/ABI exported. Below this abstraction barrier, the various OS-specific stuff would be hooked in. So "HAL" essentially just means "the project that contains the abstraction barrier, and chains out to the per-OS per-device-model implementations" - my writeup has one speculation on what this project should look like, but I won't pretend to know for sure.

Havoc

David Zeuthen

author

2003-09-18 20:35 UTC #8 lore raw

In reply to David Zeuthen

Greg,

Havoc really touched all your points much better than me, so I'm only replying to a subset of them.

Quoted 5 lines · Greg KH

But my main point is, why store this info anywhere, when it is already stored on your machine today? And it is automatically already handled properly today, without any need for "yet another abstraction layer" to handle it?

The point of HAL is really to *complement* the information that is available in, say, Linux kernel 2.6 with sysfs.

Applications care about devices with specific capabilities, not on which bus they are on, and certainly not how to retrieve it from this or that bus. They use device-category specific support libraries to do that.

The contention is to have properties for each device that links to where bus-information information is. For example, in Linux Kernel 2.6 you would have a OS.Linux.devpath property that points somewhere into /sys for *every* device.

The advantage of this: When you ask an established device-access library to do something, you can give it a device-id and it does the job. It simply looks at the OS.Linux.devpath property from the given device and does it's magic. Nothing less, Nothing more and quite future proof.

Quoted 5 lines

Yes, I know this only works on Linux, and you want to be "cross-platform". Might I suggest you get the other OSs that you want to support to also support what Linux already does, instead of trying to duplicate work?

This is not trying to duplicate work at all. It's simply shielding applications from caring about bus information etc. by moving such information into libraries. No application should know what USB, IEEE1394 and so on is...

Also getting everyone to do and emulate what Linux does is not friendly and it might slow down new and better interfaces both in future versions of Linux and other places.

Quoted 8 lines

Oh, what oses are you wanting to support? I think FreeBSD already handles much of what Linux does, with it's devfs implementation, but I haven't looked at that in a long time to verify this.
Quoted 4 lines
Applications of the HAL (e.g. desktop environments) will then use standard kernel and OS interfaces to e.g. display the amount of free space on a Storage device.

Quoted 7 lines

Quoted 1 line
I hope this clear matters up?
Not really. I'm getting the impression that you don't realize the current state of Linux and how hotplugging and driver loading and device naming currently works. A lot of what you are saying you want to do, already works today.

Well, you are right that udev + sysfs is news to me...

All, I have tried to do is to formulate requirements regarding hardware from the point of view of desktop environments. I was inspired by Havoc's paper.

I have tried to be careful to specify everything and initiating discussion before just coding away. Which is why we are having this conversation, so I got at least one of my goals right ;-)

Oh, and I'm also new to open source development!

Quoted 9 lines

The only "major" piece that is missing, is what udev is working on solving (the persistant device naming issue.) Now I agree that udev and the hotplug scripts, and the raw hotplug events should get sent using dbus events, but I was thinking that userspace programs (like GNOME and KDE) that cared about those events would be listening for them. I didn't think there needed to be a whole intermediate layer for these messages to have to go through. But hey, I'm probably missing something big here :)

I'm not sure, but these are the key points that HAL tries to solve.

1. Make access to devices as simple as possible by shielding applications from bus-specific things by moving such specifics into libraries.

2. Having a single device info file concept that OEM's ship with their devices. This device info file should cover most popular distros and OS'es and versions thereof.
It should specify what the device is and what the user needs to do in order to interact with it. In order to match a device do this it need specifics of the hardware like usb.idVendor etc.

3. Provide some place where user-made settings are stored and persisted for a device. For instance UI-things like device name, where to mount a storage device and with what options (storage only readable by the user or not etc.).

<rambling>
In fact, things that pop on in the device-list managed by the HAL could be devices managed by other hosts (USB ports on your remote display, your fridge, a remote webserver etc.)..
</rambling>

Quoted 3 lines

So is udev. And yes, I have no problem agreeing on the dbus message format.

That sounds good.

Regards,
David

Joerg Sommer

2003-09-19 20:11 UTC #9 lore raw

In reply to David Zeuthen

Quoted 2 lines · begin Greg KH

Yes it will. See my OLS paper from this year for lots more information about it.

Nice to meet you here. :) I read this paper some time ago and one question stayed open: How does udev handle device requests? devfs has the nice feature to tell modprobe that a device is requested. But I don't found any similar in your paper. How gets udev informed that a program wants to open /dev/foobar?

Bye, Jörg.

end.

Signature · 11 lines

NetBSD ist für Frauen: es läuft auf Waschmaschinen


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Greg KH

2003-09-19 23:12 UTC #10 lore raw

In reply to David Zeuthen

Quoted 9 lines · Joerg Sommer

Quoted 2 lines · begin Greg KH
Yes it will. See my OLS paper from this year for lots more information about it.
Nice to meet you here. :) I read this paper some time ago and one question stayed open: How does udev handle device requests? devfs has the nice feature to tell modprobe that a device is requested. But I don't found any similar in your paper. How gets udev informed that a program wants to open /dev/foobar?

It doesn't :)

If you _really_ need that kind of functionality, then you will have to use devfs.

thanks,

greg k-h

Greg KH

2003-09-20 00:12 UTC #11 lore raw

In reply to David Zeuthen

Quoted 14 lines · Havoc Pennington

Quoted 2 lines · Greg KH
But my main point is, why store this info anywhere, when it is already stored on your machine today?
Keep in mind the original proposal at http://ometer.com/hardware.html
which is that the HAL _merges_ information from various sources into a single list/tree of devices, and property set for each device.
So we may read some stuff from the USB hardware itself, then add some info provided by the kernel, then add some metadata from some systemwide files, then add some data that has been obtained by the desktop and stored per-user, then look at some blacklist, and finally we have a complete picture of everything known about that particular device.

Quoted 3 lines

But the exact set of things merged varies by device. The suggestion someone else posted about having "interfaces" is interesting in this respect.

Heh, watch out, that word is just too overloaded these days :)

For USB devices, it makes sense to do something like an "interface" as USB devices can be compound devices with different drivers binding to different "usb interfaces". A USB speaker is the easiest example, the usb audio driver binds to the audio portion of the device, and the hid driver binds to the buttons part of the device.

Anyway, yes, you are right, if you want to merge from lots of different sources, that is a good thing.

Quoted 3 lines

One point of the HAL is to abstract where this information is coming from. Each info source is part of the "backend" not the application-visible API.

Ok, makes sense.

Quoted 8 lines

Quoted 3 lines
And it is automatically already handled
properly today, without any need for "yet another abstraction layer" to handle it?
Things aren't handled properly today in the sense that there's no UI on it, there's no integration with applications. To add that on the desktop/app level, we first have to have a single API that will work with different kinds of device and on different operating systems.

Hm, for a lot of this a UI isn't needed, right? The user doesn't need to know that a driver was just automatically loaded for their device, correct?

Now I agree, that if something goes wrong (no driver found), a UI is a very good thing to have. I just don't want to see you all duplicate work that has already been done :)

Quoted 5 lines

One principle to keep in mind is: application code should not know the details of specific devices. If I add a new device model, apps should automatically use it. Ideally, I can even add a new kind of bus and apps can use devices on that bus; e.g. an app doesn't care if a scanner is SCSI or USB.

I completely agree. That's one big reason why most USB devices use existing kernel interfaces (SCSI, V4L, serial ports, etc.)

Quoted 7 lines

From an OEM standpoint, if I submit a patch to one or a small well-defined set of projects, then my device should get used by all the interesting apps.
You may be lumping all userspace code together. Remember that the main desktop apps are built on dozens of libraries, all with opaque long-term-frozen ABIs that create additional abstraction layers.

I probably am, sorry. Partly because I know what's going on down low in the kernel, and don't pay attention to userspace much at all, "it's just a test load for the kernel" is how most kernel developers feel :)

But remember, most OEMs do not write kernel drivers. There are a few exceptions, thankfully. And that is what is required for almost all devices to get working with linux today.

Again, there are very notable exceptions, the digital cameras supported by gphoto2 and the USB scanners supported by xsane, both using userspace only code. I'm just pretty skeptical that something can be created that would allow OEMs to plop into a userspace location that would enable device support across multiple operating systems. But please, prove me wrong, I will be very happy.

Quoted 5 lines

Quoted 4 lines
Yes, I know this only works on Linux, and you want to be "cross-platform". Might I suggest you get the other OSs that you want to support to also support what Linux already does, instead of trying to duplicate work?

Quoted 8 lines

That doesn't help, unless the other OSs have _exactly_ the same filenames, interfaces, executables, and all other details. If there's _any_ difference between any interesting versions of Linux and any interesting versions of any other kernel, then we have to have the abstraction barrier. Apps can't contain #ifdef hell, it's not practical. App authors know _nothing_ about any specific hardware models or buses, and understand a single OS if that. They are on the level of "list the cameras, get the pictures from each"

I understand this. The kernel doesn't contain #ifdef hell for that very reason. But this is my main point as to how this might get very tough for having cross platform support. The BSDs support lots of USB devices today in a very different manner than Linux does.

Quoted 8 lines

Interesting kernels at the moment seem to be Solaris, Linux, and *BSD including Darwin primarily. I don't see coordinating exact interfaces between those as practical.
If it is practical, it definitely involves writing a spec for what the interfaces are, and a conformance suite, and committing to maintaining spec conformance over time. I haven't seen anything like this for the Linux stuff yet. ;-)

I haven't either, but I would be interested in seeing such a thing.

Quoted 5 lines

Quoted 3 lines
Oh, what oses are you wanting to support? I think FreeBSD already handles much of what Linux does, with it's devfs implementation, but I haven't looked at that in a long time to verify this.
Isn't devfs a device naming solution?

Yes it is.

Quoted 4 lines

HAL really has little or nothing to do with device naming; from an app standpoint I don't think we care at all what's in /dev other than being able to find out what to use in there, see the "implementation goals" point #2 in http://ometer.com/hardware.html

Ok, David mentioned device naming, and I pointed out udev. udev's goal is to solve your point #2 for Linux. Yes, it would be great to move udev configuration into your HAL so that it can be configured in a common manner. I'll help out with this in any way possible.

Quoted 7 lines

Think of it this way: it's a bug if we have a field in the UI like:
Device Node: [ /dev/foo ]
that the user has to fill in. Sometimes we might have to do it, but it's a UI bug for the OS/desktop combo as a whole system. Instead the UI should be:
Cameras: [x] Minolta XYZ [ ] Canon
where those may be on different buses/kernel subsystems.

That would be very nice. Hopefully udev can help provide some of the underlying glue that you need to determine this kind of information for Linux. It will contain a device database that you will be able to query to get this kind of information.

Quoted 8 lines

Quoted 7 lines
Now I agree that udev and
the hotplug scripts, and the raw hotplug events should get sent using dbus events, but I was thinking that userspace programs (like GNOME and KDE) that cared about those events would be listening for them. I didn't think there needed to be a whole intermediate layer for these messages to have to go through. But hey, I'm probably missing something big here :)

Quoted 4 lines

Possibly you're expecting apps to understand the specifics of each kernel subsystem (USB, etc.) and have significant Linux-specific code. That's not feasible from the GNOME/KDE/OpenOffice.org/Mozilla/etc. standpoint.

Hm, yeah, sorry. Again, I live too low in the OS stack :)

Quoted 6 lines

To the user we display "here are your cameras"; and at some level there has to be an abstraction layer that puts things on the level of "cameras" not "USB devices" - that layer needs to be below the application, so that all apps have the same list of cameras.
(Cameras are just an example of course.)

And one of the most difficult examples, as previously mentioned (some use gphoto2, some show up as scsi devices and must be mounted, some others are not even supported at all...)

Quoted 9 lines

Quoted 3 lines
Why would loading a new driver really be of any interest to anyone? Userspace can't do anything with that. Userspace can only interact with /dev nodes.
If you have a device and you don't know which driver to use, the desktop could ask the user which driver to use. Also, if I'm an OEM and I know my device works with a standard Linux driver, but that some deployed kernels won't know that, I could ideally ship the information in a little file of some kind for the user to install.

But if the driver was successfully loaded and is working, the user should not care at all, right?

Quoted 9 lines

Hopefully this mail helps clarify some of the goals. To use hardware effectively, apps need to have a hard dependency on an ABI for doing so. Hard deps must be portable and well-encapsulated/well-defined in terms of the semantics/ABI exported. Below this abstraction barrier, the various OS-specific stuff would be hooked in. So "HAL" essentially just means "the project that contains the abstraction barrier, and chains out to the per-OS per-device-model implementations" - my writeup has one speculation on what this project should look like, but I won't pretend to know for sure.

Yes, it does help a lot.

I'm happy to see this work happening, and if there's anything that you need from the kernel, hotplug scripts, or udev that I can help out with, please let me know.

thanks,

greg k-h

Greg KH

2003-09-20 00:17 UTC #12 lore raw

In reply to David Zeuthen

Quoted 5 lines · David Zeuthen

Greg,
Havoc really touched all your points much better than me, so I'm only replying to a subset of them.

Ok, I just responded back to his message.

Quoted 7 lines · Greg KH

Quoted 5 lines
But my main point is, why store this info anywhere, when it is already stored on your machine today? And it is automatically already handled properly today, without any need for "yet another abstraction layer" to handle it?

Quoted 6 lines

The point of HAL is really to *complement* the information that is available in, say, Linux kernel 2.6 with sysfs.
Applications care about devices with specific capabilities, not on which bus they are on, and certainly not how to retrieve it from this or that bus. They use device-category specific support libraries to do that.

Agreed.

Quoted 4 lines

The contention is to have properties for each device that links to where bus-information information is. For example, in Linux Kernel 2.6 you would have a OS.Linux.devpath property that points somewhere into /sys for *every* device.

Makes sense. udev will be able to provide this to you if you so desire.

Quoted 7 lines

Quoted 5 lines
Not really. I'm getting the impression that you don't realize the current state of Linux and how hotplugging and driver loading and device naming currently works. A lot of what you are saying you want to do, already works today.
Well, you are right that udev + sysfs is news to me...

Glad you found it. Hopefully it will make your life easier for your Linux implementation.

Quoted 9 lines

All, I have tried to do is to formulate requirements regarding hardware from the point of view of desktop environments. I was inspired by Havoc's paper.
I have tried to be careful to specify everything and initiating discussion before just coding away. Which is why we are having this conversation, so I got at least one of my goals right ;-)
Oh, and I'm also new to open source development!

Well, welcome. Looking forward to a working implementation :)

Quoted 10 lines

Quoted 9 lines
The only "major" piece that is missing, is what udev is working on solving (the persistant device naming issue.) Now I agree that udev and the hotplug scripts, and the raw hotplug events should get sent using dbus events, but I was thinking that userspace programs (like GNOME and KDE) that cared about those events would be listening for them. I didn't think there needed to be a whole intermediate layer for these messages to have to go through. But hey, I'm probably missing something big here :)

Quoted 5 lines

I'm not sure, but these are the key points that HAL tries to solve.
1. Make access to devices as simple as possible by shielding applications from bus-specific things by moving such specifics into libraries.

This seems doable.

Quoted 6 lines

2. Having a single device info file concept that OEM's ship with their devices. This device info file should cover most popular distros and OS'es and versions thereof.
It should specify what the device is and what the user needs to do in order to interact with it. In order to match a device do this it need specifics of the hardware like usb.idVendor etc.

This might be much more difficult due to the ways different OSs implement different device support. But hey, it's good to have dreams.

Quoted 4 lines

3. Provide some place where user-made settings are stored and persisted for a device. For instance UI-things like device name, where to mount a storage device and with what options (storage only readable by the user or not etc.).

Some of that will be taken care of in udev for Linux. But others (like where to mount the device) would be up to you.

thanks,

greg k-h

Joerg Sommer

2003-09-20 19:31 UTC #13 lore raw

In reply to David Zeuthen

Quoted 11 lines · begin Greg KH

Quoted 9 lines · Joerg Sommer
Quoted 2 lines · begin Greg KH
Yes it will. See my OLS paper from this year for lots more information about it.
Nice to meet you here. :) I read this paper some time ago and one question stayed open: How does udev handle device requests? devfs has the nice feature to tell modprobe that a device is requested. But I don't found any similar in your paper. How gets udev informed that a program wants to open /dev/foobar?

Quoted 4 lines

It doesn't :)
If you _really_ need that kind of functionality, then you will have to use devfs.

Why you removed this? IMO it is a very useful feature to only load a module if it is requested. Do you really think this is not needed?

Jörg.
end.

Signature · 13 lines

"Computer games don't affect kids. If Pacman would have affected us as
children, we would now run around in darkened rooms, munching pills and
listening to repetetive music."


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Greg KH

2003-09-21 06:42 UTC #14 lore raw

In reply to David Zeuthen

Quoted 12 lines · Joerg Sommer

Quoted 11 lines · begin Greg KH
Quoted 9 lines · Joerg Sommer
Quoted 2 lines · begin Greg KH
Yes it will. See my OLS paper from this year for lots more information about it.
Nice to meet you here. :) I read this paper some time ago and one question stayed open: How does udev handle device requests? devfs has the nice feature to tell modprobe that a device is requested. But I don't found any similar in your paper. How gets udev informed that a program wants to open /dev/foobar?

Quoted 7 lines

Quoted 4 lines
It doesn't :)
If you _really_ need that kind of functionality, then you will have to use devfs.
Why you removed this? IMO it is a very useful feature to only load a module if it is requested. Do you really think this is not needed?

I have not removed anything, this is just something that can not be done from userspace alone (which is what udev is.)

And yes, I really do not think this is needed, as things have moved from being a "load when we open the dev node" type system, to a "load when we have found the hardware" type system.

But hey, if you really need this functionality, then use devfs. I'm not saying that udev can duplicate all of devfs's functionality, just the majority of it (this is really the only exception that I can think of right now.)

thanks,

greg k-h

David Zeuthen

author

2003-09-21 20:56 UTC #15 lore raw

In reply to David Zeuthen

Quoted 2 lines · Greg KH

Well, welcome. Looking forward to a working implementation :)

Thanks! I hope not to be too far away from some proof of concept hot-pluggable usb storage. Only hotplug and device info stuff is missing.. All the other stuff seem stable enough now.. (stable, but not exactly production quality yet)

Quoted 10 lines

Quoted 6 lines
2. Having a single device info file concept that OEM's ship with their devices. This device info file should cover most popular distros and OS'es and versions thereof.
It should specify what the device is and what the user needs to do in order to interact with it. In order to match a device do this it need specifics of the hardware like usb.idVendor etc.
This might be much more difficult due to the ways different OSs implement different device support. But hey, it's good to have dreams.

It sure is :-)

In fact, I think it need not be too difficult - remember the HAL I'm working on is essentially just a database of information (and links to information) plus some well-defined properties that makes sense for many OS'es. Plus some rules on what happens when properties change due to external events such as hot-plugging or desktop application interaction.

For, e.g., a storage device to be usable, some existing software below my HAL would have to mount into the file system.. We even have to assume a file system that you can mount stuff into. I just want a device info file format that allow people to do this in different ways for different OS'es by specifying how and where to mount it in an OS-neutral way.

(assuming we also do device interfaces, similar nice stories for other categories of devices can be made)

Plus, having device info files, the OEM can tweak, he can put in properties about addresses for tech support, logo to display etc. This will make us appear a little bit more OEM friendly. (I know that Windows support this)

The big question that remains, is really who's life we are going to make easier or harder? the app developers, the users, the kernel people, the OEM's or the device library people? I don't think necessarily everyone can win.. And we need adoption from most, if not all, of these people..

So, yeah, it's going to be very difficult. (I'm done rambling now :-))

Quoted 8 lines

Quoted 4 lines
3. Provide some place where user-made settings are stored and persisted for a device. For instance UI-things like device name, where to mount a storage device and with what options (storage only readable by the user or not etc.).
Some of that will be taken care of in udev for Linux. But others (like where to mount the device) would be up to you.

Yes. I did install 2.6.0-test5 over the weekend and played around with udev 0.2, /sys and libsysfs + tools. It seems udev can't handle USB storage devices yet? It didn't worked for me at least.

Side question: Is it really true that the only way to obtain where the SCSI subsystem mounts the usb-storage is by inspecting /var/log/messages on 2.4? I couldn't find anything on 2.5/2.6 through googling either... It's very frustrating

Further, and this is because I known little or nothing of Linux kernel programming, /sys seems a little intimidating (but very cool - I looked at it in awe for over an hour).

So, what is your opinion on this: I plan, for some devices, to have well-defined properties (that makes sense on other OS'es also) that is dynamically retrieved from well-known sources such as /sys or /proc.

An obvious example is the CPU-frequency for a CPU device - ya know, stuff that desktop application programmers like to use but a) is quite difficult and a pain to obtain (parsing /proc/cpuinfo); and b) varies over a lot of OS'es. So in this example, when the laptop user disconnects power and his CPU slows down, the property CPU.ClockFrequency is automatically updated and the desktop application will be notified. Another example might be free space for storage devices.
(Just to stress it, the keyword here is linking rather than duplication, which I why I'd like your opinion.)

More on udev: how will it fit with the linux-hotplug project?

Maybe it's already there, but it could be nice to have a set of prioritized hotplug scripts that is invoked a'la /etc/init.d.. So one for loading a kernel module, udev for naming the device and one for sending a D-BUS message, notifying, among others, the HAL. In the chain of invoking these, the environment would grow.

While we're on the topic, I'll look into the D-BUS messages I'd like from hot-plugging. Will send a proposal one of these days. Since D-BUS is a moving target, we'll probably have to change it at some point though.

Thanks,
David

Greg KH

2003-09-24 21:08 UTC #16 lore raw

In reply to David Zeuthen

Quoted 5 lines · David Zeuthen

The big question that remains, is really who's life we are going to make easier or harder? the app developers, the users, the kernel people, the OEM's or the device library people? I don't think necessarily everyone can win.. And we need adoption from most, if not all, of these people..

The users are the most important ones here.

Quoted 5 lines

Quoted 5 lines
Quoted 4 lines
3. Provide some place where user-made settings are stored and persisted for a device. For instance UI-things like device name, where to mount a storage device and with what options (storage only readable by the user or not etc.).

Quoted 7 lines

Quoted 3 lines
Some of that will be taken care of in udev for Linux. But others (like where to mount the device) would be up to you.
Yes. I did install 2.6.0-test5 over the weekend and played around with udev 0.2, /sys and libsysfs + tools. It seems udev can't handle USB storage devices yet? It didn't worked for me at least.

Should work just fine. They show up as block devices, just like any other disk (look in /sys/block). My demo at OLS 2003 used my usb camera which is a usb storage device.

Quoted 4 lines

Side question: Is it really true that the only way to obtain where the SCSI subsystem mounts the usb-storage is by inspecting /var/log/messages on 2.4? I couldn't find anything on 2.5/2.6 through googling either... It's very frustrating

Don't remember, sorry. There are a bunch of different scsi programs for doing this. Search the linux-scsi archives for links to them.

Quoted 3 lines

Further, and this is because I known little or nothing of Linux kernel programming, /sys seems a little intimidating (but very cool - I looked at it in awe for over an hour).

Heh, thanks. It's pretty impressive to finally see all of those links that were always internal to the kernel.

Quoted 4 lines

So, what is your opinion on this: I plan, for some devices, to have well-defined properties (that makes sense on other OS'es also) that is dynamically retrieved from well-known sources such as /sys or /proc.

Quoted 10 lines

An obvious example is the CPU-frequency for a CPU device - ya know, stuff that desktop application programmers like to use but a) is quite difficult and a pain to obtain (parsing /proc/cpuinfo); and b) varies over a lot of OS'es. So in this example, when the laptop user disconnects power and his CPU slows down, the property CPU.ClockFrequency is automatically updated and the desktop application will be notified. Another example might be free space for storage devices.
(Just to stress it, the keyword here is linking rather than duplication, which I why I'd like your opinion.)

I think there's already a user library to get the cpu frequency stuff out of /proc or /sys to userspace. You might want to look into that.

But yes, unifying this for user apps would be a good thing.

Quoted 1 line

More on udev: how will it fit with the linux-hotplug project?

It uses the hotplug-base package, that's all.

Quoted 5 lines

Maybe it's already there, but it could be nice to have a set of prioritized hotplug scripts that is invoked a'la /etc/init.d.. So one for loading a kernel module, udev for naming the device and one for sending a D-BUS message, notifying, among others, the HAL. In the chain of invoking these, the environment would grow.

Heh, we already have that. See the latest release of the linux-hotplug package :)

thanks,

greg k-h

David Zeuthen

author

2003-09-29 20:50 UTC #17 lore raw

In reply to David Zeuthen

Quoted 9 lines · Greg KH

Quoted 9 lines
Quoted 8 lines
Quoted 4 lines
3. Provide some place where user-made settings are stored and persisted for a device. For instance UI-things like device name, where to mount a storage device and with what options (storage only readable by the user or not etc.).
Some of that will be taken care of in udev for Linux. But others (like where to mount the device) would be up to you.

Quoted 8 lines

Quoted 3 lines
Yes. I did install 2.6.0-test5 over the weekend and played around with udev 0.2, /sys and libsysfs + tools. It seems udev can't handle USB storage devices yet? It didn't worked for me at least.
Should work just fine. They show up as block devices, just like any other disk (look in /sys/block). My demo at OLS 2003 used my usb camera which is a usb storage device.

My bad. I hadn't loaded the kernel module sd_mod. Works like a charm with this module..

Quoted 8 lines

Quoted 4 lines
Side question: Is it really true that the only way to obtain where the SCSI subsystem mounts the usb-storage is by inspecting /var/log/messages on 2.4? I couldn't find anything on 2.5/2.6 through googling either... It's very frustrating
Don't remember, sorry. There are a bunch of different scsi programs for doing this. Search the linux-scsi archives for links to them.

Hmm yes.. I searched for a long time, but found this message

http://www.dragoninc.on.ca/mail-archives/linux-usb/2002-06/0122.html

where you say this is not possible in 2.4.

Quoted 4 lines

Quoted 1 line
More on udev: how will it fit with the linux-hotplug project?
It uses the hotplug-base package, that's all.

Ah, I see.. You symlink udev to /etc/hotplug.d/default/udev.hotplug and on hotplug ACTIONd, $1=block invocations query the sys/$DEVPATH/dev file and do a mknod.

I can see that it's a clever thing to have a (pluggable) naming database and strategy, especially for those sites you mention with thousands of disks.

Do you intend udev to send a hotplug message to notify that a new device file have been created? Or is it only for kernel messages? The thinking is that one could make a generic script that broadcast each and every hotplug event. It could be as simple as

  dbus-send --dest=org.freedesktop.DBus.Broadcast org.kernel.Hotplug
  string:$1 string:"`env`"

to broadcast the hotplug event on the dbus system bus. Of course, in a proper script one would test for /usr/bin/dbus-send and /var/run/messagebus.pid.

This way, I wouldn't need my hal_hotplug program, but just listen for org.kernel.Hotplug messages on the dbus system bus and still enjoy the functionality of udev.

Quoted 9 lines

Quoted 5 lines
Maybe it's already there, but it could be nice to have a set of prioritized hotplug scripts that is invoked a'la /etc/init.d.. So one for loading a kernel module, udev for naming the device and one for sending a D-BUS message, notifying, among others, the HAL. In the chain of invoking these, the environment would grow.
Heh, we already have that. See the latest release of the linux-hotplug package :)

OK. One thing though.. When I plug in my USB storage device I get a number hotplug invocations on 2.6 (instead of only one on 2.4):

o usb, DEVPATH=/devices/pci0000:00/0000:00:10.0/0000:02:07.0/usb2/

                 2-1/2-1.3

o usb, DEVPATH=/devices/pci0000:00/0000:00:10.0/0000:02:07.0/

                 usb2/2-1/2-1.3/2-1.3:0

o scsi_host, DEVPATH=/class/scsi_host/host2

o scsi, DEVPATH=/devices/pci0000:00/0000:00:10.0/0000:02:07.0/

                  usb2/2-1/2-1.3/2-1.3:0/host2/2:0:0:0

o block, DEVPATH=/block/sda

o block, DEVPATH=/block/sda/sda1

o scsi_device, DEVPATH=/class/scsi_device/2:0:0:0

These are all handled by stuff loading modules and creating device files (udev) which is all good. But when do I know when all messages for a device is sent?

My understanding is that is doesn't make send to talk about hotplug start and hotplug end messages (though it would be nice), so I guess when receiving the second usb hotplug message I wait for a sec or two to received the first block message (since $DEVPATH/device points to the USB device). Or even better, a message from udev! (otherwise I would have to create my own device file with mknod?)

Thanks,
David

Havoc Pennington

2003-10-01 01:30 UTC #18 lore raw

In reply to David Zeuthen

Hi,

I just learned from a comment on gnomedesktop.org that I missed most of this thread, it landed in the wrong mail folder or something. ;-)

Quoted 8 lines · Greg KH

Quoted 4 lines
Things aren't handled properly today in the sense that there's no UI on it, there's no integration with applications. To add that on the desktop/app level, we first have to have a single API that will work with different kinds of device and on different operating systems.
Hm, for a lot of this a UI isn't needed, right? The user doesn't need to know that a driver was just automatically loaded for their device, correct?

That's right. The bulk of the UI should be about using devices rather than getting them connected and working.

Quoted 6 lines

Again, there are very notable exceptions, the digital cameras supported by gphoto2 and the USB scanners supported by xsane, both using userspace only code. I'm just pretty skeptical that something can be created that would allow OEMs to plop into a userspace location that would enable device support across multiple operating systems. But please, prove me wrong, I will be very happy.

Ah, I don't really mean that the OEM interface would be OS-independent; either the file plopped would have to contain OS-specific portions, or there would be a per-OS file. That's OK since we can expect an OEM to deal with each OS independently, as they only have 1 device * N OSes. It's the app developers that have to be insulated from M devices * N OSes.

The OS independent portion is the part the app sees. Some properties exported by the HAL are probably even OS-specific, but the point is there's an OS-independent core/subset. (i.e. the idea is to let you write portable code, not to keep you from punching down to OS details if you want)

Quoted 4 lines

I understand this. The kernel doesn't contain #ifdef hell for that very reason. But this is my main point as to how this might get very tough for having cross platform support. The BSDs support lots of USB devices today in a very different manner than Linux does.

I think of it as "portability by subdirectory" ;-)

You have an API that apps talk to, and then some generic code, and then a per-OS backend with a subdirectory for each backend:

hal/
include/
hal/hal.h
generic/
hal.c
linux/
usb.c
freebsd/
usb.c

It isn't so much that the freebsd/ subdir is populated immediately, but rather that when someone appears with the skills and motivation they can add that subdir without having to rearchitect everything.

The desktop userspace doesn't have a list of required platforms so much as the requirement that "if someone is actively maintaining a platform they can keep that platform working"

Quoted 3 lines

I'm happy to see this work happening, and if there's anything that you need from the kernel, hotplug scripts, or udev that I can help out with, please let me know.

I really appreciate your interest/help, btw.

I'm curious how many iterations of all this we'll go through before we're happy with it. Wait, I've worked on software long enough to know the answer is "infinite"... ;-)

Havoc