Automatic download and installation of drivers.
Automatic download and installation of drivers. That is definately a good thing to have (in general). What we need now is to find:
- the best way to to use HTTP for the task.
- the best way to organize scripts/programs to have a general framework for requesting drivers of all kinds.
- actual database servers which are globally accessible to do the work.
- actual manufacturers to simply provide downloads in their ftp servers.
For HTTP there can be simple name=value requests that are passed to the server's CGI. The server redirects the client to where the actual file resides. The following holds:
wget http://www.kernel.org/cgi-bin/getdriver?bus=usb&pid=0x1234&vid=0x5678 <http://www.kernel.org/cgi-bin/getdriver?bus=usb&pid=0x1234&vid=0x5678>
Afterwards, the driver will get installed.
For the hotplug scripts I suggest that we create a new driver.agent which will process four actions (fetch/toss/install/uninstall) with appropriate parameters. It will be triggered by any other hotplug agent requiring a certain "resource" to satisfy an important request. The key thing here is that it should be acceptable to use the standard input/output of an agent.script when its called (like an ordinary program -- diagnostics should be sent to either stderr or syslog of course). This will be useful here since it's not very convenient to pass name=value pairs in the form of environment variables. Defining std input/output as "may be useful" for the hotplug scripts should be a good thing. install/uninstall get the DRIVER_RESOURCE parameter which is the local filename of the downloaded file. If a .tar.gz, they can guess the directory it unpacks to.
If anyone has any better ideas/suggestions, please let me know. I'm already working on the client-side for this (done install/uninstall for "targz.agent" driver subagent). It should be pretty easy to make the server-side DB and CGI linking to it.
Anyone to help?
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 2 lines · Stamatis Mitrofanis
Automatic download and installation of drivers. That is definately a good thing to have (in general).
Wow, the security implementations and other complexities involved here are huge. Let try a few examples:
- installing a module requires root permissions. You generally do not want to run a compiler as root.
- Running anything as root isn't a good idea, unless the program has been audited for security problems.
- Who is going to sign/verify the driver you just downloaded to prevent a trojan from being installed?
- Lots of user machines do not have compilers installed. What then?
- Where are you going to find the .config file that the currently running kernel was built against? Without that, you will not be able to successfully build that module.
- What is the guarantee that older kernels will be able to load newer modules? Inter-kernel apis constantly change. The odds that this would work are quite slim.
- Distros test their kernels as one package, drivers included. Then they ship with those drivers. Any new drivers fall under the above problem.
And there are more problems that I can't think of right now.
But the main question I have is:
What is the real problem that you are trying to solve?
and
Why does the current kernel/driver situation not work for you?
thanks,
greg k-h
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 6 lines
What we need now is to find:
- the best way to to use HTTP for the task.
- the best way to organizescripts/programsto have a general framework for requesting drivers of all kinds.
- actual database servers which are globally accessible to do the work.
- actual manufacturers to simply provide downloads in their ftp servers.
Actually I think you've got the order backwards: find systems that need this kind of software distribution model first; then let their needs drive development of such an alternative distribution scheme (and assurance of its trustworthiness/security). I can imagine local sites hosting some servers for their local users, for example.
While I've sometimes been intrigued by such schemes, I'll have to be contrarian in this case and say it's more interesting than useful at the moment. (Even for environments like Java, which was designed for that model of software distribution.) The current Linux model is that driver distribution is tightly coupled to the associated kernel. That is known to work well. What's the motivation for another model?
- Dave
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 14 lines · you
Quoted 2 lines
Automatic download and installation of drivers. That is definately a good thing to have (in general).
Wow, the security implementations and other complexities involved here are huge. Let try a few examples:
- installing a module requires root permissions. You generally do not want to run a compiler as root.
- Running anything as root isn't a good idea, unless the program has been audited for security problems.
- Who is going to sign/verify the driver you just downloaded to prevent a trojan from being installed?
- Lots of user machines do not have compilers installed. What then?
- Where are you going to find the .config file that the currently running kernel was built against? Without that, you
IMHO there are two choices for auto-installation. The easier one is to use distribution specific packages, then most of the problems do not apply. Only if you want to distribute drivers in source form you have to take care that:
- the sources of the installed kernel are available at a well-known place
- you have a well-defined compilation environment. You have to make rules which gcc version, which make version and so on has to be available and at which location (or alternatively you need a configuration file that describes their location). You need to be propared that the user has two compilers (gcc&kgcc).
- you need to know how to install and maybe also de-install modules in the system (it should provide a script for that)
- you need the configuration file of the installed kernel
- possibly you need the ability to re-compile the kernel itself if a needed configuration option is missing (for example enable V4L if the user installs a webcam). This can be quite complex which would be a reason to do this only with CML2.
- ideally there would be a way for a installed driver package to add its own configuration options to the kernel. For example if you would install each drivers source in a directory /usr/src/linux/drivers/<driver-name>, then CML2 could automagically include all /usr/src/linux/drivers/*/Config.in files.
This, of course, is a LOT of work.
Quoted 4 lines
But the main question I have is:
What is the real problem that you are trying to solve?
and
Why does the current kernel/driver situation not work for you?
1. It makes it much easier for a user to find a driver that's not included in his distribution. Linux drivers are usually named after the chipset's name or the first device that was supported. How can a user know that his Logitech Quickcam Pro 3000 needs the pwc (philips webcam) driver? And where can he find the driver?
2. (source driver packages only) right now the situation for drivers authors who want to distribute their drivers to end-users is really bad. Either they make it complicated for their users and distribute the drivers in source form. Even now I would guess the majority of Linux users is not able to compile their own drivers, and hopefully the percentage will be much lower in a couple of years. Or the driver author provides binary packages for each version of each distribution. It is a lot of work and still won't help all users. This is probably one of the reasons that so few hardware manufacturers provide Linux drivers.
bye...
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 2 lines · Tim Jansen
2. (source driver packages only) right now the situation for drivers authors who want to distribute their drivers to end-users is really bad.
<blatant plug>
Kernel build (kbuild) 2.5 is designed to make it easy to compile add on code. The user still needs the kernel source and the .config used to build their kernel, plus their existing kernel objects. The add on source can be stored anywhere and will be compiled as if it were part of the kernel source tree.
kbuild 2.5 gives full support for add on code, including make *config, Makefile, kernel/module build, modules install, dependencies, the lot. See http://sourceforge.net/projects/kbuild for the patch, read Documentation/kbuild/kbuild-2.5.txt, in particular Shadow Trees.
</blatant plug>
Keith Owens, kbuild maintainer.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 10 lines · Tim Jansen
Quoted 4 lines
But the main question I have is:
What is the real problem that you are trying to solve?
and
Why does the current kernel/driver situation not work for you?1. It makes it much easier for a user to find a driver that's not included in his distribution. Linux drivers are usually named after the chipset's name or the first device that was supported. How can a user know that his Logitech Quickcam Pro 3000 needs the pwc (philips webcam) driver? And where can he find the driver?
The current linux-hotplug interface solves this problem. The driver itself lets the kernel/world know for what devices it supports. Coming from the other direction (what driver supports this device) is just as easy. Look at modules.usbmap and modules.pcimap for examples.
Quoted 9 lines
2. (source driver packages only) right now the situation for drivers authors who want to distribute their drivers to end-users is really bad. Either they make it complicated for their users and distribute the drivers in source form. Even now I would guess the majority of Linux users is not able to compile their own drivers, and hopefully the percentage will be much lower in a couple of years. Or the driver author provides binary packages for each version of each distribution. It is a lot of work and still won't help all users. This is probably one of the reasons that so few hardware manufacturers provide Linux drivers.
The solution to this is to either live with the nightmare of supporting binary packages for _every_ distro kernel release (1 kernel release from RedHat is 11 kernels, multiply that by different versions of that distro and the number of different distros, you have a huge mess) or simply put the driver in the kernel source tree.
By putting the driver into the tree, all the problems go away, users don't have to compile their own drivers, the drivers are all distributed by the different Linux vendors, vendors don't have to update their driver for every kernel api change, and everyone is happy. Well, the people who don't like open source drivers aren't happy, but we aren't here to make their lives easier :)
thanks,
greg k-h
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 4 lines · Greg KH
The current linux-hotplug interface solves this problem. The driver itself lets the kernel/world know for what devices it supports. Coming from the other direction (what driver supports this device) is just as easy. Look at modules.usbmap and modules.pcimap for examples.
But how does this help for drivers that are not installed yet because they are not part of the kernel? This is the reason for downloading drivers.
Quoted 4 lines
By putting the driver into the tree, all the problems go away, users don't have to compile their own drivers, the drivers are all distributed by the different Linux vendors, vendors don't have to update their driver for every kernel api change, and everyone is happy.
I doubt that this will make hardware vendors happy:
- their devices cannot be supported before they are included in a kernel. Assuming a world where new devices are released with vendor-supported linux drivers this would mean that a new device cannot be released before their drivers are in the kernel and distributions picked this kernel up.
- vendors lose control of their device drivers even though their quality is crucial for the success of their products
This is also a problem for users. Most distributions do update between 1 and 4 times updates per year. This means that users have to wait for their driver up to one year before they can buy new devices or get an update for their driver.
bye...
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 3 lines · you
that model of software distribution.) The current Linux model is that driver distribution is tightly coupled to the associated kernel. That is known to work well. What's the motivation for another model?
One motivation is to lower the barrier for hardware vendors who want to provide drivers. So far it is very difficult for them. Either they provide a dozen of RPM packages and still can not reach all Linux users (like nvidia) or they don't even try and support one or two distributions (AVM supports only SuSE on their ISDN adapter). The linux market is small enough, and by making it difficult to provide a single driver to all users you make is even less worthwhile.
The advantage for users is that they can get drivers for new devices or updates without waiting for a new release of their distribution. It will also help users of, for example, debian and slackware who wouldn't get pre-packaged drivers otherwise, at least not before the driver is in the kernel. And those who would like to update their distribution, but the driver vendor hasn't released a newer package yet (SuSe 7.3 is out, but AVM only supports 7.1 and 7.2).
bye...
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 8 lines · Tim Jansen
Quoted 4 lines · Greg KH
The current linux-hotplug interface solves this problem. The driver itself lets the kernel/world know for what devices it supports. Coming from the other direction (what driver supports this device) is just as easy. Look at modules.usbmap and modules.pcimap for examples.
But how does this help for drivers that are not installed yet because they are not part of the kernel? This is the reason for downloading drivers.
If the running (old) kernel does not know about the devices, but a future kernel does, then you have all the same problems that I listed before (kernel api, compilers, .config, etc.) Download the whole kernel, it's the only way, as there is not going to be a common binary or source api for Linux kernel drivers, like other operating systems have.
Quoted 10 lines
Quoted 4 lines
By putting the driver into the tree, all the problems go away, users don't have to compile their own drivers, the drivers are all distributed by the different Linux vendors, vendors don't have to update their driver for every kernel api change, and everyone is happy.
I doubt that this will make hardware vendors happy:
- their devices cannot be supported before they are included in a kernel. Assuming a world where new devices are released with vendor-supported linux drivers this would mean that a new device cannot be released before their drivers are in the kernel and distributions picked this kernel up.
Vendors who work with open source drivers, typically have their Linux drivers updated _before_ the hardware hits the shelves. See the keyspan and edgeport USB drivers for examples of this.
Quoted 2 lines
- vendors lose control of their device drivers even though their quality is crucial for the success of their products
Then the vendors have the maintainer of the driver be an employee. This is quite common (myself included.) Look at most all of the SCSI drivers for other examples.
Quoted 4 lines
This is also a problem for users. Most distributions do update between 1 and 4 times updates per year. This means that users have to wait for their driver up to one year before they can buy new devices or get an update for their driver.
Then if a vendor wants a specific driver built for a specific distro's kernel, due to the distro not having support for it, they build it. I've done this in the past for some companies, it's not that hard, just a pain getting all of the .config settings correct (which is what any "automated" system would have to do also.)
thanks,
greg k-h
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 5 lines
One motivation is to lower the barrier for hardware vendors who want to provide drivers. So far it is very difficult for them. Either they provide a dozen of RPM packages and still can not reach all Linux users (like nvidia) or they don't even try and support one or two distributions (AVM supports only SuSE on their ISDN adapter).
Easy fix: provide a patch to the kernel source, rather than going with a binary distribution model. The model of binary driver distributions is a problem. It's my understanding that most hardware vendors would rather not be in that game, but MSFT forced them to play that way.
Quoted 2 lines
The advantage for users is that they can get drivers for new devices or updates without waiting for a new release of their distribution.
With a patch, "knowledgeable" users can get going right away. All others are going to need to wait for a new release in any case, or will ask their "knowledgeable user" friends for help.
- Dave
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Yep, a better way to manage source code is the Linux Way! :)
Of course, it's still tricky to answer the "is there a Linux driver for this hardware". More vendors should have those answers available on their websites.
- Dave
----- Original Message -----
From: "Keith Owens" <[email protected]>
To: <[email protected]>
Sent: Tuesday, October 16, 2001 4:38 AM
Subject: Re: Automatic download and installation of drivers.
Quoted 20 lines
Quoted 2 lines · Tim Jansen
2. (source driver packages only) right now the situation for drivers authors who want to distribute their drivers to end-users is really bad.
<blatant plug>
Kernel build (kbuild) 2.5 is designed to make it easy to compile add on code. The user still needs the kernel source and the .config used to build their kernel, plus their existing kernel objects. The add on source can be stored anywhere and will be compiled as if it were part of the kernel source tree.
kbuild 2.5 gives full support for add on code, including make *config,
Makefile, kernel/module build, modules install, dependencies, the lot. See http://sourceforge.net/projects/kbuild for the patch, readDocumentation/kbuild/kbuild-2.5.txt, in particular Shadow Trees.</blatant plug>
Quoted 7 lines
Keith Owens, kbuild maintainer.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 5 lines · David Brownell
Easy fix: provide a patch to the kernel source, rather than going with a binary distribution model. The model of binary driver distributions is a problem.
With a patch, "knowledgeable" users can get going right away. All others are going to need to wait for a new release in any case
But this means, in other words, that Linux is not competitive in this area. Then regular users have to wait up to 6 months before they can use new devices because they have to wait for the next Red Hat distribution while Mac and Windows users can already use the device. It would also force them to update a stable system only for getting a new device driver.
And it would result in hardware manufactures being unable to sell their new devices to the majority of Linux users before most distributions have been updated.
bye...
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
The _full_ quote of that last paragraph contains the disproof to your claim about timeliness and competitiveness ...
Quoted 3 lines
With a patch, "knowledgeable" users can get going right away. All others are going to need to wait for a new release in any case, or will ask their "knowledgeable user" friends for help.
It's well known that even in the Windows world, most support comes from friendly local experts ("power users"). Those are the folk that get called in when a vendor's driver disk doesn't install correctly (the ones I get usually don't).
So far as I know, there isn't even an authoritative list of all the USB and CardBus devices supported by, say, the 2.4.12 kernel. Or even a good way to extract the subset of that list that's packaged as the MODULE_DEVICE_TABLE entries, short of building a kernel with every driver configured as a module (a bit painful IMO).
- Dave
----- Original Message -----
From: "Tim Jansen" <[email protected]>
To: "David Brownell" <[email protected]>; <[email protected]>
Sent: Wednesday, October 17, 2001 11:47 AM
Subject: Re: Automatic download and installation of drivers.
Quoted 16 lines · David Brownell
Quoted 5 lines
Easy fix: provide a patch to the kernel source, rather than going with a binary distribution model. The model of binary driver distributions is a problem.
With a patch, "knowledgeable" users can get going right away. All others are going to need to wait for a new release in any caseBut this means, in other words, that Linux is not competitive in this area. Then regular users have to wait up to 6 months before they can use new devices because they have to wait for the next Red Hat distribution while Mac and Windows users can already use the device. It would also force them to update a stable system only for getting a new device driver.
And it would result in hardware manufactures being unable to sell their new devices to the majority of Linux users before most distributions have been updated.
Quoted 4 lines
bye...
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 7 lines · David Brownell
Quoted 3 lines
With a patch, "knowledgeable" users can get going right away. All others are going to need to wait for a new release in any case, or will ask their "knowledgeable user" friends for help.
It's well known that even in the Windows world, most support comes from friendly local experts ("power users"). Those are the folk that get called in when a vendor's driver disk doesn't install correctly (the ones I get usually don't).
But this is one of the problems with Linux and newbies - even with today's distributions there are far too many things you need "knowledgable" users for. This is definitely not the way it should be, and Windows shows that it is not neccessary. While it is true that it the driver installation fails sometimes you can be sure that this will improve over the next years. Actually the frequent failures of driver installations are probably the main reasons that Microsoft started their driver certification program.
Quoted 3 lines
So far as I know, there isn't even an authoritative list of all the USB and CardBus devices supported by, say, the 2.4.12 kernel.
You wouldn't need this in the kernel for automatic downloads. Actually the idea of automatic downloadable drivers is to provide drivers that are NOT in the kernel. If the user connects a new device and the system doesn't find a driver for it it would request one from some server. This server could either provide a driver itself, based the device's vendor and model id, or forward the system to the vendor's server which would then try to find one for the model.
bye...
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 8 lines · Tim Jansen
You wouldn't need this in the kernel for automatic downloads. Actually the idea of automatic downloadable drivers is to provide drivers that are NOT in the kernel. If the user connects a new device and the system doesn't find a driver for it it would request one from some server. This server could either provide a driver itself, based the device's vendor and model id, or forward the system to the vendor's server which would then try to find one for the model.
But how do you resolve all of the previous points I raised about compilers, .config files, in kernel apis, and security with regards to "automatic downloads"?
thanks,
greg k-h
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoting Greg KH <[email protected]>:
Quoted 10 lines · Tim Jansen
Quoted 8 lines
You wouldn't need this in the kernel for automatic downloads. Actually the idea of automatic downloadable drivers is to provide drivers that are NOT in the kernel. If the user connects a new device and the system doesn't find a driver for it it would request one from some server. This server could either provide a driver itself, based the device's vendor and model id, or forward the system to the vendor's server which would then try to find one for the model.
Quoted 2 lines
But how do you resolve all of the previous points I raised about compilers,
Just yesterday I had to replace gcc 3.0.1 with old but working egcs-2.91.66. The C download was just 1.5 MBytes, and C++ was only 1.5 MBytes more - though you wouldn't need C++ for the kernel.
This means that gcc is just another dependency that the script will resolve before even trying to fetch and compile the driver. The size of the required download is acceptable even for modem users, and that has to be done only once.
Of course, distributions can make the gcc (suitable for kernel compilations) a required (base) package.
Quoted 1 line
.config files,
I think they are available in "kernel-devel" package or something like that. They easily can be shipped as part of the kernel RPM. The kernel-devel package is already available in RH, for example.
Of course, the kernel itself can contain a copy of its own .config file and export it through /proc fs, along with all compiler options, that would be neat :-) and only requires an optional module.
Quoted 1 line
in kernel apis,
That's not something new. APIs change all the time, everywhere, not only in OSes. Programmers just need to use #if ... #endif and maybe the driver should be marked as "suitable" or "tested" on some range of kernel releases.
Quoted 1 line
and security with regards to "automatic downloads"?
Linus already signs his kernel releases. There is no reason why driver maintainers can't join the web of trust. Then GPG will cheerfully accept signatures of partially trusted signers. It is very easy to do, publishers only need to get their public key signed by many other publishers, and the threshold for the sufficient trust can be hardcoded to be fairly high.
/***/
It is clear to me that Linux needs a mechanism for fetching and installing 3rd party drivers. The proposed technology seems to be quite simple, and requires very little development because all components are already here. I think, a single script can definitely accomplish searching, downloading, compiling (kbuild?) and installing the driver. This project is all about organizing software and people to do it right.
Thanks,
Dmitri
Signature · 2 lines
panic("esp: penguin status phase.");
(Panic message in the kernel.)Quoted 19 lines · Dmitri
Quoting Greg KH :
Quoted 10 lines · Tim Jansen
Quoted 9 lines
You wouldn't need this in the kernel for automatic downloads. Actually the idea of automatic downloadable drivers is to provide drivers that are NOT in the kernel. If the user connects a new device and the system doesn't find a driver for it it would request one from some server. This server could either provide a driver itself, based the device's vendor and model id, or forward the system to the vendor's server which would then try to find one for the model.
Quoted 3 lines
But how do you resolve all of the previous points I raised about compilers,
Just yesterday I had to replace gcc 3.0.1 with old but working egcs-2.91.66. The C download was just 1.5 MBytes, and C++ was only 1.5 MBytes more - though you wouldn't need C++ for the kernel.
Quoted 8 lines
This means that gcc is just another dependency that the script will resolve before even trying to fetch and compile the driver. The size of the required download is acceptable even for modem users, and that has to be done only once.
Of course, distributions can make the gcc (suitable for kernel compilations) a required (base) package.
I agree that a C compiler is a nice thing to have, but no one must be forced to have it (say, end users). If it becomes a "base package", every computer must have it installed and then people will start using it when it's not necessary (huge mess). The best thing to do here is to install the right version of gcc on demand (just as we do for drivers).
Quoted 10 lines
Quoted 2 lines
.config files,
I think they are available in "kernel-devel" package or something like that. They easily can be shipped as part of the kernel RPM. The kernel-devel package is already available in RH, for example.
Of course, the kernel itself can contain a copy of its own .config file and export it through /proc fs, along with all compiler options, that would be neat :-) and only requires an optional module.
Standardize _both_ as the "current kernel configuration" (/usr/src/linux/.config _and_ a proc fs entry) so that driver sources can freely use it. I'd go for the first one though, here's the rationale:
Since we are going to allow third-party kernel drivers to be compiled for a kernel, we may as well grant them the right to do everything that a "normally" (in-kernel) distributed driver can do. A driver may depend on other drivers to compile/work and they will need to be downloaded, configured and installed as well. A driver must be allowed to set (its own, of course) entries in the kernel configuration file , modifying it for other third-party drivers to read. Thus it's better to use /usr/src/linux/.config .
Quoted 7 lines
Quoted 2 lines
in kernel apis,
That's not something new. APIs change all the time, everywhere, not only in OSes. Programmers just need to use #if ... #endif and maybe the driver should be marked as "suitable" or "tested" on some range of kernel releases.
Couldn't the kernel's configuration file be useful in solving this problem? API authors could use it to describe the changes they made in the API.
Quoted 9 lines
Quoted 2 lines
and security with regards to "automatic downloads"?
Linus already signs his kernel releases. There is no reason why driver maintainers can't join the web of trust. Then GPG will cheerfully accept signatures of partially trusted signers. It is very easy to do, publishers only need to get their public key signed by many other publishers, and the threshold for the sufficient trust can be hardcoded to be fairly high.
I wish I could find something wrong with that... Then I would have something to say here. :-)
Quoted 10 lines
/***/
It is clear to me that Linux needs a mechanism for fetching and installing 3rd party drivers. The proposed technology seems to be quite simple, and requires very little development because all components are already here. I think, a single script can definitely accomplish searching, downloading, compiling (kbuild?) and installing the driver. This project is all about organizing software and people to do it right.
Yep. Our slogan should be: "Do it when you have to!". :-)
This script can very well be a hotplug agent supporting four ACTIONs. I posted a message about this a few days ago, but since I didn't put any Subject line no one must have read it. Here it is again:
/* Quote myself: */
Well, a "configure" script would be good to use for source code. It would return success/failure depending on whether the local architecture/configuration is known to work for compiling the driver. If it's not, then manual compilation and installation is required. Drivers should be tested on as many kernel versions as possible and the "configure" script should do any necessary tweakings to make the driver compile on the particular system (as GNU apps do).
By "format" I didn't refer only to a file format that drivers are packaged in but also to certain semantics -- like "how is this package compiled and installed?". So, a driver that comes in source code in a .tar.gz and has a "configure" script that must be run before make is in different "format" than a .tar.gz that doesn't need "configure".
Also, installation of a driver should have its mirror (uninstallation) as well. Of course, it's not very practical to install/uninstall a driver every time it's devices are plugged/unplugged, so it should be left to the user to uninstall the driver when necessary.
pcimodules should do the high-level management for downloading and installing a driver. It contacts the server to find out where the driver is, downloads the "resource" and then passes control to a specific "resource agent" which can install/uninstall this kind of resource. Resource agents are placed in /etc/hotplug/driver and they accept:
ACTION={install,uninstall} RESOURCE=/filename/of/resource.extension TYPE={tarball,tarball_gnu,...}
That also suggests adding a new "driver.agent" to be called by /sbin/hotplug .
If no one is willing to work on this, I'll try to set up some client-side scripts on my localhost.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 3 lines · you
But how do you resolve all of the previous points I raised about compilers, .config files, in kernel apis, and security with regards to "automatic downloads"?
- compilers
By requiring the user to have a certain gcc (make, ld..) version installed, possibly called kgcc or installed independently of the regular compiler. Its path must be be specified in a configuration file.
- .config files
The source, or at least the headers, and the .config files of the installed kernel must be available. The location could be specified in the configuration file as well. The only real problem are .config files that lack options that are needed by a driver (in other words the installed kernel does not have all features needed by the driver).
- kernel apis
Like with today's patches, even though it would be nice if source compatibility in the stable kernel series could be guaranteed.
- security
The security is not much different than with Debian's apt, Redhat Network or Suse's YOU. Either you get the driver from a trusted source, or the user has to be asked before the system downloads from a new/untrusted source. To allow a download from new/untrusted sources you need to know the root password, of course (or maybe be in a special group)
bye...
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 2 lines · Dmitri
Of course, distributions can make the gcc (suitable for kernel compilations) a required (base) package.
That will be a requirement for any users who want to compile their own or add modules onto existing kernels.
Quoted 5 lines
Quoted 1 line
.config files,
I think they are available in "kernel-devel" package or something like that. They easily can be shipped as part of the kernel RPM. The kernel-devel package is already available in RH, for example.
kbuild 2.5 introduces a standard method of installing .config and System.map at the same time that the kernel and modules are installed. The files do not have to be installed, but if they are, the default location is /lib/modules/$(KERNELRELEASE)/{.config,System.map}. Can you tell that I have got fed up of distributions putting .config and System.map in strange places?
Quoted 3 lines
Of course, the kernel itself can contain a copy of its own .config file and export it through /proc fs, along with all compiler options, that would be neat :-) and only requires an optional module.
Proposed many times, refused by Linus each time. It is kernel bloat for a user space problem that can be solved in user space. See kbuild 2.5.
Quoted 1 line
Linus already signs his kernel releases.
No he does not. The signatures on files obtained from ftp.kernel.org and mirrors are automatically generated by a script on kernel.org as files are uploaded. The signature is by ftpadmin, not the person who put the file there. It says nothing about who uploaded the file, it only proves that the file came from kernel.org and has not been tampered with since upload.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoting Keith Owens <[email protected]>:
Quoted 8 lines
Quoted 1 line
Linus already signs his kernel releases.
No he does not. The signatures on files obtained from ftp.kernel.org and mirrors are automatically generated by a script on kernel.org as files are uploaded. The signature is by ftpadmin, not the person who put the file there. It says nothing about who uploaded the file, it only proves that the file came from kernel.org and has not been tampered with since upload.
The idea is the same: the owner of the secret key signs the package. That owner can be a human or a machine. You have to trust the owner. Of course, automated signing is less secure and I don't think it can be used in this project.
In fact, what the ftp server can do is to verify the signature of the package after the developer uploads it. Only properly signed files, from known developers, are permitted to stay on the distribution site. Just another little bit of security...
Dmitri
Signature · 2 lines
panic("esp: Aiee penguin on the SCSI-bus.");
(from linux/drivers/scsi/esp.c)Quoted 5 lines · Stamatis Mitrofanis
I agree that a C compiler is a nice thing to have, but no one must be forced to have it (say, end users). If it becomes a "base package", every computer must have it installed and then people will start using it when it's not necessary (huge mess). The best thing to do here is to install the right version of gcc on demand (just as we do for drivers).
And what version of gcc would you install? Different versions for different vendor kernels and even releases :)
Quoted 11 lines
Standardize _both_ as the "current kernel configuration" (
/usr/src/linux/.config_and_ a proc fs entry) so that driver sources can freely use it. I'd go for the first one though, here's the rationale:
Since we are going to allow third-party kernel drivers to be compiled for a kernel, we may as well grant them the right to do everything that a "normally" (in-kernel) distributed driver can do. A driver may depend on other drivers to compile/work and they will need to be downloaded, configured and installed as well. A driver must be allowed to set (its own, of course) entries in the kernel configuration file , modifying it for other third-party drivers to read. Thus it's better to use/usr/src/linux/.config.
The /proc .config argument has been answered by Linus. So you can't guarantee that it will be available unless you apply a third party patch to your kernel.
But I don't keep my kernels in /usr/src/linux, and neither do most people. And vendor kernels include a number of different .config files, all of them placed in a different file within the .src.rpm. Work with the LSB if you want to push for this kind of standardization.
Quoted 11 lines
Quoted 8 lines
Quoted 3 lines
in kernel apis,
That's not something new. APIs change all the time, everywhere, not only in OSes. Programmers just need to use #if ... #endif and maybe the driver should be marked as "suitable" or "tested" on some range of kernel releases.
Couldn't the kernel's configuration file be useful in solving this problem? API authors could use it to describe the changes they made in the API.
No, the .config file is used to describe the options that the user wants built for the kernel. It is not a language that can express API changes.
How would you describe the fact that between kernels x and y, the struct usb_serial_device changed the field "lock" from being a "struct spinlock_t" to being a "struct semaphore"? Changes that that _have_ to have code changes to the driver made. That's the main problem that you are going to run into here.
Quoted 3 lines
This script can very well be a hotplug agent supporting four ACTIONs. I posted a message about this a few days ago, but since I didn't put any Subject line no one must have read it. Here it is again:
Remember /sbin/hotplug runs as root. You do _not_ want to be doing any of these actions as root :)
Good luck,
greg k-h
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 4 lines · you
But I don't keep my kernels in
/usr/src/linux, and neither do most people. And vendor kernels include a number of different .config files, all of them placed in a different file within the .src.rpm. Work with the LSB if you want to push for this kind of standardization.
You don't have to use /usr/src/linux, the system would need its own kernel tree at different location (installed by the distribution or copied by a make target similar to modules_install when you compile the kernel yourself). You shouldn't even access it yourself.
For each kernel release there must be a gcc version defined, and the compilation system must have this gcc available. This could mean that you have 10 gccs (if you have 10 different kernels installed). These gcc's should not be in the usual PATH directories (/usr/bin..) but in a private directory.
Quoted 5 lines
How would you describe the fact that between kernels x and y, the struct
usb_serial_devicechanged the field "lock" from being a "structspinlock_t" to being a "struct semaphore"? Changes that that _have_ to have code changes to the driver made. That's the main problem that you are going to run into here.
IMHO it is neccessary that stable kernels keep source compatibility.
Quoted 2 lines
Remember
/sbin/hotplugruns as root. You do _not_ want to be doing any of these actions as root :)
But it shouldn't be a problem to su to a special user for compiling.
bye...
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 5 lines · Tim Jansen
For each kernel release there must be a gcc version defined, and the compilation system must have this gcc available. This could mean that you have 10 gccs (if you have 10 different kernels installed). These gcc's should not be in the usual PATH directories (
/usr/bin..) but in a private directory.
Where do you define this version of gcc? Many versions of gcc can successfully compile a kernel.
Quoted 1 line
IMHO it is neccessary that stable kernels keep source compatibility.
That is not way Linux kernel development is done, sorry.
thanks,
greg k-h
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 2 lines · Greg KH
Where do you define this version of gcc?
Many versions of gcc can successfully compile a kernel.
The easiest way would be to include a file in the kernel that contains the version number of one compiler that is able to compile the kernel. This compiler is then used to compile the kernel.
Quoted 2 lines
Quoted 1 line
IMHO it is neccessary that stable kernels keep source compatibility.
That is not way Linux kernel development is done, sorry.
If the driver does not compile and the package containing the driver states that the driver has not been tested with the kernel version it can just abort and tell the user that the driver has not been ported the newer version yet. This is not worse than the current situation for both user and developer.
I wonder for how long it is possible to keep all existing drivers in the kernel source though.
bye...
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoting Tim Jansen <[email protected]>:
Quoted 7 lines · Greg KH
Quoted 2 lines
Where do you define this version of gcc?
Many versions of gcc can successfully compile a kernel.The easiest way would be to include a file in the kernel that contains the version number of one compiler that is able to compile the kernel. This compiler is then used to compile the kernel.
This will also make the system compiler-independent. You may have the most modern gcc 3.0.1 for your own compilations, but the kernel components will be built with tried, tested and true gcc 2.96.x (or whatever).
Dmitri
Signature · 2 lines
Linux: Where Don't We Want To Go Today? (Submitted by Pancrazio De Mauro, paraphrasing some well-known sales talk.)
Quoted 9 lines · Greg KH
Quoted 5 lines · Stamatis Mitrofanis
I agree that a C compiler is a nice thing to have, but no one must be forced to have it (say, end users). If it becomes a "base package", every computer must have it installed and then people will start using it when it's not necessary (huge mess). The best thing to do here is to install the right version of gcc on demand (just as we do for drivers).
And what version of gcc would you install? Different versions for different vendor kernels and even releases :)
I have decided to add CONFIG_KBUILD_GCC_VERSION to kbuild 2.5, it records the values of gcc -b and -V used to build the kernel. The string is automatically generated and added to the .config, _every_ source file will implicitly depend on that config variable so changing the compiler will force a complete recompile.
Building the kernel and modules from a mixture of compilers is not a supported operation.
This is almost true in kbuild 2.5 already. If you compile with 'gcc' then recompile with 'kgcc' or switch from 'gcc' to 'gcc -V 3.0.1', everything gets rebuilt because the compiler name and flags are part of the saved command, any change in a command forces a recompile of the affected objects. Adding the CONFIG_ variable will document which compiler was used and catches the pathological case where 'gcc' is used but it points to different compilers on different days.
Version information for other build utilities will be added as required.
Quoted 4 lines
But I don't keep my kernels in
/usr/src/linux, and neither do most people. And vendor kernels include a number of different .config files, all of them placed in a different file within the .src.rpm. Work with the LSB if you want to push for this kind of standardization.
Or the kbuild maintainer ;). LSB does not document down to this level of detail so I am standardizing where .config and other files go in 2.5. /lib/modules/uname -r/{.config,System.map,trees}. kbuild 2.5 is not restricted to a single source tree so the single build symlink is useless in 2.5, instead the 'trees' file defines the source tree(s) and the object tree.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 7 lines · Keith Owens
Or the kbuild maintainer ;). LSB does not document down to this level of detail so I am standardizing where .config and other files go in 2.5.
/lib/modules/uname -r/{.config,System.map,trees}. kbuild 2.5 is not restricted to a single source tree so the single build symlink is useless in 2.5, instead the 'trees' file defines the source tree(s) and the object tree.
Very nice. I am sure lots of people will appreciate this and the gcc config item. Thanks for adding them.
greg k-h
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel