MODULE_FIRMWARE for binary firmware(s)
PATCH 1/1 MODULE_FIRMWARE for binary firmware(s)
From: Jon Masters <[email protected]>
Right now, various kernel modules are being migrated over to use request_firmware in order to pull in binary firmware blobs from userland when the module is loaded. This makes sense.
However, there is right now little mechanism in place to automatically determine which binary firmware blobs must be included with a kernel in order to satisfy the prerequisites of these drivers. This affects vendors, but also regular users to a certain extent too.
The attached patch introduces MODULE_FIRMWARE as a mechanism for advertising that a particular firmware file is to be loaded - it will then show up via modinfo and could be used e.g. when packaging a kernel.
include/linux/module.h 2006-04-12 13:51:56.000000000 +0100 renamed from include/linux/module.h 2006-04-07 17:56:47.000000000 +0100 +2 -0
@@ -155,6 +155,8 @@ */ #define MODULE_VERSION(_version) MODULE_INFO(version, _version) +#define MODULE_FIRMWARE(_firmware) MODULE_INFO(firmware, _firmware) + /* Given an address, look for it in the exception tables */ const struct exception_table_entry *search_exception_tables(unsigned long add);
Jon,
would it be possible to have something less then an initrd that would
allow the firmware blob to be packaged with the kernel? Your approach is just fine if the things that will need firmware are compiled as modules (the firmware can be placed on the same filesystem image as the modules that need it) but some people prefer to build monolithic kernel images, and in those cases there is no way for the kernel to get the firmware as there is no filesystem available yet.
I know that Rob Landley is doing some tricks with firmware linux to append an initramfs to the kernel image (http://www.landley.net/code/firmware/). There is already support in the kconfig for specifying the initramfs source, why culdn't we have the build process fetch any firmware needed by the non-modular portions and store them at the end of the kernel image so that they can be found during boot.
if nothing else make a skeleton initramfs that contains the modules and just enough other logic to continue the boot process as if the initramfs wasn't involved.
right now I have a laptop that's not working on wireless due to this exact problem (ipw2200 driver, and I haven't taken the time to setup modules for it yet)
David Lang
Quoted 39 lines · Jon Masters
Date: Wed, 19 Apr 2006 00:41:56 +0100
From: Jon Masters <[email protected]>
To: [email protected], [email protected]
Subject: [PATCH]MODULE_FIRMWAREfor binary firmware(s)From: Jon Masters <[email protected]>
Right now, various kernel modules are being migrated over to use
request_firmwarein order to pull in binary firmware blobs from userland when the module is loaded. This makes sense.However, there is right now little mechanism in place to automatically determine which binary firmware blobs must be included with a kernel in order to satisfy the prerequisites of these drivers. This affects vendors, but also regular users to a certain extent too.
The attached patch introduces
MODULE_FIRMWAREas a mechanism for advertising that a particular firmware file is to be loaded - it will then show up via modinfo and could be used e.g. when packaging a kernel.Signed-off-by Jon Masters <[email protected]>diff -urN
linux-2.6.16.2_orig/include/linux/module.hlinux-2.6.16.2_dev/include/linux/module.hinclude/linux/module.h 2006-04-12 13:51:56.000000000 +0100 renamed from include/linux/module.h 2006-04-07 17:56:47.000000000 +0100 +2
-1@@ -155,6 +155,8 @@ */ #define MODULE_VERSION(_version) MODULE_INFO(version, _version) +#define MODULE_FIRMWARE(_firmware) MODULE_INFO(firmware, _firmware) + /* Given an address, look for it in the exception tables */ const struct exception_table_entry *search_exception_tables(unsigned long add); -To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 2 lines
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare
Quoted 2 lines · David Lang
would it be possible to have something less then an initrd that would
allow the firmware blob to be packaged with the kernel?
With some modifications perhaps. I don't know if I see the value tbh :-)
Quoted 2 lines
Your approach is just fine if the things that will need firmware are compiled as modules
Hmmm. Yeah. I'm not sure what the general feeling is on this - I'm tempted to say that we expect modules to be used and that if they're not then the vendor/user has to do the hoop jumping for themselves. This code won't stop you from making a monolithic kernel and satisfying any module requirements for yourself :-)
Jon.
Quoted 8 lines · Jon Masters
Quoted 2 lines
Your approach is just fine if the things that will need firmware are compiled as modules
Hmmm. Yeah. I'm not sure what the general feeling is on this - I'm tempted to say that we expect modules to be used and that if they're not then the vendor/user has to do the hoop jumping for themselves. This code won't stop you from making a monolithic kernel and satisfying any module requirements for yourself :-)
Two things with this.
1. there is no way to satisfy the firmware requirements currently
2. I thought I heard Linus state recently that makeing something only work as a module was unacceptable, officially stateing that modules are required and monolithic kernels aren't allowed anymore doesn't sound reasonable.
David Lang
Signature · 2 lines
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare
Quoted 4 lines · David Lang
2. I thought I heard Linus state recently that makeing something only work as a module was unacceptable, officially stateing that modules are required and monolithic kernels aren't allowed anymore doesn't sound reasonable.
It works fine with or without modules - it's just that if you don't have the module then you have to supply the firmware for yourself. It's just like any of the other MODULE_ tags.
Jon.
Hi Jon,
Quoted 8 lines
However, there is right now little mechanism in place to automatically determine which binary firmware blobs must be included with a kernel in order to satisfy the prerequisites of these drivers. This affects vendors, but also regular users to a certain extent too.
The attached patch introduces
MODULE_FIRMWAREas a mechanism for advertising that a particular firmware file is to be loaded - it will then show up via modinfo and could be used e.g. when packaging a kernel.
I haven't really understood what problem this solves. Is this just a standardised form of documentation, or are you imagining that an automatic tool will use this to auto include a minimal set of firmware files in an initrd? I'm thinking of something like this: (1) redhat (or whoever) ships firmware files for every driver under the sun in /lib/firmware; (2) redhat wants to allow users to have a customized initrd with only essential drivers;
(3) the tool goes through the list of essential drivers, looks up the firmware string via MODULE_FIRMWARE, finds the file in /lib/firmware, and includes it in the initrd.
All the best,
Duncan.
Quoted 1 line · Duncan Sands
Hi Jon,
Hi Duncan.
Quoted 5 lines
Quoted 5 lines
However, there is right now little mechanism in place to automatically determine which binary firmware blobs must be included with a kernel in order to satisfy the prerequisites of these drivers. This affects vendors, but also regular users to a certain extent too.
Quoted 8 lines
Quoted 3 lines
The attached patch introduces
MODULE_FIRMWAREas a mechanism for advertising that a particular firmware file is to be loaded - it will then show up via modinfo and could be used e.g. when packaging a kernel.I haven't really understood what problem this solves. Is this just a standardised form of documentation, or are you imagining that an automatic tool will use this to auto include a minimal set of firmware files in an initrd?
I'm imagining that the resultant modinfo output can be used by a tool for anyone to package up the correct firmware to go with a given driver. Right now, there's no way to do that - i.e. we've gone backwards from a standpoint of coupling a kernel with firmware. I completely understand why firmware doesn't really belong in the kernel, so let's add this :-)
Quoted 6 lines
I'm thinking of something like this: (1) redhat (or whoever) ships firmware files for every driver under the sun in
/lib/firmware; (2) redhat wants to allow users to have a customized initrd with only essential drivers;
(3) the tool goes through the list of essential drivers, looks up the firmware string viaMODULE_FIRMWARE, finds the file in/lib/firmware, and includes it in the initrd.
That kind of thing. It's not just Red Hat who benefit - anyone who wants to package up a kernel and do something with it will want to know about firmware they might need. Including everything in /lib/firmware "just in case" is as ugly as having userspace tools with duplicated logic that need to understand about the internals of a driver module.
Jon.
Quoted 8 lines
Quoted 4 lines
I haven't really understood what problem this solves. Is this just a standardised form of documentation, or are you imagining that an automatic tool will use this to auto include a minimal set of firmware files in an initrd?
I'm imagining that the resultant modinfo output can be used by a tool for anyone to package up the correct firmware to go with a given driver.
If a tool is to do the packaging, then this means that the firmware must already be present on the machine, for example in /lib/firmware. Logically speaking, that means the role of any tool is to select a subset of the files in /lib/firmware, for example a minimal set for inclusion in an initrd. After all, it can't add new files that don't exist on the machine, since where would it get them from? (I suppose it could download them from "firmware central"). Is there any use for such a tool, besides creating small initrds? How big an issue is that? Is there really any reason not to simply throw everything in /lib/firmware into any initrd that is created? But perhaps you are thinking of the suppliers of a distribution who have a gazillion firmware files that they've collected over the years, some of which are too old and some of which are too recent for the drivers that will be shipped; and the tool is to select the subset which is relevant for the shipped driver versions? i.e. the firmware selection process is not done on each end user's machine, but by the people preparing the distribution.
Quoted 4 lines
Right now, there's no way to do that - i.e. we've gone backwards from a standpoint of coupling a kernel with firmware. I completely understand why firmware doesn't really belong in the kernel, so let's add this :-)
I guess a big difference between the speedtouch and the kinds of drivers you seem to be thinking of, is that in your case there is a fairly tight coupling between firmware and driver versions: a given driver version will only work with a certain version of the firmware and vice-versa. In the case of the speedtouch, we have no control over (and not much knowledge about) which firmware gets given to people along with their modems, so there is really no coupling at all between firmware versions and driver versions.
Quoted 10 lines
Quoted 6 lines
I'm thinking of something like this: (1) redhat (or whoever) ships firmware files for every driver under the sun in
/lib/firmware; (2) redhat wants to allow users to have a customized initrd with only essential drivers;
(3) the tool goes through the list of essential drivers, looks up the firmware string viaMODULE_FIRMWARE, finds the file in/lib/firmware, and includes it in the initrd.That kind of thing. It's not just Red Hat who benefit - anyone who wants to package up a kernel and do something with it will want to know about firmware they might need.
For this they could just read the documentation. They'll need to anyway just to find out where they are supposed to get the firmware from.
Quoted 4 lines
Including everything in
/lib/firmware"just in case" is as ugly as having userspace tools with duplicated logic that need to understand about the internals of a driver module.
Don't distributions need to ship vast quantities of firmware in /lib/firmware anyway, I mean firmware for every driver in the kernel, since they don't know what hardware the end user may have? So I guess you are talking about individual users who compile their own kernels here.
Ciao,
D.
Quoted 11 lines · Duncan Sands
Quoted 8 lines
Quoted 4 lines
I haven't really understood what problem this solves. Is this just a standardised form of documentation, or are you imagining that an automatic tool will use this to auto include a minimal set of firmware files in an initrd?
I'm imagining that the resultant modinfo output can be used by a tool for anyone to package up the correct firmware to go with a given driver.
If a tool is to do the packaging, then this means that the firmware must already be present on the machine, for example in
/lib/firmware.
Yes. Although, potentially more clever things could happen in userspace in the future.
Quoted 2 lines
that means the role of any tool is to select a subset of the files in
/lib/firmware, for example a minimal set for inclusion in an initrd.
For example.
Quoted 2 lines
Is there really any reason not to simply throw everything in
/lib/firmwareinto any initrd that is created?
You /could/ just build every driver into the initrd too, and every firmware, and... but it might be nice if it was possible to know what should be where. Right now, we've lost that because decoupling firmware from the kernel means that tools outside of the kernel can't tell what firmware files should be around.
Quoted 5 lines
Quoted 4 lines
Right now, there's no way to do that - i.e. we've gone backwards from a standpoint of coupling a kernel with firmware. I completely understand why firmware doesn't really belong in the kernel, so let's add this :-)
Quoted 7 lines
I guess a big difference between the speedtouch and the kinds of drivers you seem to be thinking of, is that in your case there is a fairly tight coupling between firmware and driver versions: a given driver version will only work with a certain version of the firmware and vice-versa. In the case of the speedtouch, we have no control over (and not much knowledge about) which firmware gets given to people along with their modems, so there is really no coupling at all between firmware versions and driver versions.
I've also repeatedly said that I don't think this really helps with things like speedtouch since you can't distribute that firmware anyway. This patch does help people who play nicely with the kernel who are migrating away from shoving blobs into the kernel (quite rightly) and who supply redistributable firmware. One example might be the QLogic driver in my RFC.
Quoted 6 lines
Quoted 3 lines
That kind of thing. It's not just Red Hat who benefit - anyone who wants to package up a kernel and do something with it will want to know about firmware they might need.
For this they could just read the documentation. They'll need to anyway just to find out where they are supposed to get the firmware from.
You're assuming firmware is always on some random vendor website under a questionable license and can't be redistributed. That wasn't my first consideration - this doesn't really help much there, except you get to know what you don't have :-)
Jon.
Quoted 3 lines · Jon Masters
The attached patch introduces
MODULE_FIRMWAREas a mechanism for advertising that a particular firmware file is to be loaded - it will then show up via modinfo and could be used e.g. when packaging a kernel.
I (and several others who met at OLS) would really like for this to get upstream. Can you sling that one line patch (will forward again) into -mm please?
Jon.
PATCH 1/1 MODULE_FIRMWARE for binary firmware(s)
In reply to Jon Masters---------- Forwarded message ----------
From: Jon Masters <[email protected]>
Date: Apr 19, 2006 12:41 AM
Subject: [PATCH] MODULE_FIRMWARE for binary firmware(s)
To: [email protected], [email protected]
From: Jon Masters <[email protected]>
Right now, various kernel modules are being migrated over to use request_firmware in order to pull in binary firmware blobs from userland when the module is loaded. This makes sense.
However, there is right now little mechanism in place to automatically determine which binary firmware blobs must be included with a kernel in order to satisfy the prerequisites of these drivers. This affects vendors, but also regular users to a certain extent too.
The attached patch introduces MODULE_FIRMWARE as a mechanism for advertising that a particular firmware file is to be loaded - it will then show up via modinfo and could be used e.g. when packaging a kernel.
diff -urN linux-2.6.16.2_orig/include/linux/module.h linux-2.6.16.2_dev/include/linux/module.h
--- linux-2.6.16.2_orig/include/linux/module.h 2006-04-07 17:56:47.000000000 +0100
+++ linux-2.6.16.2_dev/include/linux/module.h 2006-04-12
13:51:56.000000000 +0100
@@ -155,6 +155,8 @@
*/
#define MODULE_VERSION(_version) MODULE_INFO(version, _version)
+#define MODULE_FIRMWARE(_firmware) MODULE_INFO(firmware, _firmware) +
/* Given an address, look for it in the exception tables */
const struct exception_table_entry *search_exception_tables(unsigned long add); -
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Sam Ravnborg
NAKQuoted 13 lines · Jon Masters
---------- Forwarded message ----------
From: Jon Masters <[email protected]>
Date: Apr 19, 2006 12:41 AM
Subject: [PATCH]MODULE_FIRMWAREfor binary firmware(s)
To: [email protected], [email protected]From: Jon Masters <[email protected]>
Right now, various kernel modules are being migrated over to use
request_firmwarein order to pull in binary firmware blobs from userland when the module is loaded. This makes sense.
Quoted 8 lines
However, there is right now little mechanism in place to automatically determine which binary firmware blobs must be included with a kernel in order to satisfy the prerequisites of these drivers. This affects vendors, but also regular users to a certain extent too.
The attached patch introduces
MODULE_FIRMWAREas a mechanism for advertising that a particular firmware file is to be loaded - it will then show up via modinfo and could be used e.g. when packaging a kernel.
NAK.
Please provide some inline documentation so users knows how it is supposed to be used etc. See MODULE_VERSION as an example. Changelog comments is not enough documentation.
Sam