MODULE_FIRMWARE for binary firmware(s)
PATCH 1/1 MODULE_FIRMWARE for binary firmware(s)
From: Jon Masters <[email protected]>
Right now, various kernel modules are being migrated over to use request_firmware in order to pull in binary firmware blobs from userland when the module is loaded. This makes sense.
However, there is right now little mechanism in place to automatically determine which binary firmware blobs must be included with a kernel in order to satisfy the prerequisites of these drivers. This affects vendors, but also regular users to a certain extent too.
The attached patch introduces MODULE_FIRMWARE as a mechanism for advertising that a particular firmware file is to be loaded - it will then show up via modinfo and could be used e.g. when packaging a kernel.
Comments added in line with all the other MODULE_ tag
include/linux/module.h +5 -0
@@ -156,6 +156,11 @@ extern struct module __this_module;*/ #define MODULE_VERSION (_version) MODULE_INFO(version, _version) +/* Optional firmware file (or files) needed by the module +* format is simply firmware file name. Multiple firmware +* files require multiple MODULE_FIRMWARE() specifiers */ +#define MODULE_FIRMWARE (_firmware) MODULE_INFO(firmware, _firmware) +/* Given an address, look for it in the exception tables */ const struct exception_table_entry *search_exception_tables (unsigned long add);
This is a reference implementation with the debian mkinitrd-tools package. It shows how to identify the firmware files necessary for drivers in the initrd and also includes a primitive system for loading them.
I've tested this with the aic94xx driver using the new MODULE_FIRMWARE() tag. Initramfs should be much easier because it already includes most of the boot time loading; all it has to do is the piece identifying the firmware for the selected modules.
James
---
mkinitrd +20 -0
@@ -950,6 +950,7 @@ add_modules_dep() { return elif ! [ $oldstyle ]; then add_modules_dep_2_5 $VERSION + add_firmware $VERSION return fi
@@ -1016,6 +1017,25 @@ add_modules_dep_2_5() { fi } +add_firmware() { + ver=$1 + set -- $FSTYPES + unset IFS + + cat modules.? | + while read junk mod junk; do + modpath=$(modprobe --set-version "$ver" --list $mod) + if [ -z "$modpath" ]; then + continue; + fi + p=$(modinfo -F firmware "$modpath" |sed 's/^/\/lib\/firmware\//') + if [ -n "$p" ]; then + echo $p + echo /usr/sbin/firmware_loader + fi + done +} + add_command() { if [ -h initrd/"$1" ]; then return
firmware_loader new file +29 -0
@@ -0,0 +1,29 @@ +#!/bin/sh -e +# +# firmware loader agent +# +FIRMWARE_DIRS="/lib/firmware" + +if [ "$SUBSYSTEM" != "firmware" ]; then + exit 0; +fi + +if [ ! -e /sys/$DEVPATH/loading ]; then + echo "/sys/$DEVPATH/ does not exist" + exit 1 +fi + +for DIR in $FIRMWARE_DIRS; do + [ -e "$DIR/$FIRMWARE" ] || continue + echo 1 > /sys/$DEVPATH/loading + cat "$DIR/$FIRMWARE" > /sys/$DEVPATH/data + echo 0 > /sys/$DEVPATH/loading + exit 0 +done + +# the firmware was not found +echo -1 > /sys/$DEVPATH/loading + +echo "Cannot find the $FIRMWARE firmware" +exit 1 +
debian/rules +1 -1
@@ -35,7 +35,7 @@ install: install -o root -g root -m 644 \ echo init linuxrc debian/initrd-tools/usr/share/initrd-tools/ install -o root -g root -m 755 \ - mkinitrd debian/initrd-tools/usr/sbin/ + mkinitrd firmware_loader debian/initrd-tools/usr/sbin/ install -o root -g root -m 644 \ mkinitrd.conf modules debian/initrd-tools/etc/mkinitrd/ ifeq ($(DEB_HOST_ARCH),powerpc)
linuxrc +4 -0
@@ -10,3 +10,7 @@ echo 256 > proc/sys/kernel/real-root-dev mount -nt tmpfs tmpfs bin || mount -nt ramfs ramfs bin echo $root > bin/root +if [ -x /usr/sbin/firmware_loader ]; then + echo /usr/sbin/firmware_loader > /proc/sys/kernel/hotplug +fi +
init +1 -0
@@ -366,6 +366,7 @@ get_cmdline [ -c /dev/.devfsd ] && DEVFS=yes mount -nt devfs devfs devfs +mount -nt sysfs sysfs sys if [ $IDE_CORE != none ] && [ -n "$ide_options" ]; then echo modprobe -k $IDE_CORE "options=\"$ide_options\"" modprobe -k $IDE_CORE options="$ide_options"
Quoted 9 lines · James Bottomley
This is a reference implementation with the debian mkinitrd-tools package. It shows how to identify the firmware files necessary for drivers in the initrd and also includes a primitive system for loading them.
I've tested this with the aic94xx driver using the new
MODULE_FIRMWARE()tag. Initramfs should be much easier because it already includes most of the boot time loading; all it has to do is the piece identifying the firmware for the selected modules.
Notice that mkinitrd-tools is dead, and will probably be removed from etch.
mkinitramfs-tools and yaird are the two currently used tools.
Friendly,
Sven Luther
Quoted 6 lines · Sven Luther
Quoted 4 lines · James Bottomley
I've tested this with the aic94xx driver using the new
MODULE_FIRMWARE()tag. Initramfs should be much easier because it already includes most of the boot time loading; all it has to do is the piece identifying the firmware for the selected modules.
...
Quoted 2 lines
Notice that mkinitrd-tools is dead, and will probably be removed from etch.
request_firmware() is dead also.
YMMV, but three years, and there are still big chunks of binary in kernel. And please don't add new useless info _in_ it.
Nobody cares.
While this implementation exists, it wasn't well designed and hard to use. As with in-kernel bootsplash and i18n, everything maybe done in userspace, only with little help from the kernel:
<http://permalink.gmane.org/gmane.linux.kernel/435955>.
Thanks.
Signature · 3 lines
-o--=O`C /. .\ (+) #oo'L O o | <___=E M ^-- | (you're barking up the wrong tree)
Quoted 3 lines · Oleg Verych
request_firmware()is dead also.
YMMV, but three years, and there are still big chunks of binary in kernel. And please don't add new useless info _in_ it.
I er don't think so.
We (as in the Kernel) are forcing drivers on to this path. You should have noticed it already with the qla2xxx. The aic94xx is the first driver that's likely found as the boot driver for a system, that's why I get to propose the reference implementation.
James
Quoted 9 lines · James Bottomley
Quoted 3 lines · Oleg Verych
request_firmware()is dead also.
YMMV, but three years, and there are still big chunks of binary in kernel. And please don't add new useless info _in_ it.I er don't think so.
Hell, what can be as easy as this:
,-
|modprobe $drv
|(dd </lib/firmware/$drv.bin>/dev/blobs && echo OK) || echo Error `-
where /dev/blobs is similar to /dev/port or even /dev/null char device. if synchronization is needed, add echo $drv >/dev/blobs, remove modprobe,
Quoted 3 lines
James
?
Signature · 4 lines
-- -o--=O`C /. .\ (+) #oo'L O o | <___=E M ^-- | (you're barking up the wrong tree)
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 13 lines · Oleg Verych
Quoted 12 lines · Greg KH
Quoted 10 lines · Oleg Verych
Quoted 3 lines
Quoted 3 lines
request_firmware()is dead also.
YMMV, but three years, and there are still big chunks of binary in kernel. And please don't add new useless info _in_ it.Hell, what can be as easy as this:
,-
|modprobe $drv
|(dd </lib/firmware/$drv.bin>/dev/blobs && echo OK) || echo Error `-
where/dev/blobsis similar to/dev/portor even/dev/nullchar device. if synchronization is needed, addecho $drv >/dev/blobs, remove modprobe,
Quoted 4 lines
Quoted 3 lines
I don't see such code in the kernel at this time. So it's not a solution, sorry.
I know. return
-ENOPATCH
Yes, and that's the only way to make changes in the kernel, sorry.
Quoted 5 lines
I'm nether a CS nor software engineer, just wondering why simple thing isn't simple _in_ the Kernel. I'm reading list "just for fun (C)" and any good word about this (IMHO) unix-way design *may* lead professional programmers to do tiny worthy things (think about kevent discussion).
If it's (i'm) stupid, please, say so (in the way Nicholas Miell did ;).
I don't think it's workable, and goes against the current way the kernel does things. But please, feel free to prove me wrong with a patch otherwise. I don't want to debate it otherwise.
I think the current way we handle firmware works quite well, especially given the wide range of different devices that it works for (everything from BIOS upgrades to different wireless driver stages).
thanks,
greg k-h
Quoted 18 lines · Greg KH
Quoted 16 lines · Oleg Verych
Quoted 14 lines · Greg KH
Quoted 11 lines · Oleg Verych
Quoted 3 lines
Quoted 3 lines
request_firmware()is dead also.
YMMV, but three years, and there are still big chunks of binary in kernel. And please don't add new useless info _in_ it.Hell, what can be as easy as this:
,-
|modprobe $drv
|(dd </lib/firmware/$drv.bin>/dev/blobs && echo OK) || echo Error `-
where/dev/blobsis similar to/dev/portor even/dev/nullchar device. if synchronization is needed, addecho $drv >/dev/blobs, remove modprobe,
...
Quoted 11 lines
Quoted 6 lines
I'm nether a CS nor software engineer, just wondering why simple thing isn't simple _in_ the Kernel. I'm reading list "just for fun (C)" and any good word about this (IMHO) unix-way design *may* lead professional programmers to do tiny worthy things (think about kevent discussion).
If it's (i'm) stupid, please, say so (in the way Nicholas Miell did ;).I don't think it's workable, and goes against the current way the kernel does things. But please, feel free to prove me wrong with a patch otherwise. I don't want to debate it otherwise.
Thanks, and OK, this is my last reply on this.
Quoted 4 lines
I think the current way we handle firmware works quite well, especially given the wide range of different devices that it works for (everything from BIOS upgrades to different wireless driver stages).
Oh, come on, even skilled developers (not particular kernel's) having difficulties with current hotplug-sysfs-modprobe thing;
in this case one couldn't easily figure out problems and way to solve them
<http://permalink.gmane.org/gmane.comp.hardware.texas-instruments.msp430.gcc.user/5444>
Goodbye.
--
-o--=O`C /. .\ (+)
#oo'L O o |
<___=E M ^-- | (you're barking up the wrong tree)
Quoted 3 lines · Greg KH
I think the current way we handle firmware works quite well, especially given the wide range of different devices that it works for (everything from BIOS upgrades to different wireless driver stages).
the current system works for many people yes, but not everyone.
I'm still waiting to find a way to get the iw2200 working without having to use modules.
there was a post a month or two ago from someone who had indicated they found a way to re-initialize it after the system came up, but after someone asked for details of how to do it there was no response.
David Lang
Quoted 10 lines · David Lang
Quoted 3 lines · Greg KH
I think the current way we handle firmware works quite well, especially given the wide range of different devices that it works for (everything from BIOS upgrades to different wireless driver stages).
the current system works for many people yes, but not everyone.
I'm still waiting to find a way to get the iw2200 working without having to use modules.
Sounds like a bug you need to pester the iw2200 developers about then. I don't think it has much to do with the firmware subsystem though :)
thanks,
greg k-h
Quoted 7 lines · Greg KH
Quoted 6 lines · David Lang
Quoted 3 lines · Greg KH
I think the current way we handle firmware works quite well, especially given the wide range of different devices that it works for (everything from BIOS upgrades to different wireless driver stages).
Quoted 7 lines
Quoted 4 lines
the current system works for many people yes, but not everyone.
I'm still waiting to find a way to get the iw2200 working without having to use modules.
Sounds like a bug you need to pester the iw2200 developers about then. I don't think it has much to do with the firmware subsystem though :)
Well, yes and no.
The ipw needs the firmware on insmod time (in contrast to bcm43xx for example, which needs it on ifconfig up time).
So ipw needs to call request_firmware at insmod time. In case of built-in, that is when the initcall happens. No userland is available and request_firmware can not call the userspace helpers to upload the firmware to sysfs.
Well, not really easy to find a sane solution for this. :)
Signature · 1 line
Greetings Michael.
Quoted 16 lines · Michael Buesch
Quoted 7 lines · Greg KH
Quoted 6 lines · David Lang
Quoted 3 lines · Greg KH
I think the current way we handle firmware works quite well, especially given the wide range of different devices that it works for (everything from BIOS upgrades to different wireless driver stages).
Quoted 7 lines
Quoted 4 lines
the current system works for many people yes, but not everyone.
I'm still waiting to find a way to get the iw2200 working without having to use modules.
Sounds like a bug you need to pester the iw2200 developers about then. I don't think it has much to do with the firmware subsystem though :)
Quoted 7 lines
Well, yes and no.
The ipw needs the firmware on insmod time (in contrast to bcm43xx for example, which needs it on ifconfig up time).
So ipw needs to callrequest_firmwareat insmod time. In case of built-in, that is when the initcall happens. No userland is available andrequest_firmwarecan not call the userspace helpers to upload the firmware to sysfs.
I dont use nor do I have access ipw hardware, but:
If it is an initcall, the initramfs should be usable at that time. A creative CONFIG_INITRAMFS_SOURCE= will help, add the firmware and a small helper that does the cat(1).
Quoted 26 lines · Olaf Hering
Quoted 16 lines · Michael Buesch
Quoted 7 lines · Greg KH
Quoted 6 lines · David Lang
Quoted 3 lines · Greg KH
I think the current way we handle firmware works quite well, especially given the wide range of different devices that it works for (everything from BIOS upgrades to different wireless driver stages).
Quoted 7 lines
Quoted 4 lines
the current system works for many people yes, but not everyone.
I'm still waiting to find a way to get the iw2200 working without having to use modules.
Sounds like a bug you need to pester the iw2200 developers about then. I don't think it has much to do with the firmware subsystem though :)
Quoted 7 lines
Well, yes and no.
The ipw needs the firmware on insmod time (in contrast to bcm43xx for example, which needs it on ifconfig up time).
So ipw needs to callrequest_firmwareat insmod time. In case of built-in, that is when the initcall happens. No userland is available andrequest_firmwarecan not call the userspace helpers to upload the firmware to sysfs.
Quoted 5 lines
I dont use nor do I have access ipw hardware, but:
If it is an initcall, the initramfs should be usable at that time. A creativeCONFIG_INITRAMFS_SOURCE=will help, add the firmware and a small helper that does the cat(1).
you are assuming that
1. modules are enabled and ipw2200 is compiled as a module
2. initrd or initramfs are in use
besides, several kernel versions ago this used to work. the current requirement is a regression as far as the user is concerned.
David Lang
Quoted 3 lines · David Lang
besides, several kernel versions ago this used to work. the current requirement is a regression as far as the user is concerned.
Then go bug the driver authors please :)
thanks,
greg k-h
Quoted 3 lines · David Lang
you are assuming that
1. modules are enabled and ipw2200 is compiled as a module
No, why?
Quoted 1 line
2. initrd or initramfs are in use
initramfs is always in use.
Quoted 5 lines · Olaf Hering
Quoted 3 lines
you are assuming that
1. modules are enabled and ipw2200 is compiled as a module
No, why?
becouse if the ipw isn't compiled as a module then it's initialized (without firmware) before the initramfs or initrd is run. if it could be initialized later without being a module then it could be initialized as part of the normal system
Quoted 3 lines
Quoted 1 line
2. initrd or initramfs are in use
initramfs is always in use.
not on my machines.
what does it use for the initramfs?
I don't enable the options in the kernel for initrd, and I don't give it any source for an initramfs.
David Lang
P.S. there was a suggestion yesterday in this thread that I haven't tested yet. I plan to test that tonight. if it works then the card can be reinitialized after the system boots, still no initrd or initramfs needed.
Quoted 4 lines · David Lang
Quoted 2 lines · Olaf Hering
Quoted 2 lines
you are assuming that
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
1. modules are enabled and ipw2200 is compiled as a module
No, why?
becouse if the ipw isn't compiled as a module then it's initialized (without firmware) before the initramfs or initrd is run. if it could be initialized later without being a module then it could be initialized as part of the normal system
Euh, ...
I wonder why you need to initialize the ipw2200 module so early ? It was my understanding that the initramfs thingy was run very early in the process, and i had even thought of moving fbdev modules into it.
Do you really need to bring up ipw2200 so early ? It is some kind of wireless device, right ?
As for initramfs, you can just cat it behind the kernel, and it should work just fine, or at least this is how it was supposed to work.
Friendly,
Sven Luther
Quoted 14 lines · Sven Luther
Quoted 4 lines · David Lang
Quoted 2 lines · Olaf Hering
Quoted 2 lines
you are assuming that
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
1. modules are enabled and ipw2200 is compiled as a module
No, why?
becouse if the ipw isn't compiled as a module then it's initialized (without firmware) before the initramfs or initrd is run. if it could be initialized later without being a module then it could be initialized as part of the normal system
Quoted 8 lines
Euh, ...
I wonder why you need to initialize the ipw2200 module so early ? It was my understanding that the initramfs thingy was run very early in the process, and i had even thought of moving fbdev modules into it.
Do you really need to bring up ipw2200 so early ? It is some kind of wireless device, right ?
if modules are not in use the device is initialized when the kernel starts up. this is before any userspace starts.
Quoted 2 lines
As for initramfs, you can just cat it behind the kernel, and it should work just fine, or at least this is how it was supposed to work.
yes, if I want to set one up.
other then this new requirement to have the ipw2200 driver as a module I have no reason to use one. normal userspace is good enough for me.
David Lang
Quoted 4 lines
Friendly,
Sven Luther
Quoted 23 lines · David Lang
Quoted 21 lines · Sven Luther
Quoted 4 lines · David Lang
Quoted 2 lines · Olaf Hering
Quoted 2 lines
you are assuming that
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
1. modules are enabled and ipw2200 is compiled as a module
No, why?
becouse if the ipw isn't compiled as a module then it's initialized (without firmware) before the initramfs or initrd is run. if it could be initialized later without being a module then it could be initialized as part of the normal system
Euh, ...
I wonder why you need to initialize the ipw2200 module so early ? It was my understanding that the initramfs thingy was run very early in the process, and
i had even thought of moving fbdev modules into it.
Quoted 6 lines
Quoted 3 lines
Do you really need to bring up ipw2200 so early ? It is some kind of wireless
device, right ?if modules are not in use the device is initialized when the kernel starts up. this is before any userspace starts.
Well. but you could do the initialization at open time too, like the other case that was mentioned here, no ?
Quoted 7 lines
Quoted 2 lines
As for initramfs, you can just cat it behind the kernel, and it should work just fine, or at least this is how it was supposed to work.
yes, if I want to set one up.
other then this new requirement to have the ipw2200 driver as a module I have no reason to use one. normal userspace is good enough for me.
Well, ok.
The real question seems to be if we want to keep the firmware inside the driver or not.
If we want to remove it, then we put, not the module, but the firmware itself with some basic userspace to load it on demand in the initramfs, and this is reason enough to create an initramfs. The fact that the builtin device is initialized before the initramfs is loaded seems like a bug to me, since the idea of the initramfs (well, one of them at least), was to initialize it early enough for this kind of things.
If on the other side, it is more important to not have an initramfs (because of security issues, or bootloader constraints or what not), then sure, there is not much choice than putting the firmware in the driver or in the kernel directly.
But again, the initramfs is just a memory space available at the end of the kernel, and there is no hardware-related constraint to access it which are in any way different from having the firmware linked in together with the kernel, so it is only a matter of organisation of code, as well as taking a decision on the above, and to act accordyingly.
Does using an initramfs really have some negative side, security related ? Would some kind of signed or encrypted initramfs be preferable there ?
Friendly,
Sven Luther
Quoted 10 lines · Sven Luther
Quoted 7 lines
Quoted 4 lines
Do you really need to bring up ipw2200 so early ? It is some kind of wireless
device, right ?if modules are not in use the device is initialized when the kernel starts up. this is before any userspace starts.
Well. but you could do the initialization at open time too, like the other case that was mentioned here, no ?
no, at least not in the current kernel. as was mentioned earlier in this thread the ipw2200 needs the firmware at initialization, but some others don't need it until open. I don't know if it's even possible to re-write the driver to do this.
Quoted 13 lines
Quoted 7 lines
Quoted 2 lines
As for initramfs, you can just cat it behind the kernel, and it should work just fine, or at least this is how it was supposed to work.
yes, if I want to set one up.
other then this new requirement to have the ipw2200 driver as a module I have no reason to use one. normal userspace is good enough for me.
Well, ok.
The real question seems to be if we want to keep the firmware inside the driver or not.
Quoted 6 lines
If we want to remove it, then we put, not the module, but the firmware itself with some basic userspace to load it on demand in the initramfs, and this is reason enough to create an initramfs. The fact that the builtin device is initialized before the initramfs is loaded seems like a bug to me, since the idea of the initramfs (well, one of them at least), was to initialize it early enough for this kind of things.
this isn't my understanding.
my understanding is that the kernel fully initializes all built-in drivers, then loads userspace and starts running it.
that userspace can be on a device that it knows how to read, or it can be userspace on initramfs so that you can load additional modules to give you access to the hardware that you want to run on.
this is needed if your root drive is a SCSI drive and you have it's driver compiled as a module for example.
this is needed if your root drive uses dm and you need to initialize the array (one advantage of md, from the user standpoint, is that it doesn't require this additional layer before use)
however this is not soon enough to supply the firmware for devices like this.
Quoted 10 lines
If on the other side, it is more important to not have an initramfs (because of security issues, or bootloader constraints or what not), then sure, there is not much choice than putting the firmware in the driver or in the kernel directly.
But again, the initramfs is just a memory space available at the end of the kernel, and there is no hardware-related constraint to access it which are in any way different from having the firmware linked in together with the kernel, so it is only a matter of organisation of code, as well as taking a decision on the above, and to act accordyingly.
if the firmware needed for any drivers compiled in was appended to the kernel the same way that initramfs is, without requireing the other things needed to make initrmfs useable I think that would be reasonable (bundling them togeather as opposed to embedding the firmware in the kernel). it may even be possible to have the firmware as files in a initramfs that contains nothing else, and the kernel knows how to read the data directly (without the hotplug firmware request userspace stuff)
Quoted 2 lines
Does using an initramfs really have some negative side, security related ? Would some kind of signed or encrypted initramfs be preferable there ?
adding an initramfs to a system that doesn't need it otherwise adds complications to the configure and boot process.
requireing modules when they weren't required before adds complication, and if/when the patch that's floating around to eliminate access to /dev/kmem is ever accepted, there are security advantages of running a kernel that doesn't have any support for run-time modifications (i.e. module loading).
I realize that many people want to make initramfs mandatory (with things like partition detection moved to userspace), but unless there is a standard initramfs that is shipped and maintained with the kernel to implement things like this (see the klibc discussion a few weeks ago) you are adding complications without much of a benifit to the user.
David Lang
Quoted 10 lines · David Lang
Quoted 8 lines · Sven Luther
Quoted 7 lines
Quoted 4 lines
Do you really need to bring up ipw2200 so early ? It is some kind of wireless
device, right ?if modules are not in use the device is initialized when the kernel starts up. this is before any userspace starts.
Quoted 7 lines
Quoted 2 lines
Well. but you could do the initialization at open time too, like the other case that was mentioned here, no ?
no, at least not in the current kernel. as was mentioned earlier in this thread the ipw2200 needs the firmware at initialization, but some others don't need it until open. I don't know if it's even possible to re-write the driver to do this.
Oh well, this doesn't explain why it is so, but i suppose you know what you speak about.
Quoted 25 lines
Quoted 14 lines
Quoted 8 lines
Quoted 3 lines
As for initramfs, you can just cat it behind the kernel, and it should work
just fine, or at least this is how it was supposed to work.yes, if I want to set one up.
other then this new requirement to have the ipw2200 driver as a module I have no reason to use one. normal userspace is good enough for me.
Well, ok.
The real question seems to be if we want to keep the firmware inside the driver or not.
Quoted 10 lines
If we want to remove it, then we put, not the module, but the firmware itself
with some basic userspace to load it on demand in the initramfs, and this is
reason enough to create an initramfs. The fact that the builtin device is initialized before the initramfs is loaded seems like a bug to me, since the
idea of the initramfs (well, one of them at least), was to initialize it early
enough for this kind of things.
Quoted 1 line
this isn't my understanding.
Indeed, in the initrd era, the ramdisk was initialized too late for this kind of stuff, but it was one of the features of the initramfs ramdisks to initialize it earlier, which made firmware loading possible.
Quoted 2 lines
my understanding is that the kernel fully initializes all built-in drivers, then loads userspace and starts running it.
Well, there is userspace and userspace.
Quoted 3 lines
that userspace can be on a device that it knows how to read, or it can be userspace on initramfs so that you can load additional modules to give you access to the hardware that you want to run on.
Yep, but initramfs is initialized ways earlier than normal userspace.
Quoted 2 lines
this is needed if your root drive is a SCSI drive and you have it's driver compiled as a module for example.
Ok.
Quoted 3 lines
this is needed if your root drive uses dm and you need to initialize the array (one advantage of md, from the user standpoint, is that it doesn't require this additional layer before use)
Ok.
Quoted 2 lines
however this is not soon enough to supply the firmware for devices like this.
Are you sure of this ? This is somewhat contrary to what i have heard, and it sure would make sense to be able to access the initramfs ramdisk much earlier.
Quoted 16 lines
Quoted 7 lines
If on the other side, it is more important to not have an initramfs (because
of security issues, or bootloader constraints or what not), then sure, there
is not much choice than putting the firmware in the driver or in the kernel directly.Quoted 8 lines
But again, the initramfs is just a memory space available at the end of the kernel, and there is no hardware-related constraint to access it which are in
any way different from having the firmware linked in together with the kernel,
so it is only a matter of organisation of code, as well as taking a decision
on the above, and to act accordyingly.
Quoted 7 lines
if the firmware needed for any drivers compiled in was appended to the kernel the same way that initramfs is, without requireing the other things needed to make initrmfs useable I think that would be reasonable (bundling them togeather as opposed to embedding the firmware in the kernel). it may even be possible to have the firmware as files in a initramfs that contains nothing else, and the kernel knows how to read the data directly (without the hotplug firmware request userspace stuff)
Indeed, and it seems to me that exactly this kind of use was indeed considered when the initramfs infrastructure was designed. Not sure about the latest bit concerning hotplug though.
Quoted 5 lines
Quoted 2 lines
Does using an initramfs really have some negative side, security related ? Would some kind of signed or encrypted initramfs be preferable there ?
adding an initramfs to a system that doesn't need it otherwise adds complications to the configure and boot process.
Yeah, well, if it is just cated at the end of the kernel, it would be rather minimal.
Quoted 4 lines
requireing modules when they weren't required before adds complication, and if/when the patch that's floating around to eliminate access to
/dev/kmemis ever accepted, there are security advantages of running a kernel that doesn't have any support for run-time modifications (i.e. module loading).
Indeed.
Quoted 5 lines
I realize that many people want to make initramfs mandatory (with things like partition detection moved to userspace), but unless there is a standard initramfs that is shipped and maintained with the kernel to implement things like this (see the klibc discussion a few weeks ago) you are adding complications without much of a benifit to the user.
Well, since debian moved to 2.6 kernels on powerpc, i have been using ramdisks, and let me assure you that from a distribution point of view, it is way more elegant and nice than the non-ramdisk solution.
Friendly,
Sven Luther
Quoted 7 lines · Sven Luther
Quoted 4 lines
no, at least not in the current kernel. as was mentioned earlier in this thread the ipw2200 needs the firmware at initialization, but some others don't need it until open. I don't know if it's even possible to re-write the driver to do this.
Oh well, this doesn't explain why it is so, but i suppose you know what you speak about.
I'm a user, not a developer. I know what is, but not nessasarily why, and I have no idea how bad it would be to change.
Quoted 26 lines
Quoted 11 lines
Quoted 10 lines
If we want to remove it, then we put, not the module, but the firmware itself
with some basic userspace to load it on demand in the initramfs, and this is
reason enough to create an initramfs. The fact that the builtin device is initialized before the initramfs is loaded seems like a bug to me, since the
idea of the initramfs (well, one of them at least), was to initialize it early
enough for this kind of things.Quoted 1 line
this isn't my understanding.
Indeed, in the initrd era, the ramdisk was initialized too late for this kind of stuff, but it was one of the features of the initramfs ramdisks to initialize it earlier, which made firmware loading possible.
Quoted 2 lines
my understanding is that the kernel fully initializes all built-in drivers, then loads userspace and starts running it.
Well, there is userspace and userspace.
Quoted 3 lines
that userspace can be on a device that it knows how to read, or it can be userspace on initramfs so that you can load additional modules to give you access to the hardware that you want to run on.
Quoted 7 lines
Yep, but initramfs is initialized ways earlier than normal userspace.
Quoted 2 lines
however this is not soon enough to supply the firmware for devices like this.
Are you sure of this ? This is somewhat contrary to what i have heard, and it sure would make sense to be able to access the initramfs ramdisk much earlier.
I could easily be wrong about this. can someone who really knows weigh in on this?
Quoted 24 lines
Quoted 16 lines
Quoted 7 lines
If on the other side, it is more important to not have an initramfs (because
of security issues, or bootloader constraints or what not), then sure, there
is not much choice than putting the firmware in the driver or in the kernel directly.Quoted 8 lines
But again, the initramfs is just a memory space available at the end of the kernel, and there is no hardware-related constraint to access it which are in
any way different from having the firmware linked in together with the kernel,
so it is only a matter of organisation of code, as well as taking a decision
on the above, and to act accordyingly.Quoted 7 lines
if the firmware needed for any drivers compiled in was appended to the kernel the same way that initramfs is, without requireing the other things needed to make initrmfs useable I think that would be reasonable (bundling them togeather as opposed to embedding the firmware in the kernel). it may even be possible to have the firmware as files in a initramfs that contains nothing else, and the kernel knows how to read the data directly (without the hotplug firmware request userspace stuff)
Quoted 3 lines
Indeed, and it seems to me that exactly this kind of use was indeed considered when the initramfs infrastructure was designed. Not sure about the latest bit concerning hotplug though.
this gets back to the question of how early this early userspace is
David Lang
Quoted 10 lines · David Lang
Quoted 9 lines
Yep, but initramfs is initialized ways earlier than normal userspace.
Quoted 2 lines
however this is not soon enough to supply the firmware for devices like this.
Are you sure of this ? This is somewhat contrary to what i have heard, and it
sure would make sense to be able to access the initramfs ramdisk much earlier.
Quoted 5 lines
I could easily be wrong about this. can someone who really knows weigh in on this?
From looking at my current boot logs it appears that initramfs is setup right
after the CPUs are brought up, so it should be available before any drivers are initialized and they should be able to get to their firmware in the initramfs as long as it's in the right path in the image. I don't have any drivers that require external firmware to test that theory out with though.
[4294668.249000] checking TSC synchronization across 2 CPUs: passed. [4294668.250000] Brought up 2 CPUs [4294668.277000] migration_cost=1000 [4294668.277000] checking if image is initramfs... it is [4294668.452000] Freeing initrd memory: 1358k freed [4294668.452000] NET: Registered protocol family 16 [4294668.453000] ACPI: bus type pci registered [4294668.453000] PCI: PCI BIOS revision 2.10 entry at 0xfd7e0, last bus=1
Jim.
Quoted 3 lines · David Lang
Quoted 1 line
initramfs is always in use.
not on my machines.
klibc can be build like:
cd linux-2.6.*
make headers_install INSTALL_HDR_PATH=/dev/shm/$$
cd ..
wget http://www.kernel.org/pub/linux/libs/klibc/Testing/klibc-1.4.29.tar.bz2
tar xfj klibc-*.tar.bz2
cd klibc-*
ln -sfvbn /dev/shm/$$ linux
make
Every other libc will do it as well, adjust the filelist as needed. Try this as CONFIG_INITRAMFS_SOURCE= :
# A simple initramfs
dir /dev 0755 0 0
nod /dev/console 0600 0 0 c 5 1
nod /dev/kmsg 0600 0 0 c 1 11
dir /sbin 0755 0 0
dir /lib 0755 0 0
dir /lib/firmware 0755 0 0
file /lib/firmware/name.ext /tmp/foofirmware.bin 0755 0 0
file /sbin/hotplug /home/olaf/kernel/hotplug.sh 0755 0 0
file /sbin/cat /home/olaf/kernel/klibc-1.4.29/usr/utils/static/cat 0755 0 0 file /sbin/mount /home/olaf/kernel/klibc-1.4.29/usr/utils/static/mount 0755 0 0 file /sbin/mkdir /home/olaf/kernel/klibc-1.4.29/usr/utils/static/mkdir 0755 0 0 file /sbin/mknod /home/olaf/kernel/klibc-1.4.29/usr/utils/static/mknod 0755 0 0 file /sbin/minips /home/olaf/kernel/klibc-1.4.29/usr/utils/static/minips 0755 0 0 file /sbin/umount /home/olaf/kernel/klibc-1.4.29/usr/utils/static/umount 0755 0 0 file /sbin/uname /home/olaf/kernel/klibc-1.4.29/usr/utils/static/uname 0755 0 0 file /sh /home/olaf/kernel/klibc-1.4.29/usr/dash/sh 0755 0 0 #optional:
file /init /home/olaf/kernel/init.sh 0755 0 0
Try this as hotplug.sh:
#!/sh
if test "$SEQNUM" -lt 42 ; then
echo "$1 SEQNUM=$SEQNUM ACTION=$ACTION DEVPATH=$DEVPATH" > /dev/kmsg fi
if test "$SEQNUM" = 1 ; then
mkdir /sys
mkdir /proc
mount -t sysfs sysfs /sys
mount -t proc proc /proc
set > /1.env
echo "$*" > /1.arg
cat /proc/cpuinfo > /cpuinfo
cat /sys/power/state > /state
umount /proc
umount /sys
fi
Try this as init.sh:
#!/sh
/sh
I assume you can adjust hotplug.sh for your ipw needs.
Quoted 10 lines · Oleg Verych
Quoted 9 lines · James Bottomley
Quoted 3 lines · Oleg Verych
request_firmware()is dead also.
YMMV, but three years, and there are still big chunks of binary in kernel. And please don't add new useless info _in_ it.I er don't think so.
Quoted 7 lines
Hell, what can be as easy as this:
,-
|modprobe $drv
|(dd </lib/firmware/$drv.bin>/dev/blobs && echo OK) || echo Error `-
where/dev/blobsis similar to/dev/portor even/dev/nullchar device. if synchronization is needed, addecho $drv >/dev/blobs, remove modprobe,
Please tell me how this is going to work, if we don't know which firmware (version) is needed before be actually initialize the device?
Signature · 1 line
Greetings Michael.
Quoted 3 lines · Sven Luther
Notice that mkinitrd-tools is dead, and will probably be removed from etch.
mkinitramfs-tools and yaird are the two currently used tools.
Yes ... I'm aware of that. That's why this is a reference implementation. initramfs should be easier ... I just don't have any initramfs systems at the moment, so I did what I had and could verify.
James
Hi James,
Quoted 6 lines
From: Jon Masters <[email protected]>
Right now, various kernel modules are being migrated over to use
request_firmwarein order to pull in binary firmware blobs from userland when the module is loaded. This makes sense.
Quoted 8 lines
However, there is right now little mechanism in place to automatically determine which binary firmware blobs must be included with a kernel in order to satisfy the prerequisites of these drivers. This affects vendors, but also regular users to a certain extent too.
The attached patch introduces
MODULE_FIRMWAREas a mechanism for advertising that a particular firmware file is to be loaded - it will then show up via modinfo and could be used e.g. when packaging a kernel.
this patch was debated, but we never came to a final conclusion. I am all for it. It is simple and can improve the current situation.
Quoted 1 line
Signed-off-by Jon Masters <[email protected]>
Regards
Marcel
Quoted 17 lines · Marcel Holtmann
Hi James,
Quoted 6 lines
From: Jon Masters <[email protected]>
Right now, various kernel modules are being migrated over to use
request_firmwarein order to pull in binary firmware blobs from userland when the module is loaded. This makes sense.Quoted 8 lines
However, there is right now little mechanism in place to automatically determine which binary firmware blobs must be included with a kernel in order to satisfy the prerequisites of these drivers. This affects vendors, but also regular users to a certain extent too.
The attached patch introduces
MODULE_FIRMWAREas a mechanism for advertising that a particular firmware file is to be loaded - it will then show up via modinfo and could be used e.g. when packaging a kernel.
Quoted 6 lines
this patch was debated, but we never came to a final conclusion. I am all for it. It is simple and can improve the current situation.
Quoted 1 line
Signed-off-by Jon Masters <[email protected]>Signed-off-by Marcel Holtmann <[email protected]>
Since your tree will depend on this change, you can add it there and add:
from me to the patch.
thanks,
greg k-h
Hi Greg,
Quoted 18 lines
Quoted 18 lines
Quoted 4 lines
Right now, various kernel modules are being migrated over to use
request_firmwarein order to pull in binary firmware blobs from userland when the module is loaded. This makes sense.Quoted 8 lines
However, there is right now little mechanism in place to automatically determine which binary firmware blobs must be included with a kernel in order to satisfy the prerequisites of these drivers. This affects vendors, but also regular users to a certain extent too.
The attached patch introduces
MODULE_FIRMWAREas a mechanism for advertising that a particular firmware file is to be loaded - it will then show up via modinfo and could be used e.g. when packaging a kernel.this patch was debated, but we never came to a final conclusion. I am all for it. It is simple and can improve the current situation.
Quoted 1 line
Signed-off-by Jon Masters <[email protected]>
Quoted 8 lines
Quoted 1 line
Signed-off-by Marcel Holtmann <[email protected]>Since your tree will depend on this change, you can add it there and add:
Signed-off-by Greg Kroah-Hartman <[email protected]>from me to the patch.
do we consider this 2.6.18 or 2.6.19 material? For the Bluetooth subsystem I can easily write a patch that makes use of it.
Regards
Marcel