Skip to contentSkip to conversation

firmware: moving drivers to request_firmware()

PATCH In review David Woodhouse 5 participants 34 messages Jun 5, 2008 – Jun 6, 2008

Subsystems (10)
lore
Share & tools
lore
#0 lore raw

cover firmware: moving drivers to request_firmware()

Ignoring the set of 'make fw->data const' patches which have mostly been already posted and are fairly trivial, this is the current content of the firmware tree at git.infradead.org/users/dwmw2/firmware-2.6.git

It starts by giving a way to build arbitrary firmware blobs into the kernel while letting request_firmware() find them there, and then sets about converting drivers to use request_firmware() instead of static data blobs, now that we can do that without forcing the users to actually satisfy the firmware requests from userspace.

Rather than including binary blobs in the git tree, we carry them as .ihex files -- which as an added bonus can have comments and licensing information appended after the EOF record.

For the USB drivers which actually _want_ Intel HEX files in their original form as individual records, we have a binary representation of such and a tool to convert the original .HEX files into .fw files in that form. And associated helpers in the kernel for validating and using them.

We also add a 'make firmware_install' target, which is intended to install the firmware blobs into /lib/firmware/ where the userspace loader to find them.

More patches to convert drivers to request_firmware() would be appreciated, if there are any people out there looking for something productive, but simple, to undertake...

Signature · 1 line
dwmw2
#1 lore raw

PATCH 1/18 firmware: allow firmware files to be built into kernel image

Some drivers have their own hacks to bypass the kernel's firmware loader and build their firmware into the kernel; this renders those unnecessary.

Other drivers don't use the firmware loader at all, because they always want the firmware to be available. This allows them to start using the firmware loader.

A third set of drivers already use the firmware loader, but can't be used without help from userspace, which sometimes requires an initrd. This allows them to work in a static kernel.

---
 drivers/base/firmware_class.c     |   33 +++++++++++++++++++++++++++++++--
 include/asm-generic/vmlinux.lds.h |    7 +++++++
 include/linux/firmware.h          |   21 +++++++++++++++++++++
 3 files changed, 59 insertions(+), 2 deletions(-)
drivers/base/firmware_class.c +31 -2
index 264b3a2..b0be1d1 100644
@@ -49,6 +49,14 @@ struct firmware_priv {
 	struct timer_list timeout;
 };
 
+#ifdef CONFIG_FW_LOADER
+extern struct builtin_fw __start_builtin_fw[];
+extern struct builtin_fw __end_builtin_fw[];
+#else /* Module case. Avoid ifdefs later; it'll all optimise out */
+static struct builtin_fw *__start_builtin_fw;
+static struct builtin_fw *__end_builtin_fw;
+#endif
+
 static void
 fw_load_abort(struct firmware_priv *fw_priv)
 {
@@ -391,13 +399,12 @@ _request_firmware(const struct firmware **firmware_p, const char *name,
 	struct device *f_dev;
 	struct firmware_priv *fw_priv;
 	struct firmware *firmware;
+	struct builtin_fw *builtin;
 	int retval;
 
 	if (!firmware_p)
 		return -EINVAL;
 
-	printk(KERN_INFO "firmware: requesting %s\n", name);
-
 	*firmware_p = firmware = kzalloc(sizeof(*firmware), GFP_KERNEL);
 	if (!firmware) {
 		printk(KERN_ERR "%s: kmalloc(struct firmware) failed\n",
@@ -406,6 +413,20 @@ _request_firmware(const struct firmware **firmware_p, const char *name,
 		goto out;
 	}
 
+	for (builtin = __start_builtin_fw; builtin != __end_builtin_fw;
+	     builtin++) {
+		if (strcmp(name, builtin->name))
+			continue;
+		printk(KERN_INFO "firmware: using built-in firmware %s\n",
+		       name);
+		firmware->size = builtin->size;
+		firmware->data = builtin->data;
+		return 0;
+	}
+
+	if (uevent)
+		printk(KERN_INFO "firmware: requesting %s\n", name);
+
 	retval = fw_setup_device(firmware, &f_dev, name, device, uevent);
 	if (retval)
 		goto error_kfree_fw;
@@ -473,8 +494,16 @@ request_firmware(const struct firmware **firmware_p, const char *name,
 void
 release_firmware(const struct firmware *fw)
 {
+	struct builtin_fw *builtin;
+
 	if (fw) {
+		for (builtin = __start_builtin_fw; builtin != __end_builtin_fw;
+		     builtin++) {
+			if (fw->data == builtin->data)
+				goto free_fw;
+		}
 		vfree(fw->data);
+	free_fw:
 		kfree(fw);
 	}
 }
include/asm-generic/vmlinux.lds.h +7 -0
index f054778..8d71a40 100644
@@ -86,6 +86,13 @@
 		VMLINUX_SYMBOL(__end_pci_fixups_resume) = .;		\
 	}								\
 									\
+	/* Built-in firmware blobs */					\
+	.builtin_fw        : AT(ADDR(.builtin_fw) - LOAD_OFFSET) {	\
+		VMLINUX_SYMBOL(__start_builtin_fw) = .;			\
+		*(.builtin_fw)						\
+		VMLINUX_SYMBOL(__end_builtin_fw) = .;			\
+	}								\
+									\
 	/* RapidIO route ops */						\
 	.rio_route        : AT(ADDR(.rio_route) - LOAD_OFFSET) {	\
 		VMLINUX_SYMBOL(__start_rio_route_ops) = .;		\
include/linux/firmware.h +21 -0
index 5e73f3f..785accf 100644
@@ -1,7 +1,10 @@
 #ifndef _LINUX_FIRMWARE_H
 #define _LINUX_FIRMWARE_H
+
 #include <linux/module.h>
 #include <linux/types.h>
+#include <linux/compiler.h>
+
 #define FIRMWARE_NAME_MAX 30 
 #define FW_ACTION_NOHOTPLUG 0
 #define FW_ACTION_HOTPLUG 1
@@ -13,6 +16,24 @@ struct firmware {
 
 struct device;
 
+struct builtin_fw {
+	char *name;
+	void *data;
+	unsigned long size;
+};
+
+/* We have to play tricks here much like stringify() to get the
+   __COUNTER__ macro to be expanded as we want it */
+#define __fw_concat1(x, y) x##y
+#define __fw_concat(x, y) __fw_concat1(x, y)
+
+#define DECLARE_BUILTIN_FIRMWARE(name, blob)				     \
+	DECLARE_BUILTIN_FIRMWARE_SIZE(name, &(blob), sizeof(blob))
+
+#define DECLARE_BUILTIN_FIRMWARE_SIZE(name, blob, size)			     \
+	static const struct builtin_fw __fw_concat(__builtin_fw,__COUNTER__) \
+	__used __section(.builtin_fw) = { name, blob, size }
+
 #if defined(CONFIG_FW_LOADER) || defined(CONFIG_FW_LOADER_MODULE)
 int request_firmware(const struct firmware **fw, const char *name,
 		     struct device *device);
Signature · 1 line
1.5.4.5
#2 lore raw

PATCH 2/18 firmware: Add CONFIG_BUILTIN_FIRMWARE option

In reply to David Woodhouse

This allows arbitrary firmware files to be included in the static kernel where the firmware loader can find them without requiring userspace to be alive.

(Updated and CONFIG_BUILTIN_FIRMWARE_DIR added with lots of help from Johannes Berg).

---
 Makefile             |    2 +-
 drivers/base/Kconfig |   33 +++++++++++++++++++
 firmware/Makefile    |   88 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 122 insertions(+), 1 deletions(-)
 create mode 100644 firmware/Makefile
Makefile +1 -1
index 8db70fe..56c243c 100644
@@ -450,7 +450,7 @@ scripts: scripts_basic include/config/auto.conf
 
 # Objects we will link into vmlinux / subdirs we need to visit
 init-y		:= init/
-drivers-y	:= drivers/ sound/
+drivers-y	:= drivers/ sound/ firmware/
 net-y		:= net/
 libs-y		:= lib/
 core-y		:= usr/
drivers/base/Kconfig +33 -0
index d7da109..f204ae7 100644
@@ -34,6 +34,39 @@ config FW_LOADER
 	  require userspace firmware loading support, but a module built outside
 	  the kernel tree does.
 
+config BUILTIN_FIRMWARE
+	string "Firmware blobs to build into the kernel binary"
+	depends on FW_LOADER
+	help
+	  This option allows firmware to be built into the kernel, for the
+	  cases where the user either cannot or doesn't want to provide it from
+	  userspace at runtime (for example, when the firmware in question is
+	  required for accessing the boot device, and the user doesn't want to
+	  use an initrd).
+
+	  This option is a string, and takes the (space-separated) names of the
+	  firmware files -- the same names which appear in MODULE_FIRMWARE()
+	  and request_firmware() in the source. These files should exist under
+	  the directory specified by the BUILTIN_FIRMWARE_DIR option, which is
+	  by default the firmware/ subdirectory of the kernel source tree.
+
+	  So, for example, you might set CONFIG_BUILTIN_FIRMWARE="usb8388.bin",
+	  copy the usb8388.bin file into the firmware/ directory, and build the
+	  kernel. Then any request_firmware("usb8388.bin") will be
+	  satisfied internally without needing to call out to userspace.
+
+config BUILTIN_FIRMWARE_DIR
+	string "Firmware blobs root directory"
+	depends on BUILTIN_FIRMWARE != ""
+	default "firmware"
+	help
+	  This option controls the directory in which the kernel build system
+	  looks for the firmware files listed in the BUILTIN_FIRMWARE option.
+	  The default is the firmware/ directory in the kernel source tree,
+	  but by changing this option you can point it elsewhere, such as
+	  the /lib/firmware/ directory or another separate directory
+	  containing firmware files.
+
 config DEBUG_DRIVER
 	bool "Driver Core verbose debug messages"
 	depends on DEBUG_KERNEL
firmware/Makefile new file +88 -0

88 lines not on this page · raw message

Signature · 1 line
1.5.4.5
#3 lore raw

PATCH 3/18 firmware: Add 'firmware_install' make target

In reply to David Woodhouse

This installs all the in-kernel-tree firmware into $(INSTALL_FW_PATH), which defaults to $(objtree)/usr/lib/firmware and is intended end up in /lib/firmware for udev to find the files.

This, in conjunction with the builtin-firmware support, makes it simple for drivers with associated firmware to move over to request_firmware() and give the user a choice of whether to have it built in to the kernel image or loaded separately from userspace.

As with kernel header installation for userspace, it intentionally pays no attention to configuration variables -- it installs _all_ available firmware blobs, unconditionally.

---
 Makefile                |    9 +++++++++
 scripts/Makefile.fwinst |   26 ++++++++++++++++++++++++++
 2 files changed, 35 insertions(+), 0 deletions(-)
 create mode 100644 scripts/Makefile.fwinst
Makefile +9 -0
index 56c243c..2e264f8 100644
@@ -995,6 +995,15 @@ depend dep:
 	@echo '*** Warning: make $@ is unnecessary now.'
 
 # ---------------------------------------------------------------------------
+# Firmware install
+INSTALL_FW_PATH=$(objtree)/usr/lib/firmware
+export INSTALL_FW_PATH
+
+PHONY += firmware_install
+firmware_install: FORCE
+	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.fwinst obj=firmware __fw_install
+
+# ---------------------------------------------------------------------------
 # Kernel headers
 INSTALL_HDR_PATH=$(objtree)/usr
 export INSTALL_HDR_PATH
scripts/Makefile.fwinst new file +26 -0
new file mode 100644
index 0000000..3cf3649
@@ -0,0 +1,26 @@
+# ==========================================================================
+# Installing firmware
+#
+# We don't include the .config, so all firmware files are in $(fw-shipped-)
+# rather than in $(fw-shipped-y) or $(fw-shipped-n).
+# ==========================================================================
+
+INSTALL := install
+
+include scripts/Kbuild.include 
+include $(srctree)/$(obj)/Makefile
+
+installed-fw := $(addprefix $(INSTALL_FW_PATH)/,$(fw-shipped-))
+installed-fw-dirs := $(sort $(dir $(installed-fw)))
+
+quiet_cmd_install = INSTALL $(subst $(srctree)/,,$@)
+      cmd_install = $(INSTALL) -m0644 $< $@
+
+$(installed-fw-dirs):
+	$(call cmd,mkdir)
+
+$(installed-fw): $(INSTALL_FW_PATH)/%: $(obj)/% | $(INSTALL_FW_PATH)/$$(dir %)/
+	$(call cmd,install)
+
+.PHONY: __fw_install
+__fw_install: $(installed-fw)
Signature · 1 line
1.5.4.5
#4 lore raw

PATCH 4/18 firmware: convert korg1212 driver to use firmware loader exclusively

In reply to David Woodhouse
---
 firmware/Makefile                      |    1 +
 firmware/WHENCE                        |   19 +
 firmware/korg/k1212.dsp.ihex           |  987 ++++++++++++++++++++++++++++++++
 sound/pci/Kconfig                      |    4 +-
 sound/pci/korg1212/korg1212-firmware.h |  987 --------------------------------
 sound/pci/korg1212/korg1212.c          |   18 -
 6 files changed, 1009 insertions(+), 1007 deletions(-)
 create mode 100644 firmware/WHENCE
 create mode 100644 firmware/korg/k1212.dsp.ihex
 delete mode 100644 sound/pci/korg1212/korg1212-firmware.h
firmware/Makefile +1 -0
index 9b5152a..1f2fb6d 100644
@@ -8,6 +8,7 @@ fwdir := $(subst ",,$(CONFIG_BUILTIN_FIRMWARE_DIR))
 fwabs := $(addprefix $(srctree)/,$(filter-out /%,$(fwdir)))$(filter /%,$(fwdir))
 
 fw-external-y := $(subst ",,$(CONFIG_BUILTIN_FIRMWARE))
+fw-shipped-$(CONFIG_SND_KORG1212_FIRMWARE_IN_KERNEL) += korg/k1212.dsp
 
 firmware-y    := $(fw-external-y) $(fw-shipped-y)
 firmware-dirs := $(sort $(patsubst %,$(objtree)/$(obj)/%/,$(dir $(firmware-y))))
firmware/WHENCE new file +19 -0
new file mode 100644
index 0000000..89ca95b
@@ -0,0 +1,19 @@
+             **********
+             * WHENCE *
+             **********
+
+This file attempts to document the origin and licensing information,
+if known, for each piece of firmware distributed for use with the Linux
+kernel.
+
+--------------------------------------------------------------------------
+
+Driver: korg1212 -- Korg 1212 IO audio device
+
+File: korg/k1212.dsp
+
+Licence: Unknown
+
+Found in alsa-firmware package in hex form; no licensing information.
+
+--------------------------------------------------------------------------
firmware/korg/k1212.dsp.ihex new file +987 -0

987 lines not on this page · raw message

sound/pci/Kconfig +2 -2
index 7e47421..66a2539 100644
@@ -734,7 +734,7 @@ config SND_INTEL8X0M
 config SND_KORG1212
 	tristate "Korg 1212 IO"
 	depends on SND
-	select FW_LOADER if !SND_KORG1212_FIRMWARE_IN_KERNEL
+	select FW_LOADER
 	select SND_PCM
 	help
 	  Say Y here to include support for Korg 1212IO soundcards.
@@ -745,7 +745,7 @@ config SND_KORG1212
 config SND_KORG1212_FIRMWARE_IN_KERNEL
 	bool "In-kernel firmware for Korg1212 driver"
 	depends on SND_KORG1212
-	default y
+	default n
 	help
 	  Say Y here to include the static firmware built in the kernel
 	  for the Korg1212 driver.  If you choose N here, you need to
sound/pci/korg1212/korg1212-firmware.h deleted +0 -987

987 lines not on this page · raw message

sound/pci/korg1212/korg1212.c +0 -18
index f4c85b5..4a44c0f 100644
@@ -260,14 +260,6 @@ enum MonitorModeSelector {
 #define COMMAND_ACK_DELAY   13         // number of RTC ticks to wait for an acknowledgement
                                        //    from the card after sending a command.
 
-#ifdef CONFIG_SND_KORG1212_FIRMWARE_IN_KERNEL
-#include "korg1212-firmware.h"
-static const struct firmware static_dsp_code = {
-	.data = (u8 *)dspCode,
-	.size = sizeof dspCode
-};
-#endif
-
 enum ClockSourceIndex {
    K1212_CLKIDX_AdatAt44_1K = 0,    // selects source as ADAT at 44.1 kHz
    K1212_CLKIDX_AdatAt48K,          // selects source as ADAT at 48 kHz
@@ -412,9 +404,7 @@ struct snd_korg1212 {
 MODULE_DESCRIPTION("korg1212");
 MODULE_LICENSE("GPL");
 MODULE_SUPPORTED_DEVICE("{{KORG,korg1212}}");
-#ifndef CONFIG_SND_KORG1212_FIRMWARE_IN_KERNEL
 MODULE_FIRMWARE("korg/k1212.dsp");
-#endif
 
 static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;     /* Index 0-MAX */
 static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR;	   /* ID for this card */
@@ -2348,9 +2338,6 @@ static int __devinit snd_korg1212_create(struct snd_card *card, struct pci_dev *
         korg1212->AdatTimeCodePhy = korg1212->sharedBufferPhy +
 		offsetof(struct KorgSharedBuffer, AdatTimeCode);
 
-#ifdef CONFIG_SND_KORG1212_FIRMWARE_IN_KERNEL
-	dsp_code = &static_dsp_code;
-#else
 	err = request_firmware(&dsp_code, "korg/k1212.dsp", &pci->dev);
 	if (err < 0) {
 		release_firmware(dsp_code);
@@ -2358,15 +2345,12 @@ static int __devinit snd_korg1212_create(struct snd_card *card, struct pci_dev *
 		snd_korg1212_free(korg1212);
 		return err;
 	}
-#endif
 
 	if (snd_dma_alloc_pages(SNDRV_DMA_TYPE_DEV, snd_dma_pci_data(pci),
 				dsp_code->size, &korg1212->dma_dsp) < 0) {
 		snd_printk(KERN_ERR "korg1212: cannot allocate dsp code memory (%zd bytes)\n", dsp_code->size);
                 snd_korg1212_free(korg1212);
-#ifndef CONFIG_SND_KORG1212_FIRMWARE_IN_KERNEL
 		release_firmware(dsp_code);
-#endif
                 return -ENOMEM;
         }
 
@@ -2376,9 +2360,7 @@ static int __devinit snd_korg1212_create(struct snd_card *card, struct pci_dev *
 
 	memcpy(korg1212->dma_dsp.area, dsp_code->data, dsp_code->size);
 
-#ifndef CONFIG_SND_KORG1212_FIRMWARE_IN_KERNEL
 	release_firmware(dsp_code);
-#endif
 
 	rc = snd_korg1212_Send1212Command(korg1212, K1212_DB_RebootCard, 0, 0, 0, 0);
 
Signature · 1 line
1.5.4.5
#5 lore raw

PATCH 5/18 firmware: convert maestro3 driver to use firmware loader exclusively

In reply to David Woodhouse
---
 firmware/Makefile                          |    2 +
 firmware/WHENCE                            |   14 +++
 firmware/ess/maestro3_assp_kernel.fw.ihex  |  120 +++++++++++++++++++
 firmware/ess/maestro3_assp_minisrc.fw.ihex |   51 ++++++++
 sound/pci/Kconfig                          |    4 +-
 sound/pci/maestro3.c                       |  171 ----------------------------
 6 files changed, 189 insertions(+), 173 deletions(-)
 create mode 100644 firmware/ess/maestro3_assp_kernel.fw.ihex
 create mode 100644 firmware/ess/maestro3_assp_minisrc.fw.ihex
firmware/Makefile +2 -0
index 1f2fb6d..d94d2fc 100644
@@ -9,6 +9,8 @@ fwabs := $(addprefix $(srctree)/,$(filter-out /%,$(fwdir)))$(filter /%,$(fwdir))
 
 fw-external-y := $(subst ",,$(CONFIG_BUILTIN_FIRMWARE))
 fw-shipped-$(CONFIG_SND_KORG1212_FIRMWARE_IN_KERNEL) += korg/k1212.dsp
+fw-shipped-$(CONFIG_SND_MAESTRO3_FIRMWARE_IN_KERNEL) += \
+		ess/maestro3_assp_kernel.fw ess/maestro3_assp_minisrc.fw
 
 firmware-y    := $(fw-external-y) $(fw-shipped-y)
 firmware-dirs := $(sort $(patsubst %,$(objtree)/$(obj)/%/,$(dir $(firmware-y))))
firmware/WHENCE +14 -0
index 89ca95b..c08dbc8 100644
@@ -17,3 +17,17 @@ Licence: Unknown
 Found in alsa-firmware package in hex form; no licensing information.
 
 --------------------------------------------------------------------------
+
+Driver: maestro3 -- ESS Allegro Maestro3 audio device
+
+File: ess/maestro3_assp_kernel.fw
+File: ess/maestro3_assp_minisrc.fw
+
+Licence: Unknown
+
+Found in alsa-firmware package in hex form with a comment claiming to
+be GPLv2+, but without source -- and with another comment saying "ESS
+drops binary dsp code images on our heads, but we don't get to see
+specs on the dsp."
+
+--------------------------------------------------------------------------
firmware/ess/maestro3_assp_kernel.fw.ihex new file +120 -0

120 lines not on this page · raw message

firmware/ess/maestro3_assp_minisrc.fw.ihex new file +51 -0

51 lines not on this page · raw message

sound/pci/Kconfig +2 -2
index 66a2539..7a039e3 100644
@@ -754,7 +754,7 @@ config SND_KORG1212_FIRMWARE_IN_KERNEL
 config SND_MAESTRO3
 	tristate "ESS Allegro/Maestro3"
 	depends on SND
-	select FW_LOADER if !SND_MAESTRO3_FIRMWARE_IN_KERNEL
+	select FW_LOADER
 	select SND_AC97_CODEC
 	help
 	  Say Y here to include support for soundcards based on ESS Maestro 3
@@ -766,7 +766,7 @@ config SND_MAESTRO3
 config SND_MAESTRO3_FIRMWARE_IN_KERNEL
 	bool "In-kernel firmware for Maestro3 driver"
 	depends on SND_MAESTRO3
-	default y
+	default n
 	help
 	  Say Y here to include the static firmware built in the kernel
 	  for the Maestro3 driver.  If you choose N here, you need to
sound/pci/maestro3.c +0 -171

218 lines not on this page · raw message

Signature · 1 line
1.5.4.5
#6 lore raw

PATCH 6/18 firmware: convert ymfpci driver to use firmware loader exclusively

In reply to David Woodhouse
---
 firmware/Makefile                 |    2 +
 firmware/WHENCE                   |   13 +
 firmware/yamaha/ds1_ctrl.fw.ihex  |  769 ++++++++++++++++++
 firmware/yamaha/ds1_dsp.fw.ihex   |    9 +
 firmware/yamaha/ds1e_ctrl.fw.ihex |  769 ++++++++++++++++++
 sound/pci/Kconfig                 |    4 +-
 sound/pci/ymfpci/ymfpci_image.h   | 1565 -------------------------------------
 sound/pci/ymfpci/ymfpci_main.c    |   63 --
 8 files changed, 1564 insertions(+), 1630 deletions(-)
 create mode 100644 firmware/yamaha/ds1_ctrl.fw.ihex
 create mode 100644 firmware/yamaha/ds1_dsp.fw.ihex
 create mode 100644 firmware/yamaha/ds1e_ctrl.fw.ihex
 delete mode 100644 sound/pci/ymfpci/ymfpci_image.h
firmware/Makefile +2 -0
index d94d2fc..6796bf5 100644
@@ -11,6 +11,8 @@ fw-external-y := $(subst ",,$(CONFIG_BUILTIN_FIRMWARE))
 fw-shipped-$(CONFIG_SND_KORG1212_FIRMWARE_IN_KERNEL) += korg/k1212.dsp
 fw-shipped-$(CONFIG_SND_MAESTRO3_FIRMWARE_IN_KERNEL) += \
 		ess/maestro3_assp_kernel.fw ess/maestro3_assp_minisrc.fw
+fw-shipped-$(CONFIG_SND_YMFPCI_FIRMWARE_IN_KERNEL) += yamaha/ds1_ctrl.fw \
+		yamaha/ds1_dsp.fw yamaha/ds1e_ctrl.fw
 
 firmware-y    := $(fw-external-y) $(fw-shipped-y)
 firmware-dirs := $(sort $(patsubst %,$(objtree)/$(obj)/%/,$(dir $(firmware-y))))
firmware/WHENCE +13 -0
index c08dbc8..ae40fb1 100644
@@ -31,3 +31,16 @@ drops binary dsp code images on our heads, but we don't get to see
 specs on the dsp."
 
 --------------------------------------------------------------------------
+
+Driver: ymfpci -- Yamaha YMF724/740/744/754 audio devices
+
+File: yamaha/ds1_ctrl.fw
+File: yamaha/ds1_dsp.fw
+File: yamaha/ds1e_ctrl.fw
+
+Licence: Unknown
+
+Found alsa-firmware package in hex form, with the following comment:
+   Copyright (c) 1997-1999 Yamaha Corporation. All Rights Reserved.
+
+--------------------------------------------------------------------------
firmware/yamaha/ds1_ctrl.fw.ihex new file +769 -0

769 lines not on this page · raw message

firmware/yamaha/ds1_dsp.fw.ihex new file +9 -0
new file mode 100644
index 0000000..acb0ba4
@@ -0,0 +1,9 @@
+:1000000081000000A40100000A0000002F00000091
+:1000100053020800170380017B4000003F8400006A
+:100020003C4801003C9401003CD805003C1C000009
+:100030007BC000003F0C05003C5021010000000087
+:1000400000000000000000000000000000000000B0
+:1000500000000000000000000000000000000000A0
+:100060000000000000000000000000000000000090
+:100070000000000000000000000000000000000080
+:00000001FF
firmware/yamaha/ds1e_ctrl.fw.ihex new file +769 -0

769 lines not on this page · raw message

sound/pci/Kconfig +2 -2
index 7a039e3..86ad1ad 100644
@@ -934,7 +934,7 @@ config SND_VX222
 config SND_YMFPCI
 	tristate "Yamaha YMF724/740/744/754"
 	depends on SND
-	select FW_LOADER if !SND_YMFPCI_FIRMWARE_IN_KERNEL
+	select FW_LOADER
 	select SND_OPL3_LIB
 	select SND_MPU401_UART
 	select SND_AC97_CODEC
@@ -948,7 +948,7 @@ config SND_YMFPCI
 config SND_YMFPCI_FIRMWARE_IN_KERNEL
 	bool "In-kernel firmware for YMFPCI driver"
 	depends on SND_YMFPCI
-	default y
+	default n
 	help
 	  Say Y here to include the static firmware built in the kernel
 	  for the YMFPCI driver.  If you choose N here, you need to
sound/pci/ymfpci/ymfpci_image.h deleted +0 -1565

1,413 lines not on this page · raw message

sound/pci/ymfpci/ymfpci_main.c +0 -63

63 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
1.5.4.5

Part of the diff is not shown here. View the raw message.

#7 lore raw

PATCH 7/18 smctr: use request_firmware()

In reply to David Woodhouse
---
 drivers/net/tokenring/Kconfig          |    9 +
 drivers/net/tokenring/smctr.c          |   56 ++-
 drivers/net/tokenring/smctr.h          |    2 -
 drivers/net/tokenring/smctr_firmware.h |  978 --------------------------------
 firmware/Makefile                      |    1 +
 firmware/WHENCE                        |   13 +
 firmware/tr_smctr.bin.ihex             |  477 ++++++++++++++++
 7 files changed, 535 insertions(+), 1001 deletions(-)
 delete mode 100644 drivers/net/tokenring/smctr_firmware.h
 create mode 100644 firmware/tr_smctr.bin.ihex
drivers/net/tokenring/Kconfig +9 -0
index e6b2e06..8274ffa 100644
@@ -168,6 +168,7 @@ config MADGEMC
 
 config SMCTR
 	tristate "SMC ISA/MCA adapter support"
+	select FW_LOADER
 	depends on (ISA || MCA_LEGACY) && (BROKEN || !64BIT)
 	---help---
 	  This is support for the ISA and MCA SMC Token Ring cards,
@@ -182,4 +183,12 @@ config SMCTR
 	  To compile this driver as a module, choose M here: the module will be
 	  called smctr.
 
+config SMCTR_FIRMWARE
+	bool "SMC ISA/MCA adapter firmware"
+	depends on SMCTR
+	---help---
+	  Build firmware for SMCTR into the kernel instead of obtaining it
+	  through udev. Say 'N'.
+	  
+       
 endif # TR
drivers/net/tokenring/smctr.c +35 -21

135 lines not on this page · raw message

drivers/net/tokenring/smctr.h +0 -2
index 88dfa2e..52df7dd 100644
@@ -1042,8 +1042,6 @@ typedef struct net_local {
         __u16            functional_address[2];
         __u16            bitwise_group_address[2];
 
-	const __u8       *ptr_ucode;
-
 	__u8		cleanup;
 
 	struct sk_buff_head SendSkbQueue;
drivers/net/tokenring/smctr_firmware.h deleted +0 -978

978 lines not on this page · raw message

firmware/Makefile +1 -0
index 6796bf5..45a4a81 100644
@@ -13,6 +13,7 @@ fw-shipped-$(CONFIG_SND_MAESTRO3_FIRMWARE_IN_KERNEL) += \
 		ess/maestro3_assp_kernel.fw ess/maestro3_assp_minisrc.fw
 fw-shipped-$(CONFIG_SND_YMFPCI_FIRMWARE_IN_KERNEL) += yamaha/ds1_ctrl.fw \
 		yamaha/ds1_dsp.fw yamaha/ds1e_ctrl.fw
+fw-shipped-$(CONFIG_SMCTR_FIRMWARE) += tr_smctr.bin
 
 firmware-y    := $(fw-external-y) $(fw-shipped-y)
 firmware-dirs := $(sort $(patsubst %,$(objtree)/$(obj)/%/,$(dir $(firmware-y))))
firmware/WHENCE +13 -0
index ae40fb1..a8fc4b6 100644
@@ -44,3 +44,16 @@ Found alsa-firmware package in hex form, with the following comment:
    Copyright (c) 1997-1999 Yamaha Corporation. All Rights Reserved.
 
 --------------------------------------------------------------------------
+
+Driver: smctr -- SMC ISA/MCA Token Ring adapter
+
+File: tr_smctr.bin
+Info: MCT.BIN v6.3C1 03/01/95
+
+Original licence info:
+
+ * This firmware is licensed to you strictly for use in conjunction
+ * with the use of SMC TokenRing adapters. There is no waranty
+ * expressed or implied about its fitness for any purpose.
+
+--------------------------------------------------------------------------
firmware/tr_smctr.bin.ihex new file +477 -0

477 lines not on this page · raw message

Signature · 1 line
1.5.4.5
#8 lore raw

PATCH 8/18 kaweth: use request_firmware()

In reply to David Woodhouse
---
 drivers/net/usb/Kconfig                   |    9 +
 drivers/net/usb/kaweth.c                  |   43 ++-
 drivers/net/usb/kawethfw.h                |  557 -----------------------------
 firmware/Makefile                         |    3 +
 firmware/WHENCE                           |   13 +
 firmware/kaweth/new_code.bin.ihex         |  206 +++++++++++
 firmware/kaweth/new_code_fix.bin.ihex     |   40 ++
 firmware/kaweth/trigger_code.bin.ihex     |   13 +
 firmware/kaweth/trigger_code_fix.bin.ihex |    3 +
 9 files changed, 313 insertions(+), 574 deletions(-)
 delete mode 100644 drivers/net/usb/kawethfw.h
 create mode 100644 firmware/kaweth/new_code.bin.ihex
 create mode 100644 firmware/kaweth/new_code_fix.bin.ihex
 create mode 100644 firmware/kaweth/trigger_code.bin.ihex
 create mode 100644 firmware/kaweth/trigger_code_fix.bin.ihex
drivers/net/usb/Kconfig +9 -0
index 0604f3f..e5617a2 100644
@@ -29,6 +29,7 @@ config USB_CATC
 
 config USB_KAWETH
 	tristate "USB KLSI KL5USB101-based ethernet device support"
+	select FW_LOADER
 	---help---
 	  Say Y here if you want to use one of the following 10Mbps only
 	  USB Ethernet adapters based on the KLSI KL5KUSB101B chipset:
@@ -66,6 +67,14 @@ config USB_KAWETH
 	  To compile this driver as a module, choose M here: the
 	  module will be called kaweth.
 
+config USB_KAWETH_FIRMWARE
+	bool "Include firmware for USB KLSI KL5USB101-based ethernet devices"
+	depends on USB_KAWETH
+	---help---
+	  This includes firmware for the "kaweth" devices in the kernel
+	  image directly, so that it doesn't need to be provided by
+	  the firmware loader. Say 'N'.
+
 config USB_PEGASUS
 	tristate "USB Pegasus/Pegasus-II based ethernet device support"
 	select MII
drivers/net/usb/kaweth.c +26 -17

102 lines not on this page · raw message

drivers/net/usb/kawethfw.h deleted +0 -557

557 lines not on this page · raw message

firmware/Makefile +3 -0
index 45a4a81..28b6ed5 100644
@@ -14,6 +14,9 @@ fw-shipped-$(CONFIG_SND_MAESTRO3_FIRMWARE_IN_KERNEL) += \
 fw-shipped-$(CONFIG_SND_YMFPCI_FIRMWARE_IN_KERNEL) += yamaha/ds1_ctrl.fw \
 		yamaha/ds1_dsp.fw yamaha/ds1e_ctrl.fw
 fw-shipped-$(CONFIG_SMCTR_FIRMWARE) += tr_smctr.bin
+fw-shipped-$(CONFIG_USB_KAWETH_FIRMWARE) += kaweth/new_code.bin \
+		kaweth/new_code_fix.bin kaweth/trigger_code.bin \
+		kaweth/trigger_code_fix.bin
 
 firmware-y    := $(fw-external-y) $(fw-shipped-y)
 firmware-dirs := $(sort $(patsubst %,$(objtree)/$(obj)/%/,$(dir $(firmware-y))))
firmware/WHENCE +13 -0
index a8fc4b6..1373f52 100644
@@ -57,3 +57,16 @@ Original licence info:
  * expressed or implied about its fitness for any purpose.
 
 --------------------------------------------------------------------------
+
+Driver: kaweth -- USB KLSI KL5USB101-based Ethernet device
+
+File: kaweth/new_code.bin
+File: kaweth/new_code_fix.bin
+File: kaweth/trigger_code.bin
+File: kaweth/trigger_code_fix.bin
+
+Licence: Unknown
+
+Found in hex form in the kernel source.
+
+--------------------------------------------------------------------------
firmware/kaweth/new_code.bin.ihex new file +206 -0

206 lines not on this page · raw message

firmware/kaweth/new_code_fix.bin.ihex new file +40 -0
new file mode 100644
index 0000000..fb35d3d
@@ -0,0 +1,40 @@
+:10000000B6C3AABBCCDD0200080028002C003400D7
+:100010003C0040004800540058005E006400680046
+:100020006E006C00720076007C00800086008A0002
+:100030009000940098009E00A600AA00B000B400B2
+:10004000B800C000C600CA00D000D400D800E0004C
+:10005000DE00E800F000FC0004010A0118012201A2
+:1000600028013A013E017E0198019C01A201AC01E8
+:10007000B201BA01C001C801D001D601F401FC01EE
+:10008000080216021A0222022A022E023E0244022C
+:100090004A025002640262026C02720286028C0200
+:1000A00090029E02BC02D002D802DC02E002E8020A
+:1000B000E602F402FE0204030C0328037C0390030F
+:1000C00094039C03A203C003D003D403EE03FA03FA
+:1000D000FE032E0432043C0440044E0476047C04E7
+:1000E00084048A048E04A604B004B804BE04D204B6
+:1000F000DC04EE0410051A0524052A05360534052E
+:100100003C05420564056A056E058605220626063D
+:100110002C06300642064A064E06560654065A0675
+:1001200060066606E806EE06F406160726072C07A4
+:10013000320736073A073E07520756075A07640741
+:1001400076077A07800784078A079E07A207DA07DF
+:10015000DE07E207E607EA07EE07F207F6070E08F2
+:10016000160818081A081C081E0820082208240867
+:10017000260828082A082C082E0832083A084608BB
+:100180004E0854085E0878087E08820886088C08A5
+:10019000900898089E08A408AA08B008AE08B408F9
+:1001A000BE08C408C208CA08C808D408E408E80899
+:1001B000F608140912091A092009260924092A092E
+:1001C0003E094C0956097009740978097E097C09B1
+:1001D000820998099C09A009A609B809DC09E8095F
+:1001E000EC09FC09120A180A1E0A420A460A4E0ABB
+:1001F000540A5A0A5E0A680A6E0A720A780A760A6D
+:100200007C0A800A840A940AA40AB80ABE0ABC0AB4
+:10021000C20AC80AC60ACC0AD00AD40AD80ADC0A1A
+:10022000E00AF20AF60AFA0A140B1A0B200B1E0B4C
+:10023000260B2E0B2C0B360B3C0B420B400B4A0BA8
+:10024000AA0BB00BB60BC00BC80BDA0BE80BEC0B10
+:10025000FA0B4A0C540C620C660C960C9A0CA00C0F
+:0E026000A60CA40CAC0CB20CB00CC00C000030
+:00000001FF
firmware/kaweth/trigger_code.bin.ihex new file +13 -0
new file mode 100644
index 0000000..c3e1658
@@ -0,0 +1,13 @@
+:10000000B6C3AABBCCDDC807A000F0075E0006009F
+:10001000F0070A000800F00900000200E7073600B8
+:100020000000F00700000400E70750C310C0F0090B
+:100030000EC00000E78701000EC097CFD70900C0AF
+:100040001702C807A000E71750C310C030D804003B
+:10005000305C08000400B0C00600C805E70500C019
+:10006000C0DF97CF49AFC007000060AF4AAF000CB8
+:100070000C0040D2001C0C0040D230000800F007F9
+:1000800000000400F0078600060067CF270C02007E
+:100090000000270C00000EC049AF64AFC00700008D
+:1000A0004BAF4AAF5ACF0000000000000000000034
+:0600B000940005000000B1
+:00000001FF
firmware/kaweth/trigger_code_fix.bin.ihex new file +3 -0
new file mode 100644
index 0000000..7712f73
@@ -0,0 +1,3 @@
+:10000000B6C3AABBCCDD0200060018003E0080008B
+:060010009800AA000000A8
+:00000001FF
Signature · 1 line
1.5.4.5
#9 lore raw

PATCH 9/18 ttusb-budget: use request_firmware()

In reply to David Woodhouse
---
 drivers/media/dvb/ttusb-budget/Kconfig             |    9 +
 drivers/media/dvb/ttusb-budget/dvb-ttusb-budget.c  |   16 +-
 .../media/dvb/ttusb-budget/dvb-ttusb-dspbootcode.h | 1644 --------------------
 firmware/Makefile                                  |    1 +
 firmware/WHENCE                                    |   10 +
 firmware/ttusb-budget/dspbootcode.bin.ihex         |  820 ++++++++++
 6 files changed, 852 insertions(+), 1648 deletions(-)
 delete mode 100644 drivers/media/dvb/ttusb-budget/dvb-ttusb-dspbootcode.h
 create mode 100644 firmware/ttusb-budget/dspbootcode.bin.ihex
drivers/media/dvb/ttusb-budget/Kconfig +9 -0
index f546bcc..7e6b141 100644
@@ -1,5 +1,6 @@
 config DVB_TTUSB_BUDGET
 	tristate "Technotrend/Hauppauge Nova-USB devices"
+	select FW_LOADER
 	depends on DVB_CORE && USB && I2C
 	select DVB_CX22700 if !DVB_FE_CUSTOMISE
 	select DVB_TDA1004X if !DVB_FE_CUSTOMISE
@@ -16,3 +17,11 @@ config DVB_TTUSB_BUDGET
 	  an external software decoder to watch TV.
 
 	  Say Y if you own such a device and want to use it.
+
+config DVB_TTUSB_BUDGET_FIRMWARE
+	bool "Firmware for Technotrend/Hauppauge Nova-USB devices"
+	depends on DVB_TTUSB_BUDGET
+	help
+	  Include DSP bootcode for the device within the kernel image,
+	  instead of having it provided from userspace by the firmware
+	  loader. Say 'N'.
drivers/media/dvb/ttusb-budget/dvb-ttusb-budget.c +12 -4
index 732ce4d..2758350 100644
@@ -19,6 +19,7 @@
 #include <linux/errno.h>
 #include <linux/jiffies.h>
 #include <linux/mutex.h>
+#include <linux/firmware.h>
 
 #include "dvb_frontend.h"
 #include "dmxdev.h"
@@ -285,13 +286,19 @@ static int master_xfer(struct i2c_adapter* adapter, struct i2c_msg *msg, int num
 	return i;
 }
 
-#include "dvb-ttusb-dspbootcode.h"
-
 static int ttusb_boot_dsp(struct ttusb *ttusb)
 {
+	const struct firmware *fw;
 	int i, err;
 	u8 b[40];
 
+	err = request_firmware(&fw, "ttusb-budget/dspbootcode.bin",
+			       &ttusb->dev->dev);
+	if (err) {
+		printk(KERN_ERR "ttusb-budget: failed to request firmware\n");
+		return err;
+	}
+
 	/* BootBlock */
 	b[0] = 0xaa;
 	b[2] = 0x13;
@@ -299,8 +306,8 @@ static int ttusb_boot_dsp(struct ttusb *ttusb)
 
 	/* upload dsp code in 32 byte steps (36 didn't work for me ...) */
 	/* 32 is max packet size, no messages should be splitted. */
-	for (i = 0; i < sizeof(dsp_bootcode); i += 28) {
-		memcpy(&b[4], &dsp_bootcode[i], 28);
+	for (i = 0; i < fw->size; i += 28) {
+		memcpy(&b[4], &fw->data[i], 28);
 
 		b[1] = ++ttusb->c;
 
@@ -1820,3 +1827,4 @@ module_exit(ttusb_exit);
 MODULE_AUTHOR("Holger Waechtler <[email protected]>");
 MODULE_DESCRIPTION("TTUSB DVB Driver");
 MODULE_LICENSE("GPL");
+MODULE_FIRMWARE("ttusb-budget/dspbootcode.bin");
drivers/media/dvb/ttusb-budget/dvb-ttusb-dspbootcode.h deleted +0 -1644

1,644 lines not on this page · raw message

firmware/Makefile +1 -0
index 28b6ed5..4e19ac1 100644
@@ -17,6 +17,7 @@ fw-shipped-$(CONFIG_SMCTR_FIRMWARE) += tr_smctr.bin
 fw-shipped-$(CONFIG_USB_KAWETH_FIRMWARE) += kaweth/new_code.bin \
 		kaweth/new_code_fix.bin kaweth/trigger_code.bin \
 		kaweth/trigger_code_fix.bin
+fw-shipped-$(CONFIG_DVB_TTUSB_BUDGET_FIRMWARE) += ttusb-budget/dspbootcode.bin
 
 firmware-y    := $(fw-external-y) $(fw-shipped-y)
 firmware-dirs := $(sort $(patsubst %,$(objtree)/$(obj)/%/,$(dir $(firmware-y))))
firmware/WHENCE +10 -0
index 1373f52..9c2a1eb 100644
@@ -70,3 +70,13 @@ Licence: Unknown
 Found in hex form in the kernel source.
 
 --------------------------------------------------------------------------
+
+Driver: ttusb-budget -- Technotrend/Hauppauge Nova-USB devices
+
+File: ttusb-budget/dspbootcode.bin
+
+Licence: Unknown
+
+Found in hex form in the kernel source.
+
+--------------------------------------------------------------------------
firmware/ttusb-budget/dspbootcode.bin.ihex new file +820 -0

820 lines not on this page · raw message

Signature · 1 line
1.5.4.5
#10 lore raw

PATCH 10/18 ihex.h: binary representation of ihex records

In reply to David Woodhouse

Some devices need their firmware as a set of {address, len, data...} records in some specific order rather than a simple blob.

The normal way of doing this kind of thing is 'ihex', which is a text format and not entirely suitable for use in the kernel.

This provides a binary representation which is very similar, but much more compact -- and a helper routine to skip to the next record, because the alignment constraints mean that everybody will screw it up for themselves otherwise.

Also a helper function which can verify that a 'struct firmware' contains a valid set of ihex records, and that following them won't run off the end of the loaded data.

---
 include/linux/ihex.h |   44 ++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 44 insertions(+), 0 deletions(-)
 create mode 100644 include/linux/ihex.h
include/linux/ihex.h new file +44 -0
new file mode 100644
index 0000000..3da4855
@@ -0,0 +1,44 @@
+/*
+ * Compact binary representation of ihex records. Some devices need their
+ * firmware loaded in strange orders rather than a single big blob, but 
+ * actually parsing ihex-as-text within the kernel seems silly. Thus,...
+ */
+
+#ifndef __LINUX_IHEX_H__
+#define __LINUX_IHEX_H__
+
+#include <linux/types.h>
+#include <linux/firmware.h>
+
+struct ihex_binrec {
+	__be32 addr;
+	uint8_t len;
+	uint8_t data[0];
+} __attribute__((aligned(4)));
+
+/* Find the next record, taking into account the 4-byte alignment */
+static inline const struct ihex_binrec *
+ihex_next_binrec(const struct ihex_binrec *rec)
+{
+	int next = ((rec->len + 4) & ~3) - 1;
+	rec = (void *)&rec->data[next];
+
+	return rec->len ? rec : NULL;
+}
+
+/* Check that ihex_next_binrec() won't take us off the end of the image... */
+static inline int ihex_validate_fw(const struct firmware *fw)
+{
+	const struct ihex_binrec *rec, *end;
+
+	rec = (void *)fw->data;
+	end = (void *)fw->data + fw->size - 4;
+
+	while (rec) {
+		if (rec >= end)
+			return -EINVAL;
+		rec = ihex_next_binrec(rec);
+	}
+	return 0;
+}
+#endif /* __LINUX_IHEX_H__ */
Signature · 1 line
1.5.4.5
#13 lore raw

PATCH 11/18 ihex: request_ihex_firmware() function to load and validate firmware

In reply to David Woodhouse

Provide a helper to load the file and validate it in one call, to simplify error handling in the drivers which are going to use it.

---
 include/linux/ihex.h |   24 ++++++++++++++++++++++++
 1 files changed, 24 insertions(+), 0 deletions(-)
include/linux/ihex.h +24 -0
index 3da4855..2f3d8b4 100644
@@ -9,6 +9,7 @@
 
 #include <linux/types.h>
 #include <linux/firmware.h>
+#include <linux/device.h>
 
 struct ihex_binrec {
 	__be32 addr;
@@ -41,4 +42,27 @@ static inline int ihex_validate_fw(const struct firmware *fw)
 	}
 	return 0;
 }
+
+/* Request firmware and validate it so that we can trust we won't
+ * run off the end while reading records... */
+static inline int request_ihex_firmware(const struct firmware **fw,
+					const char *fw_name,
+					struct device *dev)
+{
+	const struct firmware *lfw;
+	int ret;
+
+	ret = request_firmware(&lfw, fw_name, dev);
+	if (ret)
+		return ret;
+	ret = ihex_validate_fw(lfw);
+	if (ret) {
+		dev_err(dev, "Firmware \"%s\" not valid IHEX records\n",
+			fw_name);
+		release_firmware(lfw);
+		return ret;
+	}
+	*fw = lfw;
+	return 0;
+}
 #endif /* __LINUX_IHEX_H__ */
Signature · 1 line
1.5.4.5
#14 lore raw

PATCH 12/18 ihex: add ihex2fw tool for converting HEX files into firmware images

In reply to David Woodhouse

Not the straight conversion to binary which objcopy can do for us, but actually representing each record with its original {addr, length}, because some drivers need that information preserved.

Fix up 'firmware_install' to be able to build $(hostprogs-y) too.

---
 firmware/Makefile       |   14 +++
 firmware/ihex2fw.c      |  238 +++++++++++++++++++++++++++++++++++++++++++++++
 scripts/Makefile.fwinst |    7 +-
 3 files changed, 258 insertions(+), 1 deletions(-)
 create mode 100644 firmware/ihex2fw.c
firmware/Makefile +14 -0
index 4e19ac1..2583aef 100644
@@ -28,6 +28,9 @@ quiet_cmd_mkdir = MKDIR   $(patsubst $(objtree)/%,%,$@)
 quiet_cmd_ihex  = IHEX    $@
       cmd_ihex  = $(OBJCOPY) -Iihex -Obinary $< $@
 
+quiet_cmd_ihex2fw  = IHEX2FW $@
+      cmd_ihex2fw  = $(objtree)/$(obj)/ihex2fw $< $@
+
 quiet_cmd_fwbin = MK_FW   $@
       cmd_fwbin = FWNAME="$(patsubst firmware/%.gen.S,%,$@)";		     \
 		  FWSTR="$(subst /,_,$(subst .,_,$(subst -,_,$(patsubst	     \
@@ -80,9 +83,18 @@ $(patsubst %,$(obj)/%.gen.S, $(fw-external-y)): %: $(wordsize_deps) \
 $(patsubst %,$(obj)/%.gen.o, $(fw-shipped-y)): %.gen.o: %
 $(patsubst %,$(obj)/%.gen.o, $(fw-external-y)): $(obj)/%.gen.o: $(fwdir)/%
 
+# .ihex is used just as a simple way to hold binary files in a source tree
+# where binaries are frowned upon. They are directly converted with objcopy.
 $(obj)/%: $(obj)/%.ihex | $(objtree)/$(obj)/$$(dir %)
 	$(call cmd,ihex)
 
+# .HEX is also Intel HEX, but where the offset and length in each record
+# is actually meaninful, because the firmware has to be loaded in a certain
+# order rather than as a single binary blob. Thus, we convert them into our
+# more compact binary representation of ihex records (<linux/ihex.h>)
+$(obj)/%.fw: $(obj)/%.HEX $(obj)/ihex2fw | $(objtree)/$(obj)/$$(dir %)
+	$(call cmd,ihex2fw)
+
 $(firmware-dirs):
 	$(call cmd,mkdir)
 
@@ -96,3 +108,5 @@ targets := $(fw-shipped-) $(patsubst $(obj)/%,%, \
 # Without this, built-in.o won't be created when it's empty, and the
 # final vmlinux link will fail.
 obj-n := dummy
+
+hostprogs-y := ihex2fw
firmware/ihex2fw.c new file +238 -0

238 lines not on this page · raw message

scripts/Makefile.fwinst +6 -1
index 3cf3649..f72355e 100644
@@ -6,10 +6,13 @@
 # ==========================================================================
 
 INSTALL := install
+src := $(obj)
 
 include scripts/Kbuild.include 
 include $(srctree)/$(obj)/Makefile
 
+include scripts/Makefile.host
+
 installed-fw := $(addprefix $(INSTALL_FW_PATH)/,$(fw-shipped-))
 installed-fw-dirs := $(sort $(dir $(installed-fw)))
 
@@ -22,5 +25,7 @@ $(installed-fw-dirs):
 $(installed-fw): $(INSTALL_FW_PATH)/%: $(obj)/% | $(INSTALL_FW_PATH)/$$(dir %)/
 	$(call cmd,install)
 
-.PHONY: __fw_install
+.PHONY: __fw_install FORCE
 __fw_install: $(installed-fw)
+
+FORCE:
Signature · 1 line
1.5.4.5
#15 lore raw

PATCH 14/18 keyspan_pda: use request_firmware()

In reply to David Woodhouse
---
 drivers/usb/serial/Kconfig           |   17 +
 drivers/usb/serial/keyspan_pda.S     | 1124 --------------------------------
 drivers/usb/serial/keyspan_pda.c     |   51 +-
 drivers/usb/serial/keyspan_pda_fw.h  |   99 ---
 drivers/usb/serial/xircom_pgs.S      | 1192 ----------------------------------
 drivers/usb/serial/xircom_pgs_fw.h   |  103 ---
 firmware/Makefile                    |    2 +
 firmware/WHENCE                      |   14 +
 firmware/keyspan_pda/keyspan_pda.HEX |   83 +++
 firmware/keyspan_pda/keyspan_pda.S   | 1124 ++++++++++++++++++++++++++++++++
 firmware/keyspan_pda/xircom_pgs.HEX  |   87 +++
 firmware/keyspan_pda/xircom_pgs.S    | 1192 ++++++++++++++++++++++++++++++++++
 12 files changed, 2543 insertions(+), 2545 deletions(-)
 delete mode 100644 drivers/usb/serial/keyspan_pda.S
 delete mode 100644 drivers/usb/serial/keyspan_pda_fw.h
 delete mode 100644 drivers/usb/serial/xircom_pgs.S
 delete mode 100644 drivers/usb/serial/xircom_pgs_fw.h
 create mode 100644 firmware/keyspan_pda/keyspan_pda.HEX
 create mode 100644 firmware/keyspan_pda/keyspan_pda.S
 create mode 100644 firmware/keyspan_pda/xircom_pgs.HEX
 create mode 100644 firmware/keyspan_pda/xircom_pgs.S
drivers/usb/serial/Kconfig +17 -0
index 31f9cc3..c354203 100644
@@ -279,6 +279,7 @@ config USB_SERIAL_IUU
 
 config USB_SERIAL_KEYSPAN_PDA
 	tristate "USB Keyspan PDA Single Port Serial Driver"
+	select FW_LOADER
 	select USB_EZUSB
 	help
 	  Say Y here if you want to use a Keyspan PDA single port USB to
@@ -288,6 +289,13 @@ config USB_SERIAL_KEYSPAN_PDA
 	  To compile this driver as a module, choose M here: the
 	  module will be called keyspan_pda.
 
+config USB_SERIAL_KEYSPAN_PDA_FIRMWARE
+	bool "Include Keyspan PDA firmware in kernel"
+	help
+	  This option allows you to build the required firmware for the
+	  Keyspan PDA serial port into the kernel. Say 'N' and let it
+	  get loaded by udev instead.
+
 config USB_SERIAL_KEYSPAN
 	tristate "USB Keyspan USA-xxx Serial Driver"
 	select FW_LOADER
@@ -538,6 +546,7 @@ config USB_SERIAL_CYBERJACK
 
 config USB_SERIAL_XIRCOM
 	tristate "USB Xircom / Entregra Single Port Serial Driver"
+	select FW_LOADER
 	select USB_EZUSB
 	help
 	  Say Y here if you want to use a Xircom or Entregra single port USB to
@@ -547,6 +556,14 @@ config USB_SERIAL_XIRCOM
 	  To compile this driver as a module, choose M here: the
 	  module will be called keyspan_pda.
 
+config USB_SERIAL_XIRCOM_FIRMWARE
+	bool "Include Xircom / Entregra firmware in kernel"
+	help
+	  This option allows you to build the required firmware for the
+	  Xircom / Entregra serial port into the kernel. Say 'N' and let
+	  it get loaded by udev instead.
+
+
 config USB_SERIAL_OPTION
 	tristate "USB driver for GSM and CDMA modems"
 	help
drivers/usb/serial/keyspan_pda.S deleted +0 -1124

1,124 lines not on this page · raw message

drivers/usb/serial/keyspan_pda.c +24 -27

94 lines not on this page · raw message

drivers/usb/serial/keyspan_pda_fw.h deleted +0 -99

99 lines not on this page · raw message

drivers/usb/serial/xircom_pgs.S deleted +0 -1192

1,192 lines not on this page · raw message

drivers/usb/serial/xircom_pgs_fw.h deleted +0 -103

103 lines not on this page · raw message

firmware/Makefile +2 -0
index 2abd142..3b73e91 100644
@@ -30,6 +30,8 @@ fw-shipped-$(CONFIG_USB_SERIAL_KEYSPAN_USA28XB) += keyspan/usa28xb.fw
 fw-shipped-$(CONFIG_USB_SERIAL_KEYSPAN_USA28X) += keyspan/usa28x.fw
 fw-shipped-$(CONFIG_USB_SERIAL_KEYSPAN_USA49W) += keyspan/usa49w.fw
 fw-shipped-$(CONFIG_USB_SERIAL_KEYSPAN_USA49WLC) += keyspan/usa49wlc.fw
+fw-shipped-$(CONFIG_USB_SERIAL_KEYSPAN_PDA_FIRMWARE) += keyspan_pda/keyspan_pda.fw
+fw-shipped-$(CONFIG_USB_SERIAL_XIRCOM_FIRMWARE) += keyspan_pda/xircom_pgs.fw
 
 firmware-y    := $(fw-external-y) $(fw-shipped-y)
 firmware-dirs := $(sort $(patsubst %,$(objtree)/$(obj)/%/,$(dir $(firmware-y))))
firmware/WHENCE +14 -0
index a5005a5..ff58069 100644
@@ -119,3 +119,17 @@ Original licence information:
 	part, requires the inclusion of this statement."
 
 --------------------------------------------------------------------------
+
+Driver: keyspan_pda -- USB Keyspan PDA single-port serial device
+
+File: keyspan_pda/keyspan_pda.fw
+Source: keyspan_pda/keyspan_pda.S
+
+File: keyspan_pda/xircom_pgs.fw
+Source: keyspan_pda/xircom_pgs.S
+
+Licence: GPLv2+
+
+Compiled from original 8051 source into Intel HEX, used in our binary ihex form.
+
+--------------------------------------------------------------------------
firmware/keyspan_pda/keyspan_pda.HEX new file +83 -0

83 lines not on this page · raw message

firmware/keyspan_pda/keyspan_pda.S new file +1124 -0

239 lines not on this page · raw message

firmware/keyspan_pda/xircom_pgs.HEX new file +87 -0

87 lines not on this page · raw message

firmware/keyspan_pda/xircom_pgs.S new file +1192 -0

1,192 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
1.5.4.5

Part of the diff is not shown here. View the raw message.

#16 lore raw

PATCH 17/18 ti_usb_3410_5052: use request_firmware()

In reply to David Woodhouse
---
 drivers/usb/serial/Kconfig            |   17 +
 drivers/usb/serial/ti_fw_3410.h       |  885 ---------------------------------
 drivers/usb/serial/ti_fw_5052.h       |  885 ---------------------------------
 drivers/usb/serial/ti_usb_3410_5052.c |   40 +-
 firmware/Makefile                     |    3 +
 firmware/WHENCE                       |   15 +
 firmware/ti_3410.fw.ihex              |  862 ++++++++++++++++++++++++++++++++
 firmware/ti_5052.fw.ihex              |  862 ++++++++++++++++++++++++++++++++
 8 files changed, 1786 insertions(+), 1783 deletions(-)
 delete mode 100644 drivers/usb/serial/ti_fw_3410.h
 delete mode 100644 drivers/usb/serial/ti_fw_5052.h
 create mode 100644 firmware/ti_3410.fw.ihex
 create mode 100644 firmware/ti_5052.fw.ihex
drivers/usb/serial/Kconfig +17 -0
index c354203..a9a6aff 100644
@@ -525,6 +525,7 @@ config USB_SERIAL_SIERRAWIRELESS
 
 config USB_SERIAL_TI
 	tristate "USB TI 3410/5052 Serial Driver"
+	select FW_LOADER
 	help
 	  Say Y here if you want to use the TI USB 3410 or 5052
 	  serial devices.
@@ -532,6 +533,22 @@ config USB_SERIAL_TI
 	  To compile this driver as a module, choose M here: the
 	  module will be called ti_usb_3410_5052.
 
+config USB_SERIAL_TI_3410_FIRMWARE
+	bool "Include USB TI 3410 firmware in kernel"
+	depends on USB_SERIAL_TI
+	help
+	  This option allows you to build the required firmware for the
+	  USB TI 3410 devices into the kernel. Say 'N' and let it
+	  get loaded by udev instead.
+
+config USB_SERIAL_TI_5052_FIRMWARE
+	bool "Include USB TI 5052 firmware in kernel"
+	depends on USB_SERIAL_TI
+	help
+	  This option allows you to build the required firmware for the
+	  USB TI 5052 devices into the kernel. Say 'N' and let it
+	  get loaded by udev instead.
+
 config USB_SERIAL_CYBERJACK
 	tristate "USB REINER SCT cyberJack pinpad/e-com chipcard reader"
 	---help---
drivers/usb/serial/ti_fw_3410.h deleted +0 -885

885 lines not on this page · raw message

drivers/usb/serial/ti_fw_5052.h deleted +0 -885

885 lines not on this page · raw message

drivers/usb/serial/ti_usb_3410_5052.c +27 -13
index a1c8aef..a26a629 100644
@@ -84,11 +84,9 @@
 #include <asm/uaccess.h>
 #include <linux/usb.h>
 #include <linux/usb/serial.h>
+#include <linux/firmware.h>
 
 #include "ti_usb_3410_5052.h"
-#include "ti_fw_3410.h"		/* firmware image for 3410 */
-#include "ti_fw_5052.h"		/* firmware image for 5052 */
-
 
 /* Defines */
 
@@ -194,8 +192,8 @@ static int ti_command_in_sync(struct ti_device *tdev, __u8 command,
 static int ti_write_byte(struct ti_device *tdev, unsigned long addr,
 	__u8 mask, __u8 byte);
 
-static int ti_download_firmware(struct ti_device *tdev,
-	unsigned char *firmware, unsigned int firmware_size);
+static int ti_download_firmware(struct ti_device *tdev, char *fw_name);
+
 
 /* circular buffer */
 static struct circ_buf *ti_buf_alloc(void);
@@ -320,6 +318,9 @@ MODULE_DESCRIPTION(TI_DRIVER_DESC);
 MODULE_VERSION(TI_DRIVER_VERSION);
 MODULE_LICENSE("GPL");
 
+MODULE_FIRMWARE("ti_3410.fw");
+MODULE_FIRMWARE("ti_5052.fw");
+
 module_param(debug, bool, S_IRUGO | S_IWUSR);
 MODULE_PARM_DESC(debug, "Enable debugging, 0=no, 1=yes");
 
@@ -431,11 +432,9 @@ static int ti_startup(struct usb_serial *serial)
 	if (dev->descriptor.bNumConfigurations == 1) {
 
 		if (tdev->td_is_3410)
-			status = ti_download_firmware(tdev, ti_fw_3410,
-				sizeof(ti_fw_3410));
+			status = ti_download_firmware(tdev, "ti_3410.fw");
 		else
-			status = ti_download_firmware(tdev, ti_fw_5052,
-				sizeof(ti_fw_5052));
+			status = ti_download_firmware(tdev, "ti_5052.fw");
 		if (status)
 			goto free_tdev;
 
@@ -1658,8 +1657,9 @@ static int ti_write_byte(struct ti_device *tdev, unsigned long addr,
 
 
 static int ti_download_firmware(struct ti_device *tdev,
-	unsigned char *firmware, unsigned int firmware_size)
+				char *fw_name)
 {
+	const struct firmware *fw;
 	int status = 0;
 	int buffer_size;
 	int pos;
@@ -1672,16 +1672,29 @@ static int ti_download_firmware(struct ti_device *tdev,
 	unsigned int pipe = usb_sndbulkpipe(dev,
 		tdev->td_serial->port[0]->bulk_out_endpointAddress);
 
-
 	buffer_size = TI_FIRMWARE_BUF_SIZE + sizeof(struct ti_firmware_header);
+
+	if (request_firmware(&fw, fw_name, &dev->dev)) {
+		dev_err(&dev->dev, "%s - failed to load firmware \"%s\"\n",
+			__func__, fw_name);
+		return -ENOENT;
+	}
+	if (fw->size > buffer_size) {
+		dev_err(&dev->dev, "%s - firmware \"%s\" is too large\n",
+			__func__, fw_name);
+		release_firmware(fw);
+		return -EINVAL;
+	}
+
 	buffer = kmalloc(buffer_size, GFP_KERNEL);
 	if (!buffer) {
 		dev_err(&dev->dev, "%s - out of memory\n", __func__);
+		release_firmware(fw);
 		return -ENOMEM;
 	}
 
-	memcpy(buffer, firmware, firmware_size);
-	memset(buffer+firmware_size, 0xff, buffer_size-firmware_size);
+	memcpy(buffer, fw->data, fw->size);
+	memset(buffer+fw->size, 0xff, buffer_size-fw->size);
 
 	for(pos = sizeof(struct ti_firmware_header); pos < buffer_size; pos++)
 		cs = (__u8)(cs + buffer[pos]);
@@ -1699,6 +1712,7 @@ static int ti_download_firmware(struct ti_device *tdev,
 	}
 
 	kfree(buffer);
+	release_firmware(fw);
 
 	if (status) {
 		dev_err(&dev->dev, "%s - error downloading firmware, %d\n", __func__, status);
firmware/Makefile +3 -0
index 692c4b9..df672f2 100644
@@ -36,6 +36,9 @@ fw-shipped-$(CONFIG_USB_EMI26_FIRMWARE) += emi26/loader.fw emi26/firmware.fw \
 		emi26/bitstream.fw
 fw-shipped-$(CONFIG_USB_EMI62_FIRMWARE) += emi62/loader.fw emi62/bitstream.fw \
 		emi62/spdif.fw emi62/midi.fw
+fw-shipped-$(CONFIG_USB_TI_3410_FIRMWARE) += ti_3410.fw
+fw-shipped-$(CONFIG_USB_TI_5052_FIRMWARE) += ti_5052.fw
+
 firmware-y    := $(fw-external-y) $(fw-shipped-y)
 firmware-dirs := $(sort $(patsubst %,$(objtree)/$(obj)/%/,$(dir $(firmware-y))))
 
firmware/WHENCE +15 -0
index 9f22dfe..cb28fa9 100644
@@ -190,3 +190,18 @@ Converted from Intel HEX files, used in our binary representation of ihex.
 Original licence information: None
 
 --------------------------------------------------------------------------
+
+Driver: tu_usb_3410_5052 -- USB TI 3410/5052 serial device
+
+File: ti_3410.fw
+Info: firmware 9/10/04 FW3410_Special_StartWdogOnStartPort
+
+File: ti_5052.fw
+Info: firmware 9/18/04 
+
+Licence: Allegedly GPLv2+, but no source visible. Marked:
+	 Copyright (C) 2004 Texas Instruments
+
+Found in hex form in kernel source. 
+
+--------------------------------------------------------------------------
firmware/ti_3410.fw.ihex new file +862 -0

862 lines not on this page · raw message

firmware/ti_5052.fw.ihex new file +862 -0

219 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
1.5.4.5

Part of the diff is not shown here. View the raw message.

#17 lore raw

PATCH 18/18 whiteheat: use request_firmware()

In reply to David Woodhouse
---
 drivers/usb/serial/Kconfig          |    8 +
 drivers/usb/serial/whiteheat.c      |   77 +-
 drivers/usb/serial/whiteheat_fw.h   | 1669 -----------------------------------
 firmware/Makefile                   |    2 +
 firmware/WHENCE                     |   21 +
 firmware/whiteheat.HEX              | 1097 +++++++++++++++++++++++
 firmware/whiteheat_loader.HEX       |  314 +++++++
 firmware/whiteheat_loader_debug.HEX |  403 +++++++++
 8 files changed, 1895 insertions(+), 1696 deletions(-)
 delete mode 100644 drivers/usb/serial/whiteheat_fw.h
 create mode 100644 firmware/whiteheat.HEX
 create mode 100644 firmware/whiteheat_loader.HEX
 create mode 100644 firmware/whiteheat_loader_debug.HEX
drivers/usb/serial/Kconfig +8 -0
index a9a6aff..5c536a2 100644
@@ -102,6 +102,7 @@ config USB_SERIAL_CH341
 
 config USB_SERIAL_WHITEHEAT
 	tristate "USB ConnectTech WhiteHEAT Serial Driver"
+	select FW_LOADER
 	select USB_EZUSB
 	help
 	  Say Y here if you want to use a ConnectTech WhiteHEAT 4 port
@@ -110,6 +111,13 @@ config USB_SERIAL_WHITEHEAT
 	  To compile this driver as a module, choose M here: the
 	  module will be called whiteheat.
 
+config USB_SERIAL_WHITEHEAT_FIRMWARE
+	bool "Firmware for USB ConnectTech WhiteHEAT Serial Driver"
+	depends on USB_SERIAL_WHITEHEAT
+	help
+	  Builds firmware into the kernel instead of loading it normally
+	  via udev. Say 'N'.
+
 config USB_SERIAL_DIGI_ACCELEPORT
 	tristate "USB Digi International AccelePort USB Serial Driver"
 	---help---
drivers/usb/serial/whiteheat.c +50 -27

121 lines not on this page · raw message

drivers/usb/serial/whiteheat_fw.h deleted +0 -1669

1,669 lines not on this page · raw message

firmware/Makefile +2 -0
index df672f2..d60b4c4 100644
@@ -38,6 +38,8 @@ fw-shipped-$(CONFIG_USB_EMI62_FIRMWARE) += emi62/loader.fw emi62/bitstream.fw \
 		emi62/spdif.fw emi62/midi.fw
 fw-shipped-$(CONFIG_USB_TI_3410_FIRMWARE) += ti_3410.fw
 fw-shipped-$(CONFIG_USB_TI_5052_FIRMWARE) += ti_5052.fw
+fw-shipped-$(CONFIG_USB_SERIAL_WHITEHEAT_FIRMWARE) += whiteheat_loader.fw \
+		whiteheat.fw # whiteheat_loader_debug.fw
 
 firmware-y    := $(fw-external-y) $(fw-shipped-y)
 firmware-dirs := $(sort $(patsubst %,$(objtree)/$(obj)/%/,$(dir $(firmware-y))))
firmware/WHENCE +21 -0
index cb28fa9..f460541 100644
@@ -205,3 +205,24 @@ Licence: Allegedly GPLv2+, but no source visible. Marked:
 Found in hex form in kernel source. 
 
 --------------------------------------------------------------------------
+
+Driver: whiteheat -- USB ConnectTech WhiteHEAT serial device
+
+File: whiteheat.fw
+Version: 4.06
+
+File: whiteheat_loader.fw
+File: whiteheat_loader_debug.fw
+
+Licence: Allegedly GPLv2, but no source visible. Marked:
+	 Copyright (C) 2000-2002  ConnectTech Inc
+
+Debug loader claims the following behaviour:
+	Port 1 LED flashes when the vend_ax program is running
+	Port 2 LED flashes when any SETUP command arrives
+	Port 3 LED flashes when any valid VENDOR request occurs
+	Port 4 LED flashes when the EXTERNAL RAM DOWNLOAD request occurs
+
+Converted from Intel HEX files, used in our binary representation of ihex.
+
+--------------------------------------------------------------------------
firmware/whiteheat.HEX new file +1097 -0

1,097 lines not on this page · raw message

firmware/whiteheat_loader.HEX new file +314 -0

61 lines not on this page · raw message

firmware/whiteheat_loader_debug.HEX new file +403 -0

403 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
1.5.4.5

Part of the diff is not shown here. View the raw message.

#19 lore raw
In reply to Arkadiusz Miskiewicz
Quoted 11 lines · Arkadiusz Miskiewicz
Quoted 9 lines · David Woodhouse

Ignoring the set of 'make fw->data const' patches which have mostly been already posted and are fairly trivial, this is the current content of the firmware tree at git.infradead.org/users/dwmw2/firmware-2.6.git

It starts by giving a way to build arbitrary firmware blobs into the kernel while letting request_firmware() find them there, and then sets about converting drivers to use request_firmware() instead of static data blobs, now that we can do that without forcing the users to actually satisfy the firmware requests from userspace.

Quoted 6 lines

Will there be a way for userspace to know what firmware file is needed by which driver?

So for example geninitrd/mkinitrd will be able to put only required firmware files into initrd/initramfs (without a need to maintain separate database with driver <-> firmware file name mapping).

We've had that for a long time: MODULE_FIRMWARE().

Signature · 1 line
dwmw2
#18 lore raw
In reply to David Woodhouse
Quoted 9 lines · David Woodhouse

Ignoring the set of 'make fw->data const' patches which have mostly been already posted and are fairly trivial, this is the current content of the firmware tree at git.infradead.org/users/dwmw2/firmware-2.6.git

It starts by giving a way to build arbitrary firmware blobs into the kernel while letting request_firmware() find them there, and then sets about converting drivers to use request_firmware() instead of static data blobs, now that we can do that without forcing the users to actually satisfy the firmware requests from userspace.

Will there be a way for userspace to know what firmware file is needed by which driver?

So for example geninitrd/mkinitrd will be able to put only required firmware files into initrd/initramfs (without a need to maintain separate database with driver <-> firmware file name mapping).

Signature · 2 lines
Arkadiusz Miśkiewicz        PLD/Linux Team
arekm / maven.pl            http://ftp.pld-linux.org/
#20 lore raw
In reply to David Woodhouse
Quoted 2 lines · David Woodhouse

More patches to convert drivers to request_firmware() would be appreciated,

I have an emi26/emi62 patch that I need to clean up a little ...

Regards,
Clemens

#11 lore raw

Re: [PATCH 10/18] ihex.h: binary representation of ihex records

In reply to David Woodhouse
Quoted 2 lines · David Woodhouse

Some devices need their firmware as a set of {address, len, data...} records in some specific order rather than a simple blob.

At least in the case of EZ-USB-based devices, this is not a requirement. It seems most tools just do not bother to order the records in Intel hex files. Any firmware that is downloaded with the EZ-USB's "load internal" request (0xa0) can be converted to a plain memory dump.

Regards,
Clemens

#21 lore raw
In reply to David Woodhouse
Quoted 20 lines · David Woodhouse

Ignoring the set of 'make fw->data const' patches which have mostly been already posted and are fairly trivial, this is the current content of the firmware tree at git.infradead.org/users/dwmw2/firmware-2.6.git

It starts by giving a way to build arbitrary firmware blobs into the kernel while letting request_firmware() find them there, and then sets about converting drivers to use request_firmware() instead of static data blobs, now that we can do that without forcing the users to actually satisfy the firmware requests from userspace.

Rather than including binary blobs in the git tree, we carry them as .ihex files -- which as an added bonus can have comments and licensing information appended after the EOF record.

For the USB drivers which actually _want_ Intel HEX files in their original form as individual records, we have a binary representation of such and a tool to convert the original .HEX files into .fw files in that form. And associated helpers in the kernel for validating and using them.

Quoted 3 lines

We also add a 'make firmware_install' target, which is intended to install the firmware blobs into /lib/firmware/ where the userspace loader to find them.

Why do we need the .ihex format?

We should be loading firmwares exactly as they come from the vendor, without any wrappers.

Jeff

#22 lore raw
Quoted 4 lines · Jeff Garzik

Why do we need the .ihex format?

We should be loading firmwares exactly as they come from the vendor, without any wrappers.

It's just 'objcopy -Ibinary -Oihex foo foo.ihex'. It's purely for convenience of handling the binary blobs in git (and it means we can append licensing information after the EOF record too, if we want).

Signature · 1 line
dwmw2
#23 lore raw
Quoted 8 lines · David Woodhouse
Quoted 3 lines · Jeff Garzik

Why do we need the .ihex format?

We should be loading firmwares exactly as they come from the vendor, without any wrappers.

It's just 'objcopy -Ibinary -Oihex foo foo.ihex'. It's purely for convenience of handling the binary blobs in git (and it means we can append licensing information after the EOF record too, if we want).

Well, I should ask, is this purely an internal build detail?

The developer (and user) should never ever see a .ihex file, outside of an active kernel compile. Wrapping the original firmware makes it more difficult to verify, compare and/or change. In-tree, we should see the vendor firmware blobs as shipped, with no wrapping or modification or anything.

Any metadata can be added as a separate file.

If .ihex is purely a temporary thing during compile, no big deal, ignore me. Otherwise, that's something we should avoid.

Jeff

#24 lore raw
Quoted 7 lines · Jeff Garzik

Well, I should ask, is this purely an internal build detail?

The developer (and user) should never ever see a .ihex file, outside of an active kernel compile. Wrapping the original firmware makes it more difficult to verify, compare and/or change. In-tree, we should see the vendor firmware blobs as shipped, with no wrapping or modification or anything.

It's a build-and-shipping detail, at least in the kernel source tree.

Rather than putting binary blobs into git (which admittedly we could, but it's suboptimal), we just run them through 'objcopy -Oihex' first.

It's not 'wrapped'; it's not any more difficult to verify, compare or change.

Signature · 1 line
dwmw2
#25 lore raw
Quoted 5 lines · David Woodhouse
Quoted 3 lines · Jeff Garzik

Well, I should ask, is this purely an internal build detail?

The developer (and user) should never ever see a .ihex file, outside of an active kernel compile. Wrapping the original firmware makes it more difficult to verify, compare and/or change. In-tree, we should see the vendor firmware blobs as shipped, with no wrapping or modification or anything.

Quoted 7 lines

It's a build-and-shipping detail, at least in the kernel source tree.

Rather than putting binary blobs into git (which admittedly we could, but it's suboptimal), we just run them through 'objcopy -Oihex' first.

It's not 'wrapped'; it's not any more difficult to verify, compare or change.

If the sha1 sum of what is in the kernel tree differs from what the vendor provided, then it is OBVIOUSLY more difficult to verify that you have the original firmware as provided by the vendor.

Put the binary blobs into the git tree, __without modification or wrapping__.

Jeff

#26 lore raw
Quoted 6 lines · Jeff Garzik

If the sha1 sum of what is in the kernel tree differs from what the vendor provided, then it is OBVIOUSLY more difficult to verify that you have the original firmware as provided by the vendor.

Put the binary blobs into the git tree, __without modification or wrapping__.

We don't have them in that form right now. Of the firmware blobs I've encountered so far -- even the ones which were in a file on their own -- none of them are in binary form; they're _all_ in some ASCII representation which can be processed with 'diff'. That includes char arrays, arrays of larger integers which need endian-awareness, 'hex record' structures, and probably a bunch of other abominations I have yet to encounter as I work through them.

None of them have just been binary files in the source tree.

I do not believe that taking the existing ascii-source representation and turning it into binary blobs in the source tree is something which will be unanimously welcomed. But if you prove me wrong, it's trivial to switch to doing it that way. For example:

	objcopy -Iihex -Obinary firmware/tr_smctr.bin{.ihex,}
	git-rm firmware/smctr.bin.ihex
	git-add firmware/smctr.bin
	git-commit

I'm going to be making a 'shadow' tree containing the result of running 'make firmware_install', just as I have a tree for exported headers. That tree _can_ have the raw binaries¹. But I don't think it's appropriate while the firmware is still in the kernel source tree.

Signature · 5 lines
dwmw2

¹ And I plan to make another tree which pulls from that but also includes
  'distributable' firmware blobs, which the owners wouldn't put into the
  kernel tree because of the GPL requirement that would imply.
#27 lore raw
Quoted 14 lines · David Woodhouse
Quoted 4 lines · Jeff Garzik

If the sha1 sum of what is in the kernel tree differs from what the vendor provided, then it is OBVIOUSLY more difficult to verify that you have the original firmware as provided by the vendor.

Put the binary blobs into the git tree, __without modification or wrapping__.

We don't have them in that form right now. Of the firmware blobs I've encountered so far -- even the ones which were in a file on their own -- none of them are in binary form; they're _all_ in some ASCII representation which can be processed with 'diff'. That includes char arrays, arrays of larger integers which need endian-awareness, 'hex record' structures, and probably a bunch of other abominations I have yet to encounter as I work through them.

Quoted 1 line

None of them have just been binary files in the source tree.

Right. And now you are creating Yet Another Format, rather than rendering the firmware back into the preferred format: binary blob.

_If_ you are changing form of current in-tree firmwares at all, there is no excuse not use direct binary blob -- the least common denominator for all relevant operations.

Storing the firmware in .ihex is just as bad as storing the firmware in source code -- it's a pointless wrapper that makes firmware verification and updates far more difficult than they should be.

Jeff

#29 lore raw
In reply to David Woodhouse

Here is my "ideal firmware world":

* do NOT remove ability to compile firmware into the kernel (or into a module)

* firmware should be field-replaceable, even if one was compiled in

* the preferred form of firmware is one or more binary blobs, stored in a regular [filesystem|git] binary file.

* the preferred form is NOT ascii C source, or any other format other than the native format that the hardware wants. Remember, the vendor only provided an ASCII-ized firmware because our system required it that way, not because that's the preferred form.

* in-tree firmwares should be stored in one or more binary files in the tree, not in C source code or .ihex files.

* when firmware is stored as binary blob, it is easier for
(1) vendor to replace,
(2) developer/user to compare/verify using sha1,
(3) does not require a C compiler or binutils to unpack

Thus, if you are going to be touching the in-tree firmwares at all, it doesn't make sense to convert from C source to any format other than native binary.

Jeff

#28 lore raw
In reply to Jeff Garzik
Quoted 2 lines · Jeff Garzik

Right. And now you are creating Yet Another Format, rather than rendering the firmware back into the preferred format: binary blob.

At least it's simpler than the various formats we have at the moment.

Quoted 3 lines

_If_ you are changing form of current in-tree firmwares at all, there is no excuse not use direct binary blob -- the least common denominator for all relevant operations.

There's plenty of excuses. The fact that we don't have any binary blobs in the tree is a fairly big hint -- they're not easily processed by the tools we use (like 'diff').

I'm keen to avoid unnecessary controversy while I convert drivers to request_firmware(). I don't want to convert the existing ASCII representations into raw binary blobs right now, because it's not my primary focus and I don't care enough about it to fight that battle.

If you want to follow up with a patch to remove .ihex files and replace them with raw binaries -- or better still start posting patches which convert the drivers I've yet to get to, and just put raw binary blobs into firmware/ instead of .ihex as I would, then please go ahead. But please don't be offended if I don't put them in my firmware-2.6.git tree because I think that'll make it less likely to get merged.

Quoted 3 lines

Storing the firmware in .ihex is just as bad as storing the firmware in source code -- it's a pointless wrapper that makes firmware verification and updates far more difficult than they should be.

But still a lot easier than before I did it. One thing at a time, Jeff.

Signature · 1 line
dwmw2
#30 lore raw
In reply to Jeff Garzik
Quoted 3 lines · Jeff Garzik

Here is my "ideal firmware world":

* do NOT remove ability to compile firmware into the kernel

I have no intention of removing that.

Quoted 1 line

(or into a module)

That I don't care about. If you can load modules, you can handle request_firmware().

Quoted 1 line

* firmware should be field-replaceable, even if one was compiled in

That would be a useful thing; currently the built-in firmwares cannot be overridden but it wouldn't be hard to implement. Suggest it in 'diff -u' form and it might just happen.

Quoted 2 lines

* the preferred form of firmware is one or more binary blobs, stored in a regular [filesystem|git] binary file.

I don't think we have consensus on that, but as I said: post patches and let's see if they get merged.

Quoted 4 lines

* the preferred form is NOT ascii C source, or any other format other than the native format that the hardware wants. Remember, the vendor only provided an ASCII-ized firmware because our system required it that way, not because that's the preferred form.

And binary isn't the preferred form either. We can cope with binary, but it's not appropriate in the kernel source tree, imho.

When I make the shadow tree which contains the results of 'make firmware_install', that'll have the binaries.

Quoted 7 lines

* in-tree firmwares should be stored in one or more binary files in the tree, not in C source code or .ihex files.

* when firmware is stored as binary blob, it is easier for
(1) vendor to replace,
(2) developer/user to compare/verify using sha1,
(3) does not require a C compiler or binutils to unpack

All these are true, but you're still missing the point that as I have it it's already a _lot_ easier to process than when it was arrays of __be32 in some header file. I don't want to change to binary blobs as part of what I'm doing this week. That's a _separate_ issue, and I'm not even sure it's a goal I agree with.

Quoted 3 lines

Thus, if you are going to be touching the in-tree firmwares at all, it doesn't make sense to convert from C source to any format other than native binary.

I disagree. But feel free to post patches which get applied and prove me wrong. I'm not going to object if you want to go convert a few more drivers.

Signature · 1 line
dwmw2
#12 lore raw

Re: [PATCH 10/18] ihex.h: binary representation of ihex records

In reply to Clemens Ladisch
Quoted 9 lines · Clemens Ladisch
Quoted 2 lines · David Woodhouse

Some devices need their firmware as a set of {address, len, data...} records in some specific order rather than a simple blob.

At least in the case of EZ-USB-based devices, this is not a requirement. It seems most tools just do not bother to order the records in Intel hex files. Any firmware that is downloaded with the EZ-USB's "load internal" request (0xa0) can be converted to a plain memory dump.

Hm, in that case it might be worth converting this stuff into raw binaries. I'm not in a position to test, and wanted to keep my own changes as unintrusive as possible though.

Signature · 1 line
dwmw2
#31 lore raw
In reply to David Woodhouse
Quoted 6 lines · David Woodhouse
Quoted 1 line · Jeff Garzik

Thus, if you are going to be touching the in-tree firmwares at all, it doesn't make sense to convert from C source to any format other than native binary.

I disagree. But feel free to post patches which get applied and prove me wrong. I'm not going to object if you want to go convert a few more drivers.

IMO, if-and-only-if you are touching driver firmwares, you should not do anything but render them into their preferred form.

It is useless churn that increases maintenance annoyances to convert an in-tree firmware to something other than the form it's in now, so you might as well get it right.

Jeff

#32 lore raw
Quoted 6 lines · Jeff Garzik

IMO, if-and-only-if you are touching driver firmwares, you should not do anything but render them into their preferred form.

It is useless churn that increases maintenance annoyances to convert an in-tree firmware to something other than the form it's in now, so you might as well get it right.

I believe that making that change would cause extra controversy that I don't need, and make it harder for me to get what I _actually_ care about merged -- which is the basic conversion to request_firmware().

But as I said, if _you_ want to start converting drivers and posting patches which add raw binary blobs in the firmware/ directory, please feel free. If they go in without comment, I'll be happy to admit that I was wrong and start doing raw binaries in future.

Signature · 1 line
dwmw2
#33 lore raw
In reply to Jeff Garzik
Quoted 1 line

* firmware should be field-replaceable, even if one was compiled in

If you want to replace it then put it in the file system. You've said binuntils isn't acceptable in your "vision" below so field replacing compiled in ones clearly is. It's a needless overcomplication.

Quoted 2 lines

* the preferred form of firmware is one or more binary blobs, stored in a regular [filesystem|git] binary file.

That depends on the firmware. In a few cases we even have firmware source in the tree.

Quoted 5 lines

* in-tree firmwares should be stored in one or more binary files in the tree, not in C source code or .ihex files.

* when firmware is stored as binary blob, it is easier for
(1) vendor to replace,

Nope, they can't submit a diff to the tree except as an attachment which will be dropped if you do that.

Quoted 1 line

(2) developer/user to compare/verify using sha1,

sha1 doesn't appear to be totally secure for that purpose with a firmware block any more. Putting the sha1 in the header might be a good idea in some cases (eg DaveM wants to be sure precisely *which* tg3 firmware he loads)

Quoted 1 line

(3) does not require a C compiler or binutils to unpack

Why ? What is the usage scenario ? Vendor distributed firmware for PC class systems is going to be a deb or rpm that drops files into ../lib/firmware.

Quoted 3 lines

Thus, if you are going to be touching the in-tree firmwares at all, it doesn't make sense to convert from C source to any format other than native binary.

You don't mean native binary either. Native binary on some of these devices is very strange indeed. Its simply bytes in various random formats for stuffing into the device in various random ways - that's not really "native" in all cases.

Alan