firmware: speed up request_firmware()
Subsystems (3)
PATCH 1/1 firmware: speed up request_firmware()
Trying to load a large file with request_firmware() is currently very slow, due to calling vmalloc() for each write. This patch makes the allocations double in size each time.
Without this patch, loading a 12MB firmware image (FPGA bitfiles) called vmalloc() approximately 3000 times. With the patch, vmalloc() is only called 13 times, however, some memory is wasted until firmware_release() is called. Usually, this happens very quickly, so it shouldn't be a problem.
Without this patch, it took about 2 1/2 minutes to load the 12MB firmware image. With the patch, it now takes about 1/2 second.
--- drivers/base/firmware_class.c | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-)
drivers/base/firmware_class.c +6 -1
@@ -207,7 +207,12 @@ fw_realloc_buffer(struct firmware_priv *fw_priv, int min_size)if (min_size <= fw_priv->alloc_size)return 0 ; - new_size =ALIGN (min_size,PAGE_SIZE ); +/* Allocate double the current size. If that is not enough, allocate +* the minimum amount needed instead (aligned to PAGE_SIZE). */ + new_size = fw_priv->alloc_size *2 ; +if (new_size < min_size) + new_size =ALIGN (min_size,PAGE_SIZE ); + new_data =vmalloc (new_size);if (!new_data) {printk (KERN_ERR "%s: unable to alloc buffer\n" , __func__);
Signature · 1 line
1.5.4.3
Quoted 4 lines
Without this patch, loading a 12MB firmware image (FPGA bitfiles) called
vmalloc()approximately 3000 times. With the patch,vmalloc()is only called 13 times, however, some memory is wasted untilfirmware_release()is called. Usually, this happens very quickly, so it shouldn't be a problem.
Particularly on 32bit x86 vmalloc memory space is very precious so while the current buffer size default may be too small doubling repeatedly is not a good model IMHO.
Quoted 8 lines · Alan Cox
Quoted 4 lines
Without this patch, loading a 12MB firmware image (FPGA bitfiles) called
vmalloc()approximately 3000 times. With the patch,vmalloc()is only called 13 times, however, some memory is wasted untilfirmware_release()is called. Usually, this happens very quickly, so it shouldn't be a problem.Particularly on 32bit x86 vmalloc memory space is very precious so while the current buffer size default may be too small doubling repeatedly is not a good model IMHO.
How about a small exponent like 1.1?
Signature · 1 line
http://selenic.com : development and support for Mercurial and Linux
Quoted 7 lines · Ira Snyder
Without this patch, loading a 12MB firmware image (FPGA bitfiles) called
vmalloc()approximately 3000 times. With the patch,vmalloc()is only called 13 times, however, some memory is wasted untilfirmware_release()is called. Usually, this happens very quickly, so it shouldn't be a problem.
Some drivers cache firmware in memory. Doubing the amount of needed memory definitely would not be the best idea. Check drivers/net/wireless for examples.
Quoted 3 lines
Some drivers cache firmware in memory. Doubing the amount of needed memory definitely would not be the best idea. Check
drivers/net/wirelessfor examples.
A lot of drivers could perfectly happily exist with a simple iterator helper and being returned sg lists of pages. It seems that for big firmwares at least there is a root cause which is deeper than how you grow your vmalloc buffer.
Alan
Quoted 9 lines · Alan Cox
Quoted 3 lines
Some drivers cache firmware in memory. Doubing the amount of needed memory definitely would not be the best idea. Check
drivers/net/wirelessfor examples.A lot of drivers could perfectly happily exist with a simple iterator helper and being returned sg lists of pages. It seems that for big firmwares at least there is a root cause which is deeper than how you grow your vmalloc buffer.
An sg list of pages would be perfect for my usage. I didn't want to change an existing kernel interface, so I just made the easiest change that worked for me.
Another thing that could be done is trimming the vmalloc() down to the exact size needed after the firmware has finished loading. That would still waste memory until the copy from userspace is finished, though.
I'd be happy to test patches anyone comes up with.
Thanks,
Ira
Quoted 2 lines · Ira Snyder
I didn't want to change an existing kernel interface, so I just made the easiest change that worked for me.
Well, userspace does know the actual size of the image, so I see no reason why it shouldn't be able to tell the kernel about it beforehand.
Quoted 1 line
I'd be happy to test patches anyone comes up with.
=====
This adds a data_size attribute to the firmware loading device so that userspace can tell us about the firmware image size. This allows us to preallocate the buffer with the final size, thus avoiding reallocating the buffer for every page of data as it comes in.
Documentation/firmware_class/README +11 -3
@@ -21,7 +21,7 @@ kernel(driver): calls request_firmware(&fw_entry, $FIRMWARE, device) userspace: - - /sys/class/firmware/xxx/{loading,data} appear. + - /sys/class/firmware/xxx/{loading,data,data_size} appear. - hotplug gets called with a firmware identifier in $FIRMWARE and the usual hotplug environment. - hotplug: echo 1 > /sys/class/firmware/xxx/loading
@@ -29,11 +29,13 @@ kernel: Discard any previous partial load. userspace: + - hotplug: echo ... > /sys/class/firmware/xxx/data_size - hotplug: cat appropriate_firmware_image > \ /sys/class/firmware/xxx/data - kernel: grows a buffer in PAGE_SIZE increments to hold the image as it - comes in. + kernel: Copies the firmware image into a buffer of the specified size. + If the image is larger, the buffer automatically grows in + PAGE_SIZE increments. userspace: - hotplug: echo 0 > /sys/class/firmware/xxx/loading
@@ -60,6 +62,9 @@ HOTPLUG_FW_DIR=/usr/lib/hotplug/firmware/ + if [ -e /sys/$DEVPATH/data_size ]; then + stat -c %s $HOTPLUG_FW_DIR/$FIRMWARE > /sys/$DEVPATH/data_size + fi echo 1 > /sys/$DEVPATH/loading cat $HOTPLUG_FW_DIR/$FIRMWARE > /sysfs/$DEVPATH/data echo 0 > /sys/$DEVPATH/loading
@@ -73,6 +78,9 @@ - firmware_data_read() and firmware_loading_show() are just provided for testing and completeness, they are not called in normal use. + - /sys/class/firmware/xxx/data_size is optional for compatibility with + older kernels. + - There is also /sys/class/firmware/timeout which holds a timeout in seconds for the whole load operation.
Documentation/firmware_class/hotplug-script +3 -0
@@ -6,6 +6,9 @@ HOTPLUG_FW_DIR=/usr/lib/hotplug/firmware/ +if [ -e /sys/$DEVPATH/data_size ]; then + stat -c %s $HOTPLUG_FW_DIR/$FIRMWARE > /sys/$DEVPATH/data_size +fi echo 1 > /sys/$DEVPATH/loading cat $HOTPLUG_FW_DIR/$FIRMWARE > /sys/$DEVPATH/data echo 0 > /sys/$DEVPATH/loading
drivers/base/firmware_class.c +34 -0
@@ -46,6 +46,7 @@ struct firmware_priv {struct firmware *fw;unsigned long status;int alloc_size; +int size_hint;struct timer_list timeout; };
@@ -114,6 +115,32 @@ static struct class firmware_class = { .dev_release = fw_dev_release, }; +static ssize_t firmware_data_size_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ +struct firmware_priv *fw_priv =dev_get_drvdata (dev); + +return sprintf (buf,"%d\n" , fw_priv->size_hint); +} + +static ssize_t firmware_data_size_store (struct device *dev, +struct device_attribute *attr, +const char *buf,size_t count) +{ +struct firmware_priv *fw_priv =dev_get_drvdata (dev); +long value; +int err; + + err =strict_strtol (buf,10 , &value); +if (err) +return err; + fw_priv->size_hint = value; +return count; +} + +static DEVICE_ATTR(data_size, 0644, + firmware_data_size_show, firmware_data_size_store); +static ssize_t firmware_loading_show (struct device *dev,struct device_attribute *attr,char *buf) {
@@ -207,6 +234,7 @@ fw_realloc_buffer(struct firmware_priv *if (min_size <= fw_priv->alloc_size)return 0 ; + min_size =max (min_size, fw_priv->size_hint); new_size =ALIGN (min_size,PAGE_SIZE ); new_data =vmalloc (new_size);if (!new_data) {
@@ -359,6 +387,12 @@ static int fw_setup_device(struct firmwa goto error_unreg; } + retval =device_create_file (f_dev, &dev_attr_data_size); +if (retval) { +dev_err (device,"%s: device_create_file failed\n" , __func__); + goto error_unreg; + } + retval =device_create_file (f_dev, &dev_attr_loading);if (retval) {dev_err (device,"%s: device_create_file failed\n" , __func__);
Quoted 8 lines · On Friday 03 of April 2009 10:46:11 Clemens Ladisch wrote:
Quoted 2 lines · Ira Snyder
I didn't want to change an existing kernel interface, so I just made the easiest change that worked for me.
Well, userspace does know the actual size of the image, so I see no reason why it shouldn't be able to tell the kernel about it beforehand.
Right; but it will need some time for user space to catch up.
Quoted 76 lines
Quoted 1 line
I'd be happy to test patches anyone comes up with.
=====
This adds a
data_sizeattribute to the firmware loading device so that userspace can tell us about the firmware image size. This allows us to preallocate the buffer with the final size, thus avoiding reallocating the buffer for every page of data as it comes in.Signed-off-by Clemens Ladisch <[email protected]>Documentation/firmware_class/README +1
-1@@ -21,7 +21,7 @@ kernel(driver): calls request_firmware(&fw_entry, $FIRMWARE, device) userspace: - - /sys/class/firmware/xxx/{loading,data} appear. + - /sys/class/firmware/xxx/{loading,data,data_size} appear. - hotplug gets called with a firmware identifier in $FIRMWARE and the usual hotplug environment.Documentation/firmware_class/README +1
-1- hotplug: echo 1 > /sys/class/firmware/xxx/loading@@ -29,11 +29,13 @@ kernel: Discard any previous partial load. userspace: + - hotplug: echo ... > /sys/class/firmware/xxx/data_size - hotplug: cat appropriate_firmware_image > \ /sys/class/firmware/xxx/data - kernel: grows a buffer in PAGE_SIZE increments to hold the image asit - comes in.
+ kernel: Copies the firmware image into a buffer of the specified
size. + If the image is larger, the buffer automatically grows in +PAGE_SIZEincrements.userspace:
Documentation/firmware_class/README +3
-0- hotplug: echo 0 > /sys/class/firmware/xxx/loading@@ -60,6 +62,9 @@ HOTPLUG_FW_DIR=/usr/lib/hotplug/firmware/ + if [ -e /sys/$DEVPATH/data_size ]; then + stat -c %s $HOTPLUG_FW_DIR/$FIRMWARE > /sys/$DEVPATH/data_size + fi echo 1 > /sys/$DEVPATH/loading cat $HOTPLUG_FW_DIR/$FIRMWARE > /sysfs/$DEVPATH/data echo 0 > /sys/$DEVPATH/loading@@ -73,6 +78,9 @@ - firmware_data_read() and firmware_loading_show() are justprovided for testing and completeness, they are not called in normal use.
+ -
/sys/class/firmware/xxx/data_sizeis optional for compatibility
with + older kernels.
+
- There is also/sys/class/firmware/timeoutwhich holds a timeout
in seconds for the whole load operation.Documentation/firmware_class/hotplug-script +3
-0@@ -6,6 +6,9 @@ HOTPLUG_FW_DIR=/usr/lib/hotplug/firmware/ +if [ -e /sys/$DEVPATH/data_size ]; then + stat -c %s $HOTPLUG_FW_DIR/$FIRMWARE > /sys/$DEVPATH/data_size +fi echo 1 > /sys/$DEVPATH/loading cat $HOTPLUG_FW_DIR/$FIRMWARE > /sys/$DEVPATH/data echo 0 > /sys/$DEVPATH/loading
drivers/base/firmware_class.c struct firmware_priv { · line 49
Quoted 4 lines
drivers/base/firmware_class.c +1
-0@@ -46,6 +46,7 @@ struct firmware_priv {struct firmware *fw;unsigned long status;int alloc_size; +int size_hint;
Unsigned?
Quoted 5 lines
drivers/base/firmware_class.c +0
-0struct timer_list timeout; };@@ -114,6 +115,32 @@ static struct class firmware_class = { .dev_release = fw_dev_release,
drivers/base/firmware_class.c static struct class firmware_class = { · line 125
Quoted 10 lines
drivers/base/firmware_class.c +8
-0}; +static ssize_t firmware_data_size_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ +struct firmware_priv *fw_priv =dev_get_drvdata (dev); + +return sprintf (buf,"%d\n" , fw_priv->size_hint); +} +
Why would you need it? It is no importance once firmware had been loaded. I'd personally consider it as write-only.
drivers/base/firmware_class.c static struct class firmware_class = { · line 137
Quoted 12 lines
drivers/base/firmware_class.c +12
-0+static ssize_t firmware_data_size_store (struct device *dev, +struct device_attribute *attr, +const char *buf,size_t count) +{ +struct firmware_priv *fw_priv =dev_get_drvdata (dev); +long value; +int err; + + err =strict_strtol (buf,10 , &value); +if (err) +return err; + fw_priv->size_hint = value;
Should not there be some protection against using silly large values?
Quoted 30 lines
drivers/base/firmware_class.c +13
-0+return count; +} + +static DEVICE_ATTR(data_size, 0644, + firmware_data_size_show, firmware_data_size_store); +static ssize_t firmware_loading_show (struct device *dev,struct device_attribute *attr,char *buf) {@@ -207,6 +234,7 @@ fw_realloc_buffer(struct firmware_priv *if (min_size <= fw_priv->alloc_size)return 0 ; + min_size =max (min_size, fw_priv->size_hint); new_size =ALIGN (min_size,PAGE_SIZE ); new_data =vmalloc (new_size);if (!new_data) {@@ -359,6 +387,12 @@ static int fw_setup_device(struct firmwa goto error_unreg; } + retval =device_create_file (f_dev, &dev_attr_data_size); +if (retval) { +dev_err (device,"%s: device_create_file failed\n" , __func__); + goto error_unreg; + } + retval =device_create_file (f_dev, &dev_attr_loading);if (retval) {dev_err (device,"%s: device_create_file failed\n" , __func__);
Quoted 6 lines · Andrey Borzenkov
Quoted 3 lines · On Friday 03 of April 2009 10:46:11 Clemens Ladisch wrote:
Well, userspace does know the actual size of the image, so I see no reason why it shouldn't be able to tell the kernel about it beforehand.
Right; but it will need some time for user space to catch up.
It's only an optimization.
Quoted 7 lines
drivers/base/firmware_class.c struct firmware_priv { · line 49
Quoted 5 lines
drivers/base/firmware_class.c +1
-0@@ -46,6 +46,7 @@ struct firmware_priv {struct firmware *fw;unsigned long status;int alloc_size; +int size_hint;Unsigned?
It is as signed as alloc_size. ;-)
Yes, all these variables probably should be size_t.
Quoted 9 lines
drivers/base/firmware_class.c static struct class firmware_class = { · line 124
Quoted 7 lines
drivers/base/firmware_class.c +7
-0+static ssize_t firmware_data_size_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ +struct firmware_priv *fw_priv =dev_get_drvdata (dev); + +return sprintf (buf,"%d\n" , fw_priv->size_hint); +}Why would you need it?
Good question. Apparently, for the same reason why we'd need firmware_data_read ...
Quoted 6 lines
Quoted 6 lines
drivers/base/firmware_class.c +6
-0+static ssize_t firmware_data_size_store (struct device *dev, +struct device_attribute *attr, +const char *buf,size_t count) +{ +struct firmware_priv *fw_priv =dev_get_drvdata (dev); +long value;
Quoted 8 lines
drivers/base/firmware_class.c static struct class firmware_class = { · line 137
Quoted 6 lines
drivers/base/firmware_class.c +6
-0+int err; + + err =strict_strtol (buf,10 , &value); +if (err) +return err; + fw_priv->size_hint = value;Should not there be some protection against using silly large values?
It is already possible to do "cat /dev/zero > .../data". What we'd need is a limit not only on this variable but on the size of the firmware image itself.
Okay, I'll do a bunch of patches to fix these warts in the firmware loader.
Best regards,
Clemens
Quoted 11 lines · Ira Snyder
Quoted 9 lines · Alan Cox
Quoted 3 lines
Some drivers cache firmware in memory. Doubing the amount of needed memory definitely would not be the best idea. Check
drivers/net/wirelessfor examples.A lot of drivers could perfectly happily exist with a simple iterator helper and being returned sg lists of pages. It seems that for big firmwares at least there is a root cause which is deeper than how you grow your vmalloc buffer.
Quoted 7 lines
An sg list of pages would be perfect for my usage. I didn't want to change an existing kernel interface, so I just made the easiest change that worked for me.
Another thing that could be done is trimming the
vmalloc()down to the exact size needed after the firmware has finished loading. That would still waste memory until the copy from userspace is finished, though.
Or you just allocate individual pages one at a time as you're copying the data in from userspace, then vmap() them once you're _done_.
We could potentially extend this to allow drivers to say "I only need an array of pages, not contiguous VM space", but that comes later.
This approach has the benefit that we can force the virtual mapping of the firmware image to be read-only -- one step up from making it 'const' as we did already.
Quoted 1 line
I'd be happy to test patches anyone comes up with.
Totally untested here...
drivers/base/firmware_class.c +99 -26
@@ -17,7 +17,7 @@#include <linux/bitops.h> #include <linux/mutex.h> #include <linux/kthread.h> - +#include <linux/highmem.h> #include <linux/firmware.h> #include "base.h"
@@ -45,7 +45,10 @@ struct firmware_priv {struct bin_attribute attr_data;struct firmware *fw;unsigned long status; -int alloc_size; +struct page **pages; +int nr_pages; +int page_array_size; +const char *vdata;struct timer_list timeout; };
@@ -141,6 +144,7 @@ static ssize_t firmware_loading_store(struct device *dev, {struct firmware_priv *fw_priv =dev_get_drvdata (dev);int loading =simple_strtol (buf,NULL ,10 ); +int i;switch (loading) {case 1 :
@@ -151,13 +155,30 @@ static ssize_t firmware_loading_store(struct device *dev, }vfree (fw_priv->fw->data); fw_priv->fw->data =NULL ; +for (i =0 ; i < fw_priv->nr_pages; i++) +__free_page (fw_priv->pages[i]); +kfree (fw_priv->pages); + fw_priv->pages =NULL ; + fw_priv->page_array_size =0 ; + fw_priv->nr_pages =0 ; fw_priv->fw->size =0 ; - fw_priv->alloc_size =0 ;set_bit (FW_STATUS_LOADING , &fw_priv->status);mutex_unlock (&fw_lock);break ;case 0 :if (test_bit (FW_STATUS_LOADING , &fw_priv->status)) { +vfree (fw_priv->fw->data); + fw_priv->fw->data =vmap (fw_priv->pages, + fw_priv->nr_pages, +0 ,PAGE_KERNEL_RO ); +if (!fw_priv->fw->data) { +dev_err (dev,"%s: vmap() failed\n" , __func__); + goto err; + } +/* Pages will be freed by vfree() */ + fw_priv->pages =NULL ; + fw_priv->page_array_size =0 ; + fw_priv->nr_pages =0 ;complete (&fw_priv->completion);clear_bit (FW_STATUS_LOADING , &fw_priv->status);break ;
@@ -167,6 +188,7 @@ static ssize_t firmware_loading_store(struct device *dev,dev_err (dev,"%s: unexpected value (%d)\n" , __func__, loading);/* fallthrough */ case -1 : + err:fw_load_abort (fw_priv);break ; }
@@ -191,8 +213,28 @@ firmware_data_read(struct kobject *kobj, struct bin_attribute *bin_attr, ret_count = -ENODEV ; goto out; } - ret_count =memory_read_from_buffer (buffer, count, &offset, - fw->data, fw->size); +if (offset > fw->size) +return 0 ; +if (count > fw->size - offset) + count = fw->size - offset; + + ret_count = count; + +while (count) { +void *page_data; +int page_nr = offset >>PAGE_SHIFT ; +int page_ofs = offset & (PAGE_SIZE -1 ); +int page_cnt =min (PAGE_SIZE - page_ofs,PAGE_SIZE ); + + page_data =kmap (fw_priv->pages[page_nr]); + +memcpy (buffer, page_data + page_ofs, page_cnt); + +kunmap (page_data); + buffer += page_cnt; + offset += page_cnt; + count -= page_cnt; + } out:mutex_unlock (&fw_lock);return ret_count;
@@ -201,27 +243,39 @@ out:static int fw_realloc_buffer (struct firmware_priv *fw_priv,int min_size) { -u8 *new_data; -int new_size = fw_priv->alloc_size; +int pages_needed =ALIGN (min_size,PAGE_SIZE ) >>PAGE_SHIFT ; + +/* If the array of pages is too small, grow it... */ +if (fw_priv->page_array_size < pages_needed) { +int new_array_size =min (pages_needed, + fw_priv->page_array_size *2 ); +struct page **new_pages; + + new_pages =kmalloc (new_array_size *sizeof (void *), +GFP_KERNEL ); +if (!new_pages) { +fw_load_abort (fw_priv); +return -ENOMEM ; + } +memcpy (new_pages, fw_priv->pages, + fw_priv->page_array_size *sizeof (void *)); +memset (&new_pages[fw_priv->page_array_size],0 ,sizeof (void *) * + (new_array_size - fw_priv->page_array_size)); +kfree (fw_priv->pages); + fw_priv->pages = new_pages; + fw_priv->page_array_size = new_array_size; + } -if (min_size <= fw_priv->alloc_size) -return 0 ; +while (fw_priv->nr_pages < pages_needed) { + fw_priv->pages[fw_priv->nr_pages] = +alloc_page (GFP_KERNEL | __GFP_HIGHMEM); - new_size =ALIGN (min_size,PAGE_SIZE ); - new_data =vmalloc (new_size); -if (!new_data) { -printk (KERN_ERR "%s: unable to alloc buffer\n" , __func__); -/* Make sure that we don't keep incomplete data */ -fw_load_abort (fw_priv); -return -ENOMEM ; - } - fw_priv->alloc_size = new_size; -if (fw_priv->fw->data) { -memcpy (new_data, fw_priv->fw->data, fw_priv->fw->size); -vfree (fw_priv->fw->data); +if (!fw_priv->pages[fw_priv->nr_pages]) { +fw_load_abort (fw_priv); +return -ENOMEM ; + } + fw_priv->nr_pages++; } - fw_priv->fw->data = new_data; -BUG_ON (min_size > fw_priv->alloc_size);return 0 ; }
@@ -258,10 +312,25 @@ firmware_data_write(struct kobject *kobj, struct bin_attribute *bin_attr,if (retval) goto out; -memcpy ((u8 *)fw->data + offset, buffer, count); - - fw->size =max_t (size_t, offset + count, fw->size); retval = count; + +while (count) { +void *page_data; +int page_nr = offset >>PAGE_SHIFT ; +int page_ofs = offset & (PAGE_SIZE -1 ); +int page_cnt =min (PAGE_SIZE - page_ofs,PAGE_SIZE ); + + page_data =kmap (fw_priv->pages[page_nr]); + +memcpy (page_data + page_ofs, buffer, page_cnt); + +kunmap (page_data); + buffer += page_cnt; + offset += page_cnt; + count -= page_cnt; + } + + fw->size =max_t (size_t, offset, fw->size); out:mutex_unlock (&fw_lock);return retval;
@@ -277,7 +346,11 @@ static struct bin_attribute firmware_attr_data_tmpl = {static void fw_dev_release (struct device *dev) {struct firmware_priv *fw_priv =dev_get_drvdata (dev); +int i; +for (i =0 ; i < fw_priv->nr_pages; i++) +__free_page (fw_priv->pages[i]); +kfree (fw_priv->pages);kfree (fw_priv);kfree (dev);
Signature · 2 lines
David Woodhouse Open Source Technology Centre [email protected] Intel Corporation
Rather than calling vmalloc() repeatedly to grow the firmware image as we receive data from userspace, just allocate and fill individual pages. Then vmap() the whole lot in one go when we're done.
A quick test with a 337KiB iwlagn firmware shows the time taken for request_firmware() going from ~32ms to ~5ms after I apply this patch.
---
Various failure modes and corner cases, including firmware_data_read(), are still untested.
drivers/base/firmware_class.c +99 -26
@@ -17,7 +17,7 @@#include <linux/bitops.h> #include <linux/mutex.h> #include <linux/kthread.h> - +#include <linux/highmem.h> #include <linux/firmware.h> #include "base.h"
@@ -45,7 +45,10 @@ struct firmware_priv {struct bin_attribute attr_data;struct firmware *fw;unsigned long status; -int alloc_size; +struct page **pages; +int nr_pages; +int page_array_size; +const char *vdata;struct timer_list timeout; };
@@ -141,6 +144,7 @@ static ssize_t firmware_loading_store(struct device *dev, {struct firmware_priv *fw_priv =dev_get_drvdata (dev);int loading =simple_strtol (buf,NULL ,10 ); +int i;switch (loading) {case 1 :
@@ -151,13 +155,30 @@ static ssize_t firmware_loading_store(struct device *dev, }vfree (fw_priv->fw->data); fw_priv->fw->data =NULL ; +for (i =0 ; i < fw_priv->nr_pages; i++) +__free_page (fw_priv->pages[i]); +kfree (fw_priv->pages); + fw_priv->pages =NULL ; + fw_priv->page_array_size =0 ; + fw_priv->nr_pages =0 ; fw_priv->fw->size =0 ; - fw_priv->alloc_size =0 ;set_bit (FW_STATUS_LOADING , &fw_priv->status);mutex_unlock (&fw_lock);break ;case 0 :if (test_bit (FW_STATUS_LOADING , &fw_priv->status)) { +vfree (fw_priv->fw->data); + fw_priv->fw->data =vmap (fw_priv->pages, + fw_priv->nr_pages, +0 ,PAGE_KERNEL_RO ); +if (!fw_priv->fw->data) { +dev_err (dev,"%s: vmap() failed\n" , __func__); + goto err; + } +/* Pages will be freed by vfree() */ + fw_priv->pages =NULL ; + fw_priv->page_array_size =0 ; + fw_priv->nr_pages =0 ;complete (&fw_priv->completion);clear_bit (FW_STATUS_LOADING , &fw_priv->status);break ;
@@ -167,6 +188,7 @@ static ssize_t firmware_loading_store(struct device *dev,dev_err (dev,"%s: unexpected value (%d)\n" , __func__, loading);/* fallthrough */ case -1 : + err:fw_load_abort (fw_priv);break ; }
@@ -191,8 +213,28 @@ firmware_data_read(struct kobject *kobj, struct bin_attribute *bin_attr, ret_count = -ENODEV ; goto out; } - ret_count =memory_read_from_buffer (buffer, count, &offset, - fw->data, fw->size); +if (offset > fw->size) +return 0 ; +if (count > fw->size - offset) + count = fw->size - offset; + + ret_count = count; + +while (count) { +void *page_data; +int page_nr = offset >>PAGE_SHIFT ; +int page_ofs = offset & (PAGE_SIZE -1 ); +int page_cnt =min (PAGE_SIZE - page_ofs, count); + + page_data =kmap (fw_priv->pages[page_nr]); + +memcpy (buffer, page_data + page_ofs, page_cnt); + +kunmap (page_data); + buffer += page_cnt; + offset += page_cnt; + count -= page_cnt; + } out:mutex_unlock (&fw_lock);return ret_count;
@@ -201,27 +243,39 @@ out:static int fw_realloc_buffer (struct firmware_priv *fw_priv,int min_size) { -u8 *new_data; -int new_size = fw_priv->alloc_size; +int pages_needed =ALIGN (min_size,PAGE_SIZE ) >>PAGE_SHIFT ; + +/* If the array of pages is too small, grow it... */ +if (fw_priv->page_array_size < pages_needed) { +int new_array_size =max (pages_needed, + fw_priv->page_array_size *2 ); +struct page **new_pages; + + new_pages =kmalloc (new_array_size *sizeof (void *), +GFP_KERNEL ); +if (!new_pages) { +fw_load_abort (fw_priv); +return -ENOMEM ; + } +memcpy (new_pages, fw_priv->pages, + fw_priv->page_array_size *sizeof (void *)); +memset (&new_pages[fw_priv->page_array_size],0 ,sizeof (void *) * + (new_array_size - fw_priv->page_array_size)); +kfree (fw_priv->pages); + fw_priv->pages = new_pages; + fw_priv->page_array_size = new_array_size; + } -if (min_size <= fw_priv->alloc_size) -return 0 ; +while (fw_priv->nr_pages < pages_needed) { + fw_priv->pages[fw_priv->nr_pages] = +alloc_page (GFP_KERNEL | __GFP_HIGHMEM); - new_size =ALIGN (min_size,PAGE_SIZE ); - new_data =vmalloc (new_size); -if (!new_data) { -printk (KERN_ERR "%s: unable to alloc buffer\n" , __func__); -/* Make sure that we don't keep incomplete data */ -fw_load_abort (fw_priv); -return -ENOMEM ; - } - fw_priv->alloc_size = new_size; -if (fw_priv->fw->data) { -memcpy (new_data, fw_priv->fw->data, fw_priv->fw->size); -vfree (fw_priv->fw->data); +if (!fw_priv->pages[fw_priv->nr_pages]) { +fw_load_abort (fw_priv); +return -ENOMEM ; + } + fw_priv->nr_pages++; } - fw_priv->fw->data = new_data; -BUG_ON (min_size > fw_priv->alloc_size);return 0 ; }
@@ -258,10 +312,25 @@ firmware_data_write(struct kobject *kobj, struct bin_attribute *bin_attr,if (retval) goto out; -memcpy ((u8 *)fw->data + offset, buffer, count); - - fw->size =max_t (size_t, offset + count, fw->size); retval = count; + +while (count) { +void *page_data; +int page_nr = offset >>PAGE_SHIFT ; +int page_ofs = offset & (PAGE_SIZE -1 ); +int page_cnt =min (PAGE_SIZE - page_ofs, count); + + page_data =kmap (fw_priv->pages[page_nr]); + +memcpy (page_data + page_ofs, buffer, page_cnt); + +kunmap (page_data); + buffer += page_cnt; + offset += page_cnt; + count -= page_cnt; + } + + fw->size =max_t (size_t, offset, fw->size); out:mutex_unlock (&fw_lock);return retval;
@@ -277,7 +346,11 @@ static struct bin_attribute firmware_attr_data_tmpl = {static void fw_dev_release (struct device *dev) {struct firmware_priv *fw_priv =dev_get_drvdata (dev); +int i; +for (i =0 ; i < fw_priv->nr_pages; i++) +__free_page (fw_priv->pages[i]); +kfree (fw_priv->pages);kfree (fw_priv);kfree (dev);
Signature · 2 lines
David Woodhouse Open Source Technology Centre [email protected] Intel Corporation