dfu: samsung: Commits to allow sending and storing large files via DFU
cover dfu: samsung: Commits to allow sending and storing large files via DFU
Those commits allow sending and storing large files (e.g. 26 MiB) on the Odroid XU3 board.
On purpose they are sent together to shed light on the process of tunning board to be capable of large files transfer.
I would be _REALLY_ grateful for excessive testing, since I'm not able to test this code on NAND based devices. Please support me and test this patch for potential regressions.
This change applies on the u-boot-master tree.
SHA1: aada3d062a0cacde1e783af2ac560f2a0fdf3682
Lukasz Majewski (2):
dfu: usb: f_dfu: Set deferred call for dfu_flush() function
dfu: odroid xu3: Define DFU_MANIFEST_POLL_TIMEOUT to handle large
files transmission and storage
common/cmd_dfu.c | 20 ++++++++++++++++++++ drivers/usb/gadget/f_dfu.c | 11 +++-------- include/configs/odroid_xu3.h | 1 + include/dfu.h | 25 +++++++++++++++++++++++++ 4 files changed, 49 insertions(+), 8 deletions(-)
Signature · 1 line
2.0.0.rc2
PATCH 1/2 dfu: usb: f_dfu: Set deferred call for dfu_flush() function
This patch fixes situation when one would like to write large file into medium with the file system (fat, ext4, etc).
This change sets file size limitation to the DFU internal buffer size.
Since u-boot is not supporting interrupts and seek on file systems, it becomes challenging to store large file appropriately.
To reproduce this error - create large file (around 26 MiB) and sent it to the target board.
Lets examine the flow of USB transactions:
0. DFU uses EP0 with 64B MPS [Max Packet Size]
1. Send file - OUT (PC->target) - dat_26MiB.img is sent with 4096 B transactions
2. Get status - OUT (PC->target) - wait for DFU_STATE_dfuDNLOAD_IDLE (0x05) sent
from target board - IN transaction (target->PC)
3. The whole file content is sent to target - OUT (PC->target) with ZLP [Zero Length Packet]
Now the interesting part starts:
4. OUT (PC->target) Setup transaction (request to share DFU state)
5. IN (target->PC) - reply the current DFU state
- In the UDC driver the req->completion (with dfu_flush) is called after successful IN transfer.
- The dfu_flush() (called from req->completion callback) saves the whole file at once (u-boot doesn't support seek on fs). Such operation takes considerable time. When the file is large - e.g. 26MiB - this time may be more than 5 seconds.
6. OUT (PC->target) - ZLP, is send in the same time when dfu_flush() writes data to eMMC memory.
The dfu-util application has hard coded timeout on USB transaction completion set to 5 seconds (it uses libusb calls).
When the file to store is large (e.g. 26 MiB) the time needed to write it may excess the dfu-util timeout and following error message will be displayed:
"unable to read DFU status" on the HOST PC console.
This change is supposed to leverage DFU's part responsible for storing files on file systems. Other DFU operations - i.e. raw/partition write to NAND and eMMC should work as before.
The only functional change is the error reporting. When dfu_flush() fails the u-boot prompt will exit with error information and dfu-util application exits afterwards as well.
Test HW:
- Odroid XU3 (Exynos5433) - test with large file
- Trats (Exynos4210) - test for regression - eMMC, raw,
--- common/cmd_dfu.c | 20 ++++++++++++++++++++ drivers/usb/gadget/f_dfu.c | 11 +++-------- include/dfu.h | 25 +++++++++++++++++++++++++ 3 files changed, 48 insertions(+), 8 deletions(-)
common/cmd_dfu.c +20 -0
@@ -79,6 +79,26 @@ static int do_dfu(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])if (ctrlc ()) goto exit; +if (dfu_get_defer_flush ()) { +/* +* Call to usb_gadget_handle_interrupts() is necessary +* to act on ZLP OUT transaction from HOST PC after +* transmitting the whole file. +* +* If this ZLP OUT packet is NAK'ed, the HOST libusb +* function fails after timeout (by default it is set to +* 5 seconds). In such situation the dfu-util program +* exits with error message. +*/ +usb_gadget_handle_interrupts (controller_index); + ret =dfu_flush (dfu_get_defer_flush (),NULL ,0 ,0 ); +dfu_set_defer_flush (NULL ); +if (ret) { +error ("Deferred dfu_flush() failed!" ); + goto exit; + } + } +WATCHDOG_RESET ();usb_gadget_handle_interrupts (controller_index); }
drivers/usb/gadget/f_dfu.c +3 -8
@@ -44,6 +44,8 @@ struct f_dfu {unsigned int poll_timeout; }; +struct dfu_entity *dfu_defer_flush; + typedef int (*dfu_state_fn) (struct f_dfu *, const struct usb_ctrlrequest *, struct usb_gadget *,
@@ -167,14 +169,7 @@ static void dnload_request_complete(struct usb_ep *ep, struct usb_request *req)static void dnload_request_flush (struct usb_ep *ep,struct usb_request *req) {struct f_dfu *f_dfu = req->context; -int ret; - - ret =dfu_flush (dfu_get_entity (f_dfu->altsetting), req->buf, - req->length, f_dfu->blk_seq_num); -if (ret) { - f_dfu->dfu_status = DFU_STATUS_errUNKNOWN; - f_dfu->dfu_state = DFU_STATE_dfuERROR; - } +dfu_set_defer_flush (dfu_get_entity (f_dfu->altsetting)); }static inline int dfu_get_manifest_timeout (struct dfu_entity *dfu)
include/dfu.h +25 -0
@@ -163,6 +163,31 @@ int dfu_read(struct dfu_entity *de, void *buf, int size, int blk_seq_num);int dfu_write (struct dfu_entity *de,void *buf,int size,int blk_seq_num);int dfu_flush (struct dfu_entity *de,void *buf,int size,int blk_seq_num); +/* +* dfu_defer_flush - pointer to store dfu_entity for deferred flashing. +* It should be NULL when not used. +*/ +extern struct dfu_entity *dfu_defer_flush; +/** +* dfu_get_defer_flush - get current value of dfu_defer_flush pointer +* +* @return - value of the dfu_defer_flush pointer +*/ +static inline struct dfu_entity *dfu_get_defer_flush (void ) +{ +return dfu_defer_flush; +} + +/** +* dfu_set_defer_flush - set the dfu_defer_flush pointer +* +* @param dfu - pointer to the dfu_entity, which should be written +*/ +static inline void dfu_set_defer_flush (struct dfu_entity *dfu) +{ + dfu_defer_flush = dfu; +} +/** * dfu_write_from_mem_addr - write data from memory to DFU managed medium *
Signature · 1 line
2.0.0.rc2
PATCH 1/2 dfu: usb: f_dfu: Set deferred call for dfu_flush() function
Quoted 3 lines · Lukasz Majewski
This patch fixes situation when one would like to write large file into medium with the file system (fat, ext4, etc).
This change sets file size limitation to the DFU internal buffer size.
I tested this on NVIDIA's Jetson TX1 using test/py's test_dfu, with existing transfer sizes (63 bytes ... 8MB) rather than attempting to do anything with large files.
PATCH 1/2 dfu: usb: f_dfu: Set deferred call for dfu_flush() function
In reply to Lukasz MajewskiHello Lukasz,
Am 28.01.2016 um 17:46 schrieb Lukasz Majewski:
Quoted 56 lines
This patch fixes situation when one would like to write large file into medium with the file system (fat, ext4, etc).
This change sets file size limitation to the DFU internal buffer size.Since u-boot is not supporting interrupts and seek on file systems, it becomes challenging to store large file appropriately.
To reproduce this error - create large file (around 26 MiB) and sent it to the target board.
Lets examine the flow of USB transactions:
0. DFU uses EP0 with 64B MPS [Max Packet Size]
1. Send file - OUT (
PC->target) - dat_26MiB.img is sent with 4096 B transactions2. Get status - OUT (
PC->target) - wait forDFU_STATE_dfuDNLOAD_IDLE(0x05) sentfrom target board - IN transaction (target->PC)3. The whole file content is sent to target - OUT (
PC->target) with ZLP [Zero Length Packet]Now the interesting part starts:
4. OUT (
PC->target) Setup transaction (request to share DFU state)5. IN (
target->PC) - reply the current DFU state
- In the UDC driver thereq->completion(withdfu_flush) is called after successful IN transfer.
- Thedfu_flush()(called fromreq->completioncallback) saves the whole file at once (u-boot doesn't support seek on fs). Such operation takes considerable time. When the file is large - e.g. 26MiB - this time may be more than 5 seconds.6. OUT (
PC->target) - ZLP, is send in the same time whendfu_flush()writes data to eMMC memory.
The dfu-util application has hard coded timeout on USB transaction completion set to 5 seconds (it uses libusb calls).When the file to store is large (e.g. 26 MiB) the time needed to write it may excess the dfu-util timeout and following error message will be displayed:
"unable to read DFU status" on the HOST PC console.This change is supposed to leverage DFU's part responsible for storing files on file systems. Other DFU operations - i.e. raw/partition write to NAND and eMMC should work as before.
The only functional change is the error reporting. When
dfu_flush()fails the u-boot prompt will exit with error information and dfu-util application exits afterwards as well.Test HW:
- Odroid XU3 (Exynos5433) - test with large file
- Trats (Exynos4210) - test for regression - eMMC, raw,
Quoted 7 lines
Signed-off-by Lukasz Majewski <[email protected]>Reported-by Alex Gdalevich <[email protected]>--- common/cmd_dfu.c | 20 ++++++++++++++++++++ drivers/usb/gadget/f_dfu.c | 11 +++-------- include/dfu.h | 25 +++++++++++++++++++++++++ 3 files changed, 48 insertions(+), 8 deletions(-)
Tested on the dxr2 board with an etamin module containing a 4GiB DDP nand with a download size from 400MiB ... worked.
(This test is not yet in my nightly builds ...)
bye,
Heiko
BTW:
I used not the "dfu_nand" module, instead I am just developing a "dfu_mtd" for accessing mtd partitions. So mtd partitions with underlying mtd concatenated nand devices for example can be programmed with dfu ... seems first version works now ...
Quoted 98 lines
common/cmd_dfu.c +20
-0@@ -79,6 +79,26 @@ static int do_dfu(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])if (ctrlc ()) goto exit; +if (dfu_get_defer_flush ()) { +/* +* Call to usb_gadget_handle_interrupts() is necessary +* to act on ZLP OUT transaction from HOST PC after +* transmitting the whole file. +* +* If this ZLP OUT packet is NAK'ed, the HOST libusb +* function fails after timeout (by default it is set to +* 5 seconds). In such situation the dfu-util program +* exits with error message. +*/ +usb_gadget_handle_interrupts (controller_index); + ret =dfu_flush (dfu_get_defer_flush (),NULL ,0 ,0 ); +dfu_set_defer_flush (NULL ); +if (ret) { +error ("Deferred dfu_flush() failed!" ); + goto exit; + } + } +WATCHDOG_RESET ();usb_gadget_handle_interrupts (controller_index); }drivers/usb/gadget/f_dfu.c +3
-8@@ -44,6 +44,8 @@ struct f_dfu {unsigned int poll_timeout; }; +struct dfu_entity *dfu_defer_flush; + typedef int (*dfu_state_fn) (struct f_dfu *, const struct usb_ctrlrequest *, struct usb_gadget *,@@ -167,14 +169,7 @@ static void dnload_request_complete(struct usb_ep *ep, struct usb_request *req)static void dnload_request_flush (struct usb_ep *ep,struct usb_request *req) {struct f_dfu *f_dfu = req->context; -int ret; - - ret =dfu_flush (dfu_get_entity (f_dfu->altsetting), req->buf, - req->length, f_dfu->blk_seq_num); -if (ret) { - f_dfu->dfu_status = DFU_STATUS_errUNKNOWN; - f_dfu->dfu_state = DFU_STATE_dfuERROR; - } +dfu_set_defer_flush (dfu_get_entity (f_dfu->altsetting)); }static inline int dfu_get_manifest_timeout (struct dfu_entity *dfu)include/dfu.h +25
-0@@ -163,6 +163,31 @@ int dfu_read(struct dfu_entity *de, void *buf, int size, int blk_seq_num);int dfu_write (struct dfu_entity *de,void *buf,int size,int blk_seq_num);int dfu_flush (struct dfu_entity *de,void *buf,int size,int blk_seq_num); +/* +* dfu_defer_flush - pointer to store dfu_entity for deferred flashing. +* It should be NULL when not used. +*/ +extern struct dfu_entity *dfu_defer_flush; +/** +* dfu_get_defer_flush - get current value of dfu_defer_flush pointer +* +* @return - value of the dfu_defer_flush pointer +*/ +static inline struct dfu_entity *dfu_get_defer_flush (void ) +{ +return dfu_defer_flush; +} + +/** +* dfu_set_defer_flush - set the dfu_defer_flush pointer +* +* @param dfu - pointer to the dfu_entity, which should be written +*/ +static inline void dfu_set_defer_flush (struct dfu_entity *dfu) +{ + dfu_defer_flush = dfu; +} +/** * dfu_write_from_mem_addr - write data from memory to DFU managed medium *
Signature · 2 lines
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
PATCH 1/2 dfu: usb: f_dfu: Set deferred call for dfu_flush() function
Hi Heiko,
Quoted 71 lines
Hello Lukasz,
Am 28.01.2016 um 17:46 schrieb Lukasz Majewski:
Quoted 60 lines
This patch fixes situation when one would like to write large file into medium with the file system (fat, ext4, etc).
This change sets file size limitation to the DFU internal buffer size.Since u-boot is not supporting interrupts and seek on file systems, it becomes challenging to store large file appropriately.
To reproduce this error - create large file (around 26 MiB) and sent it to the target board.
Lets examine the flow of USB transactions:
0. DFU uses EP0 with 64B MPS [Max Packet Size]
1. Send file - OUT (
PC->target) - dat_26MiB.img is sent with 4096 B transactions2. Get status - OUT (
PC->target) - wait forDFU_STATE_dfuDNLOAD_IDLE(0x05) sent from target board - IN transaction (target->PC)3. The whole file content is sent to target - OUT (
PC->target) with ZLP [Zero Length Packet]Now the interesting part starts:
4. OUT (
PC->target) Setup transaction (request to share DFU state)5. IN (
target->PC) - reply the current DFU state
- In the UDC driver thereq->completion(withdfu_flush) is
called after successful IN transfer.
- Thedfu_flush()(called fromreq->completioncallback)
saves the whole file at once (u-boot doesn't support seek on fs).
Such operation takes considerable time. When the file is large - e.g. 26MiB - this time may be more than 5
seconds.6. OUT (
PC->target) - ZLP, is send in the same time whendfu_flush()writes data to eMMC memory.
The dfu-util application has hard coded timeout on USB transaction completion set to 5 seconds (it uses libusb calls).When the file to store is large (e.g. 26 MiB) the time needed to write it may excess the dfu-util timeout and following error message will be displayed: "unable to read DFU status" on the HOST PC console.
This change is supposed to leverage DFU's part responsible for storing files on file systems. Other DFU operations - i.e. raw/partition write to NAND and eMMC should work as before.
The only functional change is the error reporting. When
dfu_flush()fails the u-boot prompt will exit with error information and dfu-util application exits afterwards as well.Test HW:
- Odroid XU3 (Exynos5433) - test with large file
- Trats (Exynos4210) - test for regression - eMMC, raw,Quoted 7 lines
Signed-off-by Lukasz Majewski <[email protected]>Reported-by Alex Gdalevich <[email protected]>--- common/cmd_dfu.c | 20 ++++++++++++++++++++ drivers/usb/gadget/f_dfu.c | 11 +++-------- include/dfu.h | 25 +++++++++++++++++++++++++ 3 files changed, 48 insertions(+), 8 deletions(-)
Quoted 5 lines
Tested on the dxr2 board with an etamin module containing a 4GiB DDP nand with a download size from 400MiB ... worked.
(This test is not yet in my nightly builds ...)Tested-by Heiko Schocher <[email protected]>
Thanks for testing!
Quoted 8 lines
bye,
Heiko
BTW:
I used not the "dfu_nand" module, instead I am just developing a "dfu_mtd" for accessing mtd partitions. So mtd partitions with underlying mtd concatenated nand devices for example can be programmed with dfu ... seems first version works now ...
Very nice to hear. I'm looking forward for your patches for review.
BTW. I do plan to setup tbot for my private Beagle Bone Black in the weekend.
Quoted 112 lines
Quoted 111 lines
common/cmd_dfu.c +3
-0@@ -79,6 +79,26 @@ static int do_dfu(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) if (ctrlc()) goto exit; +if (dfu_get_defer_flush ()) { +/* +* Call to usb_gadget_handle_interrupts() is necessary
common/cmd_dfu.c +1
-0+* to act on ZLP OUT transaction from HOST PC after
common/cmd_dfu.c +3
-0+* transmitting the whole file. +* +* If this ZLP OUT packet is NAK'ed, the HOST libusb
common/cmd_dfu.c +1
-0+* function fails after timeout (by default it is set to
common/cmd_dfu.c +1
-0+* 5 seconds). In such situation the dfu-util program
common/cmd_dfu.c +3
-0+* exits with error message. +*/ +
usb_gadget_handle_interrupts(controller_index);common/cmd_dfu.c +1
-0+ ret = dfu_flush(dfu_get_defer_flush(),NULL, 0, 0);
common/cmd_dfu.c +3
-0+dfu_set_defer_flush (NULL ); +if (ret) { + error("Deferred dfu_flush()failed!");
common/cmd_dfu.c +4
-0+ goto exit; + } + } +WATCHDOG_RESET ();usb_gadget_handle_interrupts (controller_index); }drivers/usb/gadget/f_dfu.c +2
-0@@ -44,6 +44,8 @@ struct f_dfu {unsigned int poll_timeout; }; +struct dfu_entity *dfu_defer_flush; + typedef int (*dfu_state_fn) (struct f_dfu *, const struct usb_ctrlrequest *, struct usb_gadget *,@@ -167,14 +169,7 @@ static void dnload_request_complete(struct
usb_ep*ep, structusb_request*req) static voiddnload_request_flush(struct usb_ep *ep, struct usb_request *req){
structf_dfu*f_dfu=req->context;drivers/usb/gadget/f_dfu.c +0
-3-int ret; - - ret = dfu_flush(dfu_get_entity(f_dfu->altsetting),
req->buf,drivers/usb/gadget/f_dfu.c +1
-5- req->length, f_dfu->blk_seq_num); -if (ret) { - f_dfu->dfu_status = DFU_STATUS_errUNKNOWN; - f_dfu->dfu_state = DFU_STATE_dfuERROR; - } +dfu_set_defer_flush (dfu_get_entity (f_dfu->altsetting)); }static inline int dfu_get_manifest_timeout (struct dfu_entity *dfu)include/dfu.h +0
-0@@ -163,6 +163,31 @@ int dfu_read(struct dfu_entity *de, void *buf, int size, int blk_seq_num); int dfu_write(struct dfu_entity *de,void *buf, int size, int
blk_seq_num); intdfu_flush(struct dfu_entity *de, void *buf, int size, int blk_seq_num);+/*
+ *dfu_defer_flush- pointer to storedfu_entityfor deferred flashing.
+ * It should be NULL when not used.
+ */include/dfu.h +3
-0+extern struct dfu_entity *dfu_defer_flush; +/** +* dfu_get_defer_flush - get current value of dfu_defer_flush pointer
+ *
+ * @return - value of thedfu_defer_flushpointer
+ */include/dfu.h +15
-0+static inline struct dfu_entity *dfu_get_defer_flush (void ) +{ +return dfu_defer_flush; +} + +/** +* dfu_set_defer_flush - set the dfu_defer_flush pointer +* +* @param dfu - pointer to the dfu_entity, which should be written +*/ +static inline void dfu_set_defer_flush (struct dfu_entity *dfu) +{ + dfu_defer_flush = dfu; +} +/** * dfu_write_from_mem_addr - write data from memory to DFU managed medium *
Signature · 5 lines
Best regards, Lukasz Majewski Samsung R&D Institute Poland (SRPOL) | Linux Platform Group
PATCH 2/2 dfu: odroid xu3: Define DFU_MANIFEST_POLL_TIMEOUT to handle large files transmission and storage
In reply to Lukasz MajewskiAs tested on the Odroid XU3, large files to be stored on the file system require considerable time to be physically written to the medium.
The default 300 ms is not enough to store large file (e.g. 26 MiB). To fix this situation the DFU_MANIFEST_POLL_TIMEOUT has been defined.
It is used to cease the communication with dfu-util and allow the target board to store the data on file system.
--- include/configs/odroid_xu3.h | 1 + 1 file changed, 1 insertion(+)
include/configs/odroid_xu3.h +1 -0
@@ -69,6 +69,7 @@#define CONFIG_CMD_DFU #define CONFIG_SYS_DFU_DATA_BUF_SIZE SZ_32M#define DFU_DEFAULT_POLL_TIMEOUT 300 +#define DFU_MANIFEST_POLL_TIMEOUT 25000/* THOR */ #define CONFIG_G_DNL_THOR_VENDOR_NUM CONFIG_G_DNL_VENDOR_NUM
Signature · 1 line
2.0.0.rc2
cover dfu: samsung: Commits to allow sending and storing large files via DFU
In reply to Lukasz MajewskiHello Lukasz,
Am 28.01.2016 um 17:46 schrieb Lukasz Majewski:
Quoted 16 lines
Those commits allow sending and storing large files (e.g. 26 MiB) on the Odroid XU3 board.
On purpose they are sent together to shed light on the process of tunning board to be capable of large files transfer.
I would be _REALLY_ grateful for excessive testing, since I'm not able to test this code on NAND based devices. Please support me and test this patch for potential regressions.
This change applies on the u-boot-master tree.
SHA1:aada3d062a0cacde1e783af2ac560f2a0fdf3682Lukasz Majewski (2):
dfu: usb:f_dfu: Set deferred call fordfu_flush()function
dfu: odroid xu3: DefineDFU_MANIFEST_POLL_TIMEOUTto handle large
files transmission and storage
Quoted 5 lines
common/cmd_dfu.c | 20 ++++++++++++++++++++ drivers/usb/gadget/f_dfu.c | 11 +++-------- include/configs/odroid_xu3.h | 1 + include/dfu.h | 25 +++++++++++++++++++++++++ 4 files changed, 49 insertions(+), 8 deletions(-)
Tested on the smartweb board (NAND):
http://xeidos.ddns.net/buildbot/builders/smartweb_dfu/builds/49/steps/shell/logs/tbotlog
with current U-Boot mainline.
bye,
Heiko
Signature · 2 lines
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany