Skip to contentSkip to conversation

usb: gadget: f_dfu: write req->actual bytes

PATCH New Felipe Balbi 3 participants 10 messages Feb 13, 2017 – Feb 22, 2017 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

PATCH 1/1 usb: gadget: f_dfu: write req->actual bytes

Hi Lukasz,

Quoted 20 lines · Lukasz Majewski

Hi Felipe,

Thanks for the patch.
Please see my comments below.

On 13 Feb 2017 11:42 am, Felipe Balbi <[email protected]> wrote:

Hi,

Quoted 10 lines · Marek Vasut
Quoted 8 lines · Andy Shevchenko

From: Felipe Balbi <[email protected]>

If last packet is short, we shouldn't write req->length bytes to non-volatile media, we should write only what's available to us, which is held in req->actual.

Quoted 6 lines
Quoted 1 line

Since I have no clue about DFU internals, I will wait for Lukasz's Ack.

you don't need to have any clues about DFU internals to realise that this fixes an actual bug, see below:

I don't know your exact use case. Please keep in mind that we most

eMMC :-)

Quoted 1 line

work on NAND and eMMC, which require the whole block write.

Well, then the file should have been padded already before sending it over USB, right? :-)

You shouldn't write req->length if you don't receive req->length as you are, potentially, writing garbage to the storage medium :-)

Quoted 2 lines

However, I will setup test environment (after changing the job it was gone), test your patch and then let you know.

cool

Quoted 47 lines
Quoted 10 lines
Quoted 10 lines

drivers/usb/gadget/f_dfu.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-) 
drivers/usb/gadget/f_dfu.c +0 -0
index 8e7c981657..64cdfa7c98 100644 
@@ -159,7 +159,7 @@ static void dnload_request_complete(struct usb_ep *ep, struct usb_request *req) 
 int ret; 
 
Quoted 3 lines

drivers/usb/gadget/f_dfu.c static void dnload_request_complete(struct usb_ep *ep, struct usb_request *req) · line 162

Quoted 3 lines
drivers/usb/gadget/f_dfu.c +1 -1
 ret = dfu_write(dfu_get_entity(f_dfu->altsetting), req->buf, 
- req->length, f_dfu->blk_seq_num); 
+ req->actual, f_dfu->blk_seq_num); 

DFU driver queues a request to USB controller. Per the gadget API req->length contains maximum amount of data to be transmitted. req->actual is written by USB controller with the actual amount of data that we transmitted.

In the case of IN (TX), upon completion req->length and req->actual should always be equal (unless errors show up, etc)

In the case of OUT (RX), upon completion req->actual MAY BE less than req->length and that's not an error. Say host sent us a short packet which causes early termination of transfer.

With that in mind, let's consider the situation where we're receiving data from host using DFU. Let's assume that we have a 4096 byte buffer for transfers and we're receiving a binary that's 7679 bytes in size.

Here's what we will do (pseudo-code):

int remaining = 7679;
char buf[4096];

while (remaining) {
req->length = 4096;
req->buf = buf;
usb_ep_queue(req);

/* wait for completion */

remaining -= req->actual;

dfu_write(buf, req->length); /* this is the error */ }

Quoted 7 lines

Can you see here that in the last packet we will write 4096 bytes when we should write only 3583?

In principle you are right. I need to check if this change will not introduce regressions.

Can you share your use case?

Intel Edison running v2017.03-rc1 + patches (see [1]), flashing u-boot.bin over DFU (see [2] for details). Without $subject, image has to be aligned to 4096 bytes as below:

$ dd if=u-boot.bin of=u-boot-4k.bin bs=4k seek=1 && truncate -s %4096 u-boot-4k.bin

With $subject, I don't need truncate. We still need the 4096 byte of zeroes in the beginning of the image for other reasons (which I really don't know why at this point).

[1] https://github.com/andy-shev/u-boot/tree/edison
[2] https://communities.intel.com/message/435516#435516

Signature · 8 lines
balbi
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170213/233a9b20/attachment.sig>
#2 lore raw

PATCH 1/1 usb: gadget: f_dfu: write req->actual bytes

Hi Felipe,

Quoted 29 lines

Hi Lukasz,

Quoted 25 lines · Lukasz Majewski

Hi Felipe,

Thanks for the patch.
Please see my comments below.

On 13 Feb 2017 11:42 am, Felipe Balbi
<[email protected]> wrote:

Hi,

Quoted 13 lines · Marek Vasut
Quoted 9 lines · Andy Shevchenko

From: Felipe Balbi <[email protected]>

If last packet is short, we shouldn't write req->length bytes to non-volatile media, we should write only what's available to us, which is held in req->actual.

Signed-off-by Andy Shevchenko

<[email protected]>

Since I have no clue about DFU internals, I will wait for Lukasz's Ack.

Quoted 6 lines
Quoted 4 lines

you don't need to have any clues about DFU internals to realise
that this fixes an actual bug, see below:

I don't know your exact use case. Please keep in mind that we most

eMMC :-)

Ok.

Quoted 8 lines
Quoted 2 lines

work on NAND and eMMC, which require the whole block write.

Well, then the file should have been padded already before sending it over USB, right? :-)

You shouldn't write req->length if you don't receive req->length as you are, potentially, writing garbage to the storage medium :-)

Yes. Correct....

Quoted 69 lines
Quoted 3 lines

However, I will setup test environment (after changing the job it was gone), test your patch and then let you know.

cool

Quoted 48 lines
Quoted 10 lines
Quoted 10 lines

drivers/usb/gadget/f_dfu.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-) 
(quoted diff) +0 -0

b/drivers/usb/gadget/f_dfu.c index 8e7c981657..64cdfa7c98 100644 --- a/drivers/usb/gadget/f_dfu.c
+++ b/drivers/usb/gadget/f_dfu.c

(quoted diff) +0 -0
@@ -159,7 +159,7 @@ static void dnload_request_complete(struct
 usb_ep *ep, struct usb_request *req) int ret; 
 
Quoted 3 lines

Review static void dnload_request_complete(struct · line 162

Quoted 3 lines
(quoted diff) +1 -1
 ret = dfu_write(dfu_get_entity(f_dfu->altsetting), req->buf, 
- req->length, f_dfu->blk_seq_num); 
+ req->actual, f_dfu->blk_seq_num); 

DFU driver queues a request to USB controller. Per the gadget API req->length contains maximum amount of data to be transmitted. req->actual is written by USB controller with the
actual amount of data that we transmitted.

In the case of IN (TX), upon completion req->length and
req->actual should always be equal (unless errors show up, etc)

In the case of OUT (RX), upon completion req->actual MAY BE less
than req->length and that's not an error. Say host sent us a short packet which causes early termination of transfer.

With that in mind, let's consider the situation where we're
receiving data from host using DFU. Let's assume that we have a 4096 byte buffer for transfers and we're receiving a binary that's 7679 bytes in size.

Here's what we will do (pseudo-code):

int remaining = 7679;
char buf[4096];

while (remaining) {
req->length = 4096;
req->buf = buf;
usb_ep_queue(req);

/* wait for completion */

remaining -= req->actual;

dfu_write(buf, req->length); /* this is the error */ }

Quoted 7 lines

Can you see here that in the last packet we will write 4096 bytes
when we should write only 3583?

In principle you are right. I need to check if this change will not introduce regressions.

Can you share your use case?

Intel Edison running v2017.03-rc1 + patches (see [1]), flashing u-boot.bin over DFU (see [2] for details). Without $subject, image has to be aligned to 4096 bytes as below:

$ dd if=u-boot.bin of=u-boot-4k.bin bs=4k seek=1 && truncate -s %4096
u-boot-4k.bin
Quoted 7 lines

With $subject, I don't need truncate. We still need the 4096 byte of zeroes in the beginning of the image for other reasons (which I really don't know why at this point).

[1] https://github.com/andy-shev/u-boot/tree/edison
[2] https://communities.intel.com/message/435516#435516

Ok. I will check this. Thanks for pointing out :-)

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de -------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170213/b8a5fe80/attachment.sig>

#4 lore raw

PATCH 1/1 usb: gadget: f_dfu: write req->actual bytes

In reply to Lukasz Majewski

Hi Lukasz,

Quoted 67 lines · Lukasz Majewski
Quoted 67 lines
Quoted 48 lines
Quoted 10 lines
Quoted 10 lines

drivers/usb/gadget/f_dfu.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-) 
(quoted diff) +0 -0

b/drivers/usb/gadget/f_dfu.c index 8e7c981657..64cdfa7c98 100644 --- a/drivers/usb/gadget/f_dfu.c
+++ b/drivers/usb/gadget/f_dfu.c

(quoted diff) +0 -0
@@ -159,7 +159,7 @@ static void dnload_request_complete(struct
 usb_ep *ep, struct usb_request *req) int ret; 
 
Quoted 3 lines

Review static void dnload_request_complete(struct · line 162

Quoted 3 lines
(quoted diff) +1 -1
 ret = dfu_write(dfu_get_entity(f_dfu->altsetting), req->buf, 
- req->length, f_dfu->blk_seq_num); 
+ req->actual, f_dfu->blk_seq_num); 

DFU driver queues a request to USB controller. Per the gadget API req->length contains maximum amount of data to be transmitted. req->actual is written by USB controller with the
actual amount of data that we transmitted.

In the case of IN (TX), upon completion req->length and
req->actual should always be equal (unless errors show up, etc)

In the case of OUT (RX), upon completion req->actual MAY BE less
than req->length and that's not an error. Say host sent us a short packet which causes early termination of transfer.

With that in mind, let's consider the situation where we're
receiving data from host using DFU. Let's assume that we have a 4096 byte buffer for transfers and we're receiving a binary that's 7679 bytes in size.

Here's what we will do (pseudo-code):

int remaining = 7679;
char buf[4096];

while (remaining) {
req->length = 4096;
req->buf = buf;
usb_ep_queue(req);

/* wait for completion */

remaining -= req->actual;

dfu_write(buf, req->length); /* this is the error */ }

Quoted 7 lines

Can you see here that in the last packet we will write 4096 bytes
when we should write only 3583?

In principle you are right. I need to check if this change will not introduce regressions.

Can you share your use case?

Intel Edison running v2017.03-rc1 + patches (see [1]), flashing u-boot.bin over DFU (see [2] for details). Without $subject, image has to be aligned to 4096 bytes as below:

$ dd if=u-boot.bin of=u-boot-4k.bin bs=4k seek=1 && truncate -s %4096
u-boot-4k.bin

With $subject, I don't need truncate. We still need the 4096 byte of zeroes in the beginning of the image for other reasons (which I really don't know why at this point).

Quoted 5 lines
Quoted 3 lines

[1] https://github.com/andy-shev/u-boot/tree/edison
[2] https://communities.intel.com/message/435516#435516

Ok. I will check this. Thanks for pointing out :-)

Any updates here? I'd like to send Tangier Soc and Intel Edison Board support but I kinda depend on this patch making upstream. I can resend as part of the "add intel edison" series.

Let me know

Signature · 8 lines
balbi
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170221/f5cf04fc/attachment.sig>
#5 lore raw

PATCH 1/1 usb: gadget: f_dfu: write req->actual bytes

Hi Felipe,

Quoted 76 lines

Hi Lukasz,

Quoted 72 lines · Lukasz Majewski
Quoted 71 lines
Quoted 49 lines
Quoted 11 lines
Quoted 11 lines

drivers/usb/gadget/f_dfu.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-) 
(quoted diff) +0 -0

b/drivers/usb/gadget/f_dfu.c index 8e7c981657..64cdfa7c98 100644 --- a/drivers/usb/gadget/f_dfu.c
+++ b/drivers/usb/gadget/f_dfu.c

(quoted diff) +0 -0
@@ -159,7 +159,7 @@ static void

dnload_request_complete(struct usb_ep *ep, struct
usb_request *req) int ret;

Quoted 3 lines
Quoted 3 lines

ret = dfu_write(dfu_get_entity(f_dfu->altsetting), req->buf,
- req->length, f_dfu->blk_seq_num);
+ req->actual, f_dfu->blk_seq_num);

DFU driver queues a request to USB controller. Per the gadget
API req->length contains maximum amount of data to be
transmitted. req->actual is written by USB controller with the
actual amount of data that we transmitted.

In the case of IN (TX), upon completion req->length and
req->actual should always be equal (unless errors show up, etc)

In the case of OUT (RX), upon completion req->actual MAY BE less
than req->length and that's not an error. Say host sent us a short packet which causes early termination of transfer.

With that in mind, let's consider the situation where we're
receiving data from host using DFU. Let's assume that we have a 4096 byte buffer for transfers and we're receiving a binary that's 7679 bytes in size.

Here's what we will do (pseudo-code):

int remaining = 7679;
char buf[4096];

while (remaining) {
req->length = 4096;
req->buf = buf;
usb_ep_queue(req);

/* wait for completion */

remaining -= req->actual;

dfu_write(buf, req->length); /* this is the error */ }

Quoted 7 lines

Can you see here that in the last packet we will write 4096
bytes when we should write only 3583?

In principle you are right. I need to check if this change will not introduce regressions.

Can you share your use case?

Intel Edison running v2017.03-rc1 + patches (see [1]), flashing u-boot.bin over DFU (see [2] for details). Without $subject, image has to be aligned to 4096 bytes as below:

$ dd if=u-boot.bin of=u-boot-4k.bin bs=4k seek=1 && truncate -s
%4096 u-boot-4k.bin

With $subject, I don't need truncate. We still need the 4096 byte of zeroes in the beginning of the image for other reasons (which I really don't know why at this point).

[1] https://github.com/andy-shev/u-boot/tree/edison
[2] https://communities.intel.com/message/435516#435516

Quoted 7 lines
Quoted 1 line

Ok. I will check this. Thanks for pointing out :-)

Any updates here? I'd like to send Tangier Soc and Intel Edison Board support but I kinda depend on this patch making upstream. I can resend as part of the "add intel edison" series.

Let me know

I'm setting up /test/py/dfu now on BBB. I will let you know by EOD.

Quoted 1 line

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de -------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170221/79a34d28/attachment.sig>

#6 lore raw

PATCH 1/1 usb: gadget: f_dfu: write req->actual bytes

Hi,

Quoted 75 lines · Lukasz Majewski
Quoted 75 lines
Quoted 73 lines · Lukasz Majewski
Quoted 64 lines
Quoted 49 lines
Quoted 11 lines
Quoted 11 lines

drivers/usb/gadget/f_dfu.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-) 
(quoted diff) +0 -0

b/drivers/usb/gadget/f_dfu.c index 8e7c981657..64cdfa7c98 100644 --- a/drivers/usb/gadget/f_dfu.c
+++ b/drivers/usb/gadget/f_dfu.c

(quoted diff) +0 -0
@@ -159,7 +159,7 @@ static void

dnload_request_complete(struct usb_ep *ep, struct
usb_request *req) int ret;

Quoted 3 lines
Quoted 3 lines

ret = dfu_write(dfu_get_entity(f_dfu->altsetting), req->buf,
- req->length, f_dfu->blk_seq_num);
+ req->actual, f_dfu->blk_seq_num);

DFU driver queues a request to USB controller. Per the gadget
API req->length contains maximum amount of data to be
transmitted. req->actual is written by USB controller with the
actual amount of data that we transmitted.

In the case of IN (TX), upon completion req->length and
req->actual should always be equal (unless errors show up, etc)

In the case of OUT (RX), upon completion req->actual MAY BE less
than req->length and that's not an error. Say host sent us a short packet which causes early termination of transfer.

With that in mind, let's consider the situation where we're
receiving data from host using DFU. Let's assume that we have a 4096 byte buffer for transfers and we're receiving a binary that's 7679 bytes in size.

Here's what we will do (pseudo-code):

int remaining = 7679;
char buf[4096];

while (remaining) {
req->length = 4096;
req->buf = buf;
usb_ep_queue(req);

/* wait for completion */

remaining -= req->actual;

dfu_write(buf, req->length); /* this is the error */ }

Quoted 7 lines

Can you see here that in the last packet we will write 4096
bytes when we should write only 3583?

In principle you are right. I need to check if this change will not introduce regressions.

Can you share your use case?

Intel Edison running v2017.03-rc1 + patches (see [1]), flashing u-boot.bin over DFU (see [2] for details). Without $subject, image has to be aligned to 4096 bytes as below:

$ dd if=u-boot.bin of=u-boot-4k.bin bs=4k seek=1 && truncate -s
%4096 u-boot-4k.bin
Quoted 7 lines

With $subject, I don't need truncate. We still need the 4096 byte of zeroes in the beginning of the image for other reasons (which I really don't know why at this point).

[1] https://github.com/andy-shev/u-boot/tree/edison
[2] https://communities.intel.com/message/435516#435516

Ok. I will check this. Thanks for pointing out :-)

Quoted 7 lines
Quoted 5 lines

Any updates here? I'd like to send Tangier Soc and Intel Edison Board support but I kinda depend on this patch making upstream. I can resend as part of the "add intel edison" series.

Let me know

I'm setting up /test/py/dfu now on BBB. I will let you know by EOD.

Here's what I used for testing:

#!/usr/bin/env ruby

$errors = 0
$success = 0
$total = 0

$initial_size = 64
$max_size = 8 * 1024 * 1024 # 8MiB
$current_size = $initial_size

def create_file(size)
dd if=/dev/zero of=file.bin bs=#{size} count=1 > /dev/null 2>&1
end

def send_file(size)
dfu-util -v -d 8087:0a99 -D file.bin > /dev/null 2>&1
end

def transmit(size)
create_file(size)
send_file(size)
end

def transmit_and_check(size)
transmit(size)

if $?.success?

       $success += 1
       print "."
  else
       $errors += 1
       print "F"

end

$total += 1
end

while $current_size <= $max_size do
transmit_and_check($current_size - 1)
transmit_and_check($current_size)
transmit_and_check($current_size + 1)

$current_size *= 2
end

puts "\n"
puts "Test Done"
puts "Successes: #{$success}"
puts "Failures: #{$errors}"
puts "Total: #{$total}"

Signature · 8 lines
balbi
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170221/ca59e706/attachment-0001.sig>
#7 lore raw

PATCH 1/1 usb: gadget: f_dfu: write req->actual bytes

Hi.

Quoted 85 lines

Hi,

Quoted 81 lines · Lukasz Majewski
Quoted 81 lines
Quoted 70 lines · Lukasz Majewski
Quoted 70 lines
Quoted 51 lines
Quoted 11 lines
Quoted 11 lines

drivers/usb/gadget/f_dfu.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-) 
(quoted diff) +0 -0

b/drivers/usb/gadget/f_dfu.c index 8e7c981657..64cdfa7c98 100644 --- a/drivers/usb/gadget/f_dfu.c
+++ b/drivers/usb/gadget/f_dfu.c

(quoted diff) +0 -0
@@ -159,7 +159,7 @@ static void

dnload_request_complete(struct usb_ep *ep, struct
usb_request *req) int ret;

Quoted 4 lines
Quoted 4 lines

ret = dfu_write(dfu_get_entity(f_dfu->altsetting), req->buf,
- req->length, f_dfu->blk_seq_num);
+ req->actual, f_dfu->blk_seq_num);

DFU driver queues a request to USB controller. Per the gadget
API req->length contains maximum amount of data to be
transmitted. req->actual is written by USB controller with
the actual amount of data that we transmitted.

In the case of IN (TX), upon completion req->length and
req->actual should always be equal (unless errors show up, etc)

In the case of OUT (RX), upon completion req->actual MAY BE
less than req->length and that's not an error. Say host sent us a short packet which causes early termination of transfer.

With that in mind, let's consider the situation where we're
receiving data from host using DFU. Let's assume that we have a 4096 byte buffer for transfers and we're receiving a binary that's 7679 bytes in size.

Here's what we will do (pseudo-code):

int remaining = 7679;
char buf[4096];

while (remaining) {
req->length = 4096;
req->buf = buf;
usb_ep_queue(req);

/* wait for completion */

remaining -= req->actual;

dfu_write(buf, req->length); /* this is the error */ }

Quoted 7 lines

Can you see here that in the last packet we will write 4096
bytes when we should write only 3583?

In principle you are right. I need to check if this change will not introduce regressions.

Can you share your use case?

Intel Edison running v2017.03-rc1 + patches (see [1]), flashing u-boot.bin over DFU (see [2] for details). Without $subject, image has to be aligned to 4096 bytes as below:

$ dd if=u-boot.bin of=u-boot-4k.bin bs=4k seek=1 && truncate -s
%4096 u-boot-4k.bin

With $subject, I don't need truncate. We still need the 4096 byte of zeroes in the beginning of the image for other reasons (which I really don't know why at this point).

Quoted 5 lines
Quoted 3 lines

[1] https://github.com/andy-shev/u-boot/tree/edison
[2] https://communities.intel.com/message/435516#435516

Ok. I will check this. Thanks for pointing out :-)

Any updates here? I'd like to send Tangier Soc and Intel Edison Board support but I kinda depend on this patch making upstream. I can resend as part of the "add intel edison" series.

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Let me know

I'm setting up /test/py/dfu now on BBB. I will let you know by EOD.

Here's what I used for testing:

I do appreciate that you tested it - even better, that with different approach.

However, some time ago Stephen Warren has rewritten tests for DFU, UMS to use some python infrastructure. Those tests (especially DFU, test corner cases - ZLP, +/-1B to packet size, etc).

If you want, you can add your board to them.

More info ./test/py {dfu, ums} and ./Documentation.

(Or you can wait a bit, so I will share my "fresh" BBB setup).

Best regards,
?ukasz

Quoted 43 lines

#!/usr/bin/env ruby

$errors = 0
$success = 0
$total = 0

$initial_size = 64
$max_size = 8 * 1024 * 1024 # 8MiB
$current_size = $initial_size

def create_file(size)
dd if=/dev/zero of=file.bin bs=#{size} count=1 > /dev/null 2>&1
end

def send_file(size)
dfu-util -v -d 8087:0a99 -D file.bin > /dev/null 2>&1
end

def transmit(size)
create_file(size)
send_file(size)
end

def transmit_and_check(size)
transmit(size)

if $?.success?

       $success += 1
       print "."
  else
       $errors += 1
       print "F"

end

$total += 1
end

while $current_size <= $max_size do
transmit_and_check($current_size - 1)
transmit_and_check($current_size)
transmit_and_check($current_size + 1)

Quoted 9 lines

$current_size *= 2
end

puts "\n"
puts "Test Done"
puts "Successes: #{$success}"
puts "Failures: #{$errors}"
puts "Total: #{$total}"

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de -------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170221/b74f4fc9/attachment.sig>

#8 lore raw

PATCH 1/1 usb: gadget: f_dfu: write req->actual bytes

Hi,

Quoted 86 lines · Lukasz Majewski
Quoted 86 lines
Quoted 84 lines · Lukasz Majewski
Quoted 75 lines
Quoted 74 lines · Lukasz Majewski
Quoted 73 lines
Quoted 51 lines
Quoted 11 lines
Quoted 11 lines

drivers/usb/gadget/f_dfu.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-) 
(quoted diff) +0 -0

b/drivers/usb/gadget/f_dfu.c index 8e7c981657..64cdfa7c98 100644 --- a/drivers/usb/gadget/f_dfu.c
+++ b/drivers/usb/gadget/f_dfu.c

(quoted diff) +0 -0
@@ -159,7 +159,7 @@ static void

dnload_request_complete(struct usb_ep *ep, struct
usb_request *req) int ret;

Quoted 4 lines
Quoted 4 lines

ret = dfu_write(dfu_get_entity(f_dfu->altsetting), req->buf,
- req->length, f_dfu->blk_seq_num);
+ req->actual, f_dfu->blk_seq_num);

DFU driver queues a request to USB controller. Per the gadget
API req->length contains maximum amount of data to be
transmitted. req->actual is written by USB controller with
the actual amount of data that we transmitted.

In the case of IN (TX), upon completion req->length and
req->actual should always be equal (unless errors show up, etc)

In the case of OUT (RX), upon completion req->actual MAY BE
less than req->length and that's not an error. Say host sent us a short packet which causes early termination of transfer.

With that in mind, let's consider the situation where we're
receiving data from host using DFU. Let's assume that we have a 4096 byte buffer for transfers and we're receiving a binary that's 7679 bytes in size.

Here's what we will do (pseudo-code):

int remaining = 7679;
char buf[4096];

while (remaining) {
req->length = 4096;
req->buf = buf;
usb_ep_queue(req);

/* wait for completion */

remaining -= req->actual;

dfu_write(buf, req->length); /* this is the error */ }

Quoted 7 lines

Can you see here that in the last packet we will write 4096
bytes when we should write only 3583?

In principle you are right. I need to check if this change will not introduce regressions.

Can you share your use case?

Intel Edison running v2017.03-rc1 + patches (see [1]), flashing u-boot.bin over DFU (see [2] for details). Without $subject, image has to be aligned to 4096 bytes as below:

$ dd if=u-boot.bin of=u-boot-4k.bin bs=4k seek=1 && truncate -s
%4096 u-boot-4k.bin

With $subject, I don't need truncate. We still need the 4096 byte of zeroes in the beginning of the image for other reasons (which I really don't know why at this point).

[1] https://github.com/andy-shev/u-boot/tree/edison
[2] https://communities.intel.com/message/435516#435516

Quoted 7 lines
Quoted 1 line

Ok. I will check this. Thanks for pointing out :-)

Any updates here? I'd like to send Tangier Soc and Intel Edison Board support but I kinda depend on this patch making upstream. I can resend as part of the "add intel edison" series.

Let me know

I'm setting up /test/py/dfu now on BBB. I will let you know by EOD.

Quoted 8 lines
Quoted 1 line

Here's what I used for testing:

I do appreciate that you tested it - even better, that with different approach.

However, some time ago Stephen Warren has rewritten tests for DFU, UMS to use some python infrastructure. Those tests (especially DFU, test corner cases - ZLP, +/-1B to packet size, etc).

that's exactly what I tested :-)

Quoted 1 line

If you want, you can add your board to them.

I'll see how to do that and maybe add to the TODO list

Signature · 1 line
balbi
#9 lore raw

PATCH 1/1 usb: gadget: f_dfu: write req->actual bytes

Hi Felipe,

Quoted 101 lines

Hi,

Quoted 88 lines · Lukasz Majewski
Quoted 88 lines
Quoted 86 lines · Lukasz Majewski
Quoted 76 lines
Quoted 75 lines · Lukasz Majewski
Quoted 74 lines
Quoted 52 lines
Quoted 11 lines
Quoted 11 lines

drivers/usb/gadget/f_dfu.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-) 
(quoted diff) +0 -0

b/drivers/usb/gadget/f_dfu.c index 8e7c981657..64cdfa7c98 100644 --- a/drivers/usb/gadget/f_dfu.c +++ b/drivers/usb/gadget/f_dfu.c @@ -159,7 +159,7 @@ static void dnload_request_complete(struct usb_ep *ep, struct usb_request *req) int ret;

Quoted 4 lines
Quoted 4 lines

ret = dfu_write(dfu_get_entity(f_dfu->altsetting), req->buf,
- req->length, f_dfu->blk_seq_num);
+ req->actual, f_dfu->blk_seq_num);

DFU driver queues a request to USB controller. Per the
gadget API req->length contains maximum amount of data to be transmitted. req->actual is written by USB controller with the actual amount of data that we transmitted.

In the case of IN (TX), upon completion req->length and
req->actual should always be equal (unless errors show up, etc)

In the case of OUT (RX), upon completion req->actual MAY
BE less than req->length and that's not an error. Say host sent us a short packet which causes early termination of transfer.

With that in mind, let's consider the situation where
we're receiving data from host using DFU. Let's assume that we have a 4096 byte buffer for transfers and we're receiving a binary that's 7679 bytes in size.

Here's what we will do (pseudo-code):

int remaining = 7679;
char buf[4096];

while (remaining) {
req->length = 4096;
req->buf = buf;
usb_ep_queue(req);

/* wait for completion */

remaining -= req->actual;

dfu_write(buf, req->length); /* this is the error */ }

Quoted 7 lines

Can you see here that in the last packet we will write
4096 bytes when we should write only 3583?

In principle you are right. I need to check if this change will not introduce regressions.

Can you share your use case?

Intel Edison running v2017.03-rc1 + patches (see [1]), flashing u-boot.bin over DFU (see [2] for details). Without $subject, image has to be aligned to 4096 bytes as below:

$ dd if=u-boot.bin of=u-boot-4k.bin bs=4k seek=1 && truncate
-s %4096 u-boot-4k.bin

With $subject, I don't need truncate. We still need the 4096 byte of zeroes in the beginning of the image for other reasons (which I really don't know why at this point).

[1] https://github.com/andy-shev/u-boot/tree/edison
[2] https://communities.intel.com/message/435516#435516

Quoted 7 lines
Quoted 1 line

Ok. I will check this. Thanks for pointing out :-)

Any updates here? I'd like to send Tangier Soc and Intel Edison Board support but I kinda depend on this patch making upstream. I can resend as part of the "add intel edison" series.

Let me know

I'm setting up /test/py/dfu now on BBB. I will let you know by EOD.

Quoted 8 lines
Quoted 1 line

Here's what I used for testing:

I do appreciate that you tested it - even better, that with different approach.

However, some time ago Stephen Warren has rewritten tests for DFU, UMS to use some python infrastructure. Those tests (especially DFU, test corner cases - ZLP, +/-1B to packet size, etc).

Quoted 6 lines

that's exactly what I tested :-)

Quoted 1 line

If you want, you can add your board to them.

I'll see how to do that and maybe add to the TODO list

Test HW: BBB (am335x) - with tests/py/dfu

I've added your patch to u-boot-dfu tree. I will send PR tomorrow.

You may also want to look into patches from Patrick Delaunay (those also will be added to PR):

http://patchwork.ozlabs.org/patch/704131/
http://patchwork.ozlabs.org/patch/706492/
http://patchwork.ozlabs.org/patch/706493/

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de

#10 lore raw

PATCH 1/1 usb: gadget: f_dfu: write req->actual bytes

Hi,

Lukasz Majewski <[email protected]> writes:

[snip]

Quoted 14 lines
Quoted 7 lines
Quoted 6 lines

I do appreciate that you tested it - even better, that with different approach.

However, some time ago Stephen Warren has rewritten tests for DFU, UMS to use some python infrastructure. Those tests (especially DFU, test corner cases - ZLP, +/-1B to packet size, etc).

Quoted 6 lines

that's exactly what I tested :-)

Quoted 1 line

If you want, you can add your board to them.

I'll see how to do that and maybe add to the TODO list

Quoted 7 lines

Test HW: BBB (am335x) - with tests/py/dfu

I've added your patch to u-boot-dfu tree. I will send PR tomorrow.

yeah, cool thanks ;-)

Quoted 6 lines

You may also want to look into patches from Patrick Delaunay (those also will be added to PR):

http://patchwork.ozlabs.org/patch/704131/
http://patchwork.ozlabs.org/patch/706492/
http://patchwork.ozlabs.org/patch/706493/

they look fine to my eyes. I have two extra patches (coming shortly) to make USB Command Verifier tool (from USB-IF) happy with our DFU implementation.

Signature · 8 lines
balbi
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170222/7552f501/attachment.sig>