test.py and tftp crc32 test?

U-Boot Tom Rini 3 participants 7 messages Aug 15, 2016 – Aug 16, 2016

Share & tools

Tom Rini

author

2016-08-15 22:49 UTC #0 lore raw

Hey guys,

Is anyone else running the crc32 tftp tests with test.py? I'm trying to do it locally but even with a 2MiB file it looks like somehow the crc32 is never captured in the output. I've already made sure that the crc32 value is in lowercase to match the U-Boot output and running the test steps in console gives me the expected output. And the rest of the network tests pass, any ideas? Thanks!

Signature · 8 lines

Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160815/c8262136/attachment.sig>

Stephen Warren

2016-08-15 22:59 UTC #1 lore raw

Quoted 8 lines · Tom Rini

Hey guys,
Is anyone else running the crc32 tftp tests with test.py? I'm trying to do it locally but even with a 2MiB file it looks like somehow the crc32 is never captured in the output. I've already made sure that the crc32 value is in lowercase to match the U-Boot output and running the test steps in console gives me the expected output. And the rest of the network tests pass, any ideas? Thanks!

I'm not certain that anyone other than myself is running test.py on real HW (vs. sandbox where it's trivial).

If you look at test-log.html it should show what U-Boot outputs on the console, and where any error was detected in the test. Alternativeluy, add "-s" to the invocation and U-Boot output should show up on stdout while the tests are running. What's the error reported there; just a timeout waiting for the CRC? Here's an example of what test_net_tftpboot looks like for me:

Quoted 12 lines

Tegra210 (P2371-2180) # tftpboot 80000000 ubtest-readable.bin Using eth_rtl8169 device
TFTP from server 10.20.204.51; our IP address is 10.20.204.52 Filename 'ubtest-readable.bin'.
Load address: 0x80000000
Loading: *%08#################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 ####################
	 2.9 MiB/s

Quoted 5 lines

done
Bytes transferred = 5058624 (4d3040 hex)
Tegra210 (P2371-2180) # crc32 80000000 $filesize
crc32 for 80000000 ... 804d303f ==> c2244b26
Tegra210 (P2371-2180) #

In the past, I have seen tests pass when run manually but not under test.py, e.g. due to heap fragmentation, state, or corrupted memory due to earlier tests, which weren't run in the manual case. Might that be the issue?

Tom Rini

author

2016-08-15 23:20 UTC #2 lore raw

Quoted 38 lines · Stephen Warren

Quoted 8 lines · Tom Rini
Hey guys,
Is anyone else running the crc32 tftp tests with test.py? I'm trying to do it locally but even with a 2MiB file it looks like somehow the crc32 is never captured in the output. I've already made sure that the crc32 value is in lowercase to match the U-Boot output and running the test steps in console gives me the expected output. And the rest of the network tests pass, any ideas? Thanks!
I'm not certain that anyone other than myself is running test.py on real HW (vs. sandbox where it's trivial).
If you look at test-log.html it should show what U-Boot outputs on the console, and where any error was detected in the test. Alternativeluy, add "-s" to the invocation and U-Boot output should show up on stdout while the tests are running. What's the error reported there; just a timeout waiting for the CRC? Here's an example of what test_net_tftpboot looks like for me:
Quoted 12 lines
Tegra210 (P2371-2180) # tftpboot 80000000 ubtest-readable.bin Using eth_rtl8169 device
TFTP from server 10.20.204.51; our IP address is 10.20.204.52 Filename 'ubtest-readable.bin'.
Load address: 0x80000000
Loading: *%08#################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 ####################
	 2.9 MiB/s
Quoted 5 lines
done
Bytes transferred = 5058624 (4d3040 hex)
Tegra210 (P2371-2180) # crc32 80000000 $filesize
crc32 for 80000000 ... 804d303f ==> c2244b26
Tegra210 (P2371-2180) #

Quoted 4 lines

In the past, I have seen tests pass when run manually but not under test.py, e.g. due to heap fragmentation, state, or corrupted memory due to earlier tests, which weren't run in the manual case. Might that be the issue?

Adding in -s this is part of what I see:

=> .s=> ping $serverip
Waiting for Ethernet connection... done.
Using sms0 device
host 192.168.0.3 is alive
=> .=> tftpboot 80000000 1MiBtest.bin
Waiting for Ethernet connection... done.
Using sms0 device
TFTP from server 192.168.0.3; our IP address is 192.168.0.140 Filename '1MiBtest.bin'.
Load address: 0x80000000
Loading: #################################################################

         #################################################################
         #################################################################
         ##########
         171.9 KiB/s

done
Bytes transferred = 1048576 (100000 hex)
=> crc32 80000000 $filesize
CRC32 for 80000000 ... 800fffff ==> F2fa737e0
=>

For some reason there's an extra 'F' at the start of the crc32 output. Looking at it in the html file, it looks all correct there. In the output from test.py the captured stdout end just before the CRC32 value would be. I've tried larger test binaries and get the same output so I'd assume if there was some heap corruption or similar, I'd not tickle it with bigger files too (2MiB, 4MiB and I think even 8MiB are small enough to be downloaded in the time the test allows). Thanks!

Signature · 8 lines

Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160815/ac061009/attachment.sig>

Stephen Warren

2016-08-16 03:35 UTC #3 lore raw

Quoted 67 lines · Tom Rini

Quoted 38 lines · Stephen Warren
Quoted 8 lines · Tom Rini
Hey guys,
Is anyone else running the crc32 tftp tests with test.py? I'm trying to do it locally but even with a 2MiB file it looks like somehow the crc32 is never captured in the output. I've already made sure that the crc32 value is in lowercase to match the U-Boot output and running the test steps in console gives me the expected output. And the rest of the network tests pass, any ideas? Thanks!
I'm not certain that anyone other than myself is running test.py on real HW (vs. sandbox where it's trivial).
If you look at test-log.html it should show what U-Boot outputs on the console, and where any error was detected in the test. Alternativeluy, add "-s" to the invocation and U-Boot output should show up on stdout while the tests are running. What's the error reported there; just a timeout waiting for the CRC? Here's an example of what test_net_tftpboot looks like for me:
Quoted 12 lines
Tegra210 (P2371-2180) # tftpboot 80000000 ubtest-readable.bin Using eth_rtl8169 device
TFTP from server 10.20.204.51; our IP address is 10.20.204.52 Filename 'ubtest-readable.bin'.
Load address: 0x80000000
Loading: *%08#################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 ####################
	 2.9 MiB/s
Quoted 5 lines
done
Bytes transferred = 5058624 (4d3040 hex)
Tegra210 (P2371-2180) # crc32 80000000 $filesize
crc32 for 80000000 ... 804d303f ==> c2244b26
Tegra210 (P2371-2180) #
Quoted 4 lines
In the past, I have seen tests pass when run manually but not under test.py, e.g. due to heap fragmentation, state, or corrupted memory due to earlier tests, which weren't run in the manual case. Might that be the issue?
Adding in -s this is part of what I see:
=> .s=> ping $serverip
Waiting for Ethernet connection... done.
Using sms0 device
host 192.168.0.3 is alive
=> .=> tftpboot 80000000 1MiBtest.bin
Waiting for Ethernet connection... done.
Using sms0 device
TFTP from server 192.168.0.3; our IP address is 192.168.0.140 Filename '1MiBtest.bin'.
Load address: 0x80000000
Loading: #################################################################
         #################################################################
         #################################################################
         ##########
         171.9 KiB/s
done
Bytes transferred = 1048576 (100000 hex)
=> crc32 80000000 $filesize
CRC32 for 80000000 ... 800fffff ==> F2fa737e0
=>

Quoted 7 lines

For some reason there's an extra 'F' at the start of the crc32 output. Looking at it in the html file, it looks all correct there. In the output from test.py the captured stdout end just before the CRC32 value would be. I've tried larger test binaries and get the same output so I'd assume if there was some heap corruption or similar, I'd not tickle it with bigger files too (2MiB, 4MiB and I think even 8MiB are small enough to be downloaded in the time the test allows). Thanks!

The "F" is coming from the test infrastructure, not U-Boot's output. A character is printed per test, such as "." for pass, "s" for skip, and "F" for fail. For some reason, the test is seeing a failure before it's read the output from the crc32 command.

Ah, I see the issue now: the command prompt (=>) in use is part of the output from the crc32 command, so the test infra-structure thinks the ==> printed by crc32 is the end of the crc32 output. Perhaps test.py should look for "[\r\n]${prompt}" rather than "${prompt}", or this board could have a prompt that's slightly less likely to trigger false matches.

Heiko Schocher

2016-08-16 05:19 UTC #4 lore raw

Hello Stephen,

Am 16.08.2016 um 05:35 schrieb Stephen Warren:

Quoted 76 lines · Tom Rini

Quoted 67 lines · Stephen Warren
Quoted 38 lines · Tom Rini
Quoted 8 lines
Hey guys,
Is anyone else running the crc32 tftp tests with test.py? I'm trying to do it locally but even with a 2MiB file it looks like somehow the crc32 is never captured in the output. I've already made sure that the crc32 value is in lowercase to match the U-Boot output and running the test steps in console gives me the expected output. And the rest of the network tests pass, any ideas? Thanks!
I'm not certain that anyone other than myself is running test.py on real HW (vs. sandbox where it's trivial).
If you look at test-log.html it should show what U-Boot outputs on the console, and where any error was detected in the test. Alternativeluy, add "-s" to the invocation and U-Boot output should show up on stdout while the tests are running. What's the error reported there; just a timeout waiting for the CRC? Here's an example of what test_net_tftpboot looks like for me:
Quoted 12 lines
Tegra210 (P2371-2180) # tftpboot 80000000 ubtest-readable.bin Using eth_rtl8169 device
TFTP from server 10.20.204.51; our IP address is 10.20.204.52 Filename 'ubtest-readable.bin'.
Load address: 0x80000000
Loading: *%08#################################################################
     #################################################################
     #################################################################
     #################################################################
     #################################################################
     ####################
     2.9 MiB/s
Quoted 5 lines
done
Bytes transferred = 5058624 (4d3040 hex)
Tegra210 (P2371-2180) # crc32 80000000 $filesize
crc32 for 80000000 ... 804d303f ==> c2244b26
Tegra210 (P2371-2180) #
Quoted 4 lines
In the past, I have seen tests pass when run manually but not under test.py, e.g. due to heap fragmentation, state, or corrupted memory due to earlier tests, which weren't run in the manual case. Might that be the issue?
Adding in -s this is part of what I see:
=> .s=> ping $serverip
Waiting for Ethernet connection... done.
Using sms0 device
host 192.168.0.3 is alive
=> .=> tftpboot 80000000 1MiBtest.bin
Waiting for Ethernet connection... done.
Using sms0 device
TFTP from server 192.168.0.3; our IP address is 192.168.0.140 Filename '1MiBtest.bin'.
Load address: 0x80000000
Loading: #################################################################
         #################################################################
         #################################################################
         ##########
         171.9 KiB/s
done
Bytes transferred = 1048576 (100000 hex)
=> crc32 80000000 $filesize
CRC32 for 80000000 ... 800fffff ==> F2fa737e0
=>
Quoted 7 lines
For some reason there's an extra 'F' at the start of the crc32 output. Looking at it in the html file, it looks all correct there. In the output from test.py the captured stdout end just before the CRC32 value would be. I've tried larger test binaries and get the same output so I'd assume if there was some heap corruption or similar, I'd not tickle it with bigger files too (2MiB, 4MiB and I think even 8MiB are small enough to be downloaded in the time the test allows). Thanks!

Quoted 8 lines

The "F" is coming from the test infrastructure, not U-Boot's output. A character is printed per test, such as "." for pass, "s" for skip, and "F" for fail. For some reason, the test is seeing a failure before it's read the output from the crc32 command.
Ah, I see the issue now: the command prompt (=>) in use is part of the output from the crc32 command, so the test infra-structure thinks the ==> printed by crc32 is the end of the crc32 output. Perhaps test.py should look for "[\r\n]${prompt}" rather than "${prompt}", or this board could have a prompt that's slightly less likely to trigger false matches.

Argh, yes, I had the same problem and added an "ignore list" to tbot, see:
https://github.com/hsdenx/tbot/blob/master/src/common/tbot_connection_paramiko.py#L39

bye,
Heiko

Signature · 2 lines

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

Tom Rini

author

2016-08-16 11:55 UTC #5 lore raw

In reply to Stephen Warren

Quoted 76 lines · Stephen Warren

Quoted 67 lines · Tom Rini
Quoted 38 lines · Stephen Warren
Quoted 8 lines · Tom Rini
Hey guys,
Is anyone else running the crc32 tftp tests with test.py? I'm trying to do it locally but even with a 2MiB file it looks like somehow the crc32 is never captured in the output. I've already made sure that the crc32 value is in lowercase to match the U-Boot output and running the test steps in console gives me the expected output. And the rest of the network tests pass, any ideas? Thanks!
I'm not certain that anyone other than myself is running test.py on real HW (vs. sandbox where it's trivial).
If you look at test-log.html it should show what U-Boot outputs on the console, and where any error was detected in the test. Alternativeluy, add "-s" to the invocation and U-Boot output should show up on stdout while the tests are running. What's the error reported there; just a timeout waiting for the CRC? Here's an example of what test_net_tftpboot looks like for me:
Quoted 12 lines
Tegra210 (P2371-2180) # tftpboot 80000000 ubtest-readable.bin Using eth_rtl8169 device
TFTP from server 10.20.204.51; our IP address is 10.20.204.52 Filename 'ubtest-readable.bin'.
Load address: 0x80000000
Loading: *%08#################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 ####################
	 2.9 MiB/s
Quoted 5 lines
done
Bytes transferred = 5058624 (4d3040 hex)
Tegra210 (P2371-2180) # crc32 80000000 $filesize
crc32 for 80000000 ... 804d303f ==> c2244b26
Tegra210 (P2371-2180) #
Quoted 4 lines
In the past, I have seen tests pass when run manually but not under test.py, e.g. due to heap fragmentation, state, or corrupted memory due to earlier tests, which weren't run in the manual case. Might that be the issue?
Adding in -s this is part of what I see:
=> .s=> ping $serverip
Waiting for Ethernet connection... done.
Using sms0 device
host 192.168.0.3 is alive
=> .=> tftpboot 80000000 1MiBtest.bin
Waiting for Ethernet connection... done.
Using sms0 device
TFTP from server 192.168.0.3; our IP address is 192.168.0.140 Filename '1MiBtest.bin'.
Load address: 0x80000000
Loading: #################################################################
        #################################################################
        #################################################################
        ##########
        171.9 KiB/s
done
Bytes transferred = 1048576 (100000 hex)
=> crc32 80000000 $filesize
CRC32 for 80000000 ... 800fffff ==> F2fa737e0
=>
Quoted 7 lines
For some reason there's an extra 'F' at the start of the crc32 output. Looking at it in the html file, it looks all correct there. In the output from test.py the captured stdout end just before the CRC32 value would be. I've tried larger test binaries and get the same output so I'd assume if there was some heap corruption or similar, I'd not tickle it with bigger files too (2MiB, 4MiB and I think even 8MiB are small enough to be downloaded in the time the test allows). Thanks!

Quoted 4 lines

The "F" is coming from the test infrastructure, not U-Boot's output. A character is printed per test, such as "." for pass, "s" for skip, and "F" for fail. For some reason, the test is seeing a failure before it's read the output from the crc32 command.

Ah, OK.

Quoted 6 lines

Ah, I see the issue now: the command prompt (=>) in use is part of the output from the crc32 command, so the test infra-structure thinks the ==> printed by crc32 is the end of the crc32 output. Perhaps test.py should look for "[\r\n]${prompt}" rather than "${prompt}", or this board could have a prompt that's slightly less likely to trigger false matches.

My quick attempts to modify self.prompt_escaped haven't worked out, so I'd be happy to test a patch. I'd rather not have to change the prompt, but it wouldn't be the end of the world if I did at least. Thanks!

Signature · 8 lines

Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160816/53146f22/attachment.sig>

Heiko Schocher

2016-08-16 05:14 UTC #6 lore raw

In reply to Stephen Warren

Hello Stephen,

Am 16.08.2016 um 00:59 schrieb Stephen Warren:

Quoted 10 lines · Tom Rini

Quoted 8 lines
Hey guys,
Is anyone else running the crc32 tftp tests with test.py? I'm trying to do it locally but even with a 2MiB file it looks like somehow the crc32 is never captured in the output. I've already made sure that the crc32 value is in lowercase to match the U-Boot output and running the test steps in console gives me the expected output. And the rest of the network tests pass, any ideas? Thanks!

Quoted 2 lines

I'm not certain that anyone other than myself is running test.py on real HW (vs. sandbox where it's trivial).

I run it on real hw, as I start test.py from tbot on at91, am335x and imx6 based boards ...
(Ok, I had to admit that I did not found time since my vacation to look,
why my cyclic automated tests with buildbot not work ...) but for an older
example:
http://xeidos.ddns.net/tests/test_db_auslesen.php#60
(may slow as tbot and this webserver runs on a raspberry pi)

and click on "test py result" to see test.py output ...

But sorry, yes, I did not run the crc32 tftp tests ...

bye,
Heiko

Quoted 31 lines

If you look at test-log.html it should show what U-Boot outputs on the console, and where any error was detected in the test. Alternativeluy, add "-s" to the invocation and U-Boot output should show up on stdout while the tests are running. What's the error reported there; just a timeout waiting for the CRC? Here's an example of what test_net_tftpboot looks like for me:
Quoted 12 lines
Tegra210 (P2371-2180) # tftpboot 80000000 ubtest-readable.bin Using eth_rtl8169 device
TFTP from server 10.20.204.51; our IP address is 10.20.204.52 Filename 'ubtest-readable.bin'.
Load address: 0x80000000
Loading: *%08#################################################################
     #################################################################
     #################################################################
     #################################################################
     #################################################################
     ####################
     2.9 MiB/s
Quoted 5 lines
done
Bytes transferred = 5058624 (4d3040 hex)
Tegra210 (P2371-2180) # crc32 80000000 $filesize
crc32 for 80000000 ... 804d303f ==> c2244b26
Tegra210 (P2371-2180) #
In the past, I have seen tests pass when run manually but not under test.py, e.g. due to heap fragmentation, state, or corrupted memory due to earlier tests, which weren't run in the manual case. Might that be the issue?
_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

Signature · 2 lines

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany