test_sleep.py: make sleep time and margin configurable
PATCH 1/1 test_sleep.py: make sleep time and margin configurable
make the sleep time and the margin configurable.
---
travis build:
https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225
This patch is needed as I start test/py now within tbot [1]. On some configurations U-Boot is compiled on a build machine for example in munich, while the board under test is in my lab in hungary.
So the 0.25 seconds default margin is often to low because of latencies on the net.
See as an example configuration (within tbot):
https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29
[1] http://tbot.tools/modules/tc.html#u-boot-test-py
test/py/tests/test_sleep.py | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)
test/py/tests/test_sleep.py +11 -3
@@ -11,6 +11,12 @@ change test behavior. # Setup env__sleep_accurate to False if time is not accurate on your platform env__sleep_accurate = False +# Setup env__sleep_time time in seconds board is set to sleep +env__sleep_time = 3 + +# Setup env__sleep_margin set a margin for any system overhead +env__sleep_margin = 0.25 + """ def test_sleep(u_boot_console):
@@ -23,13 +29,15 @@ def test_sleep(u_boot_console): if u_boot_console.config.buildconfig.get('config_cmd_misc', 'n') != 'y': pytest.skip('sleep command not supported') + # 3s isn't too long, but is enough to cross a few second boundaries. - sleep_time = 3 + sleep_time = u_boot_console.config.env.get('env__sleep_time', 3) + sleep_margin = u_boot_console.config.env.get('env__sleep_margin', 0.25) tstart = time.time() u_boot_console.run_command('sleep %d' % sleep_time) tend = time.time() elapsed = tend - tstart assert elapsed >= (sleep_time - 0.01) if not u_boot_console.config.gdbserver: - # 0.25s margin is hopefully enough to account for any system overhead. - assert elapsed < (sleep_time + 0.25) + # margin is hopefully enough to account for any system overhead. + assert elapsed < (sleep_time + sleep_margin)
Signature · 1 line
2.24.1
PATCH 1/1 test_sleep.py: make sleep time and margin configurable
Quoted 17 lines · Heiko Schocher
make the sleep time and the margin configurable.
Signed-off-by Heiko Schocher <[email protected]>---
travis build:
https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225This patch is needed as I start test/py now within tbot [1]. On some configurations U-Boot is compiled on a build machine for example in munich, while the board under test is in my lab in hungary.
So the 0.25 seconds default margin is often to low because of latencies on the net.
Quoted 8 lines
See as an example configuration (within tbot):
https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29
[1] http://tbot.tools/modules/tc.html#u-boot-test-py
test/py/tests/test_sleep.py | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)
Related, at some point we should change sandbox to fake the time movement since this test currently waits for three seconds even on sandbox.
PATCH 1/1 test_sleep.py: make sleep time and margin configurable
Quoted 24 lines · Simon Glass
Quoted 23 lines · Heiko Schocher
make the sleep time and the margin configurable.
Signed-off-by Heiko Schocher <[email protected]>---
travis build:
https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225This patch is needed as I start test/py now within tbot [1]. On some configurations U-Boot is compiled on a build machine for example in munich, while the board under test is in my lab in hungary.
So the 0.25 seconds default margin is often to low because of latencies on the net.
See as an example configuration (within tbot):
https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29
Quoted 8 lines
Quoted 2 lines
test/py/tests/test_sleep.py | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)Reviewed-by Simon Glass <[email protected]>Related, at some point we should change sandbox to fake the time movement since this test currently waits for three seconds even on sandbox.
We definitely shouldn't do that; that's the exact kind of failure this test is intended to detect.
PATCH 1/1 test_sleep.py: make sleep time and margin configurable
Hi Stephen,
Quoted 29 lines · Stephen Warren
Quoted 27 lines · Simon Glass
Quoted 25 lines · Heiko Schocher
make the sleep time and the margin configurable.
Signed-off-by Heiko Schocher <[email protected]>---
travis build:
https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225This patch is needed as I start test/py now within tbot [1]. On some configurations U-Boot is compiled on a build machine for example in munich, while the board under test is in my lab in hungary.
So the 0.25 seconds default margin is often to low because of latencies on the net.
See as an example configuration (within tbot):
https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29
[1] http://tbot.tools/modules/tc.html#u-boot-test-py
test/py/tests/test_sleep.py | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)
Quoted 8 lines
Quoted 5 lines
Reviewed-by Simon Glass <[email protected]>Related, at some point we should change sandbox to fake the time movement since this test currently waits for three seconds even on sandbox.
We definitely shouldn't do that; that's the exact kind of failure this test is intended to detect.
No, we're not looking for bugs in sandbox's time handling. We are just testing the plumbing associated with delaying (timer driver, etc.).
Regards,
Simon
PATCH 1/1 test_sleep.py: make sleep time and margin configurable
Quoted 38 lines · Simon Glass
Hi Stephen,
Quoted 35 lines · Stephen Warren
Quoted 32 lines · Simon Glass
Quoted 17 lines · Heiko Schocher
make the sleep time and the margin configurable.
Signed-off-by Heiko Schocher <[email protected]>---
travis build:
https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225This patch is needed as I start test/py now within tbot [1]. On some configurations U-Boot is compiled on a build machine for example in munich, while the board under test is in my lab in hungary.
So the 0.25 seconds default margin is often to low because of latencies on the net.
Quoted 8 lines
See as an example configuration (within tbot):
https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29
[1] http://tbot.tools/modules/tc.html#u-boot-test-py
test/py/tests/test_sleep.py | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)Reviewed-by Simon Glass <[email protected]>Related, at some point we should change sandbox to fake the time movement since this test currently waits for three seconds even on sandbox.
Quoted 5 lines
Quoted 2 lines
We definitely shouldn't do that; that's the exact kind of failure this test is intended to detect.
No, we're not looking for bugs in sandbox's time handling. We are just testing the plumbing associated with delaying (timer driver, etc.).
The entire purpose of the test is to look for bugs in the backend implementation of the time handling. I should know; I wrote the test!
PATCH 1/1 test_sleep.py: make sleep time and margin configurable
Hi Stephen,
Quoted 40 lines · Stephen Warren
Quoted 38 lines · Simon Glass
Hi Stephen,
Quoted 35 lines · Stephen Warren
Quoted 32 lines · Simon Glass
Quoted 17 lines · Heiko Schocher
make the sleep time and the margin configurable.
Signed-off-by Heiko Schocher <[email protected]>---
travis build:
https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225This patch is needed as I start test/py now within tbot [1]. On some configurations U-Boot is compiled on a build machine for example in munich, while the board under test is in my lab in hungary.
So the 0.25 seconds default margin is often to low because of latencies on the net.
Quoted 8 lines
See as an example configuration (within tbot):
https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29
[1] http://tbot.tools/modules/tc.html#u-boot-test-py
test/py/tests/test_sleep.py | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)Reviewed-by Simon Glass <[email protected]>Related, at some point we should change sandbox to fake the time movement since this test currently waits for three seconds even on sandbox.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
We definitely shouldn't do that; that's the exact kind of failure this test is intended to detect.
No, we're not looking for bugs in sandbox's time handling. We are just testing the plumbing associated with delaying (timer driver, etc.).
The entire purpose of the test is to look for bugs in the backend implementation of the time handling. I should know; I wrote the test!
OK and we have time_ut.c as well.
Perhaps we should disable this test on sandbox then? It really doesn't make sense to be testing timeouts on sandbox IMO and it costs us 3 seconds on each test run.
Regards,
Simon
PATCH 1/1 test_sleep.py: make sleep time and margin configurable
Quoted 49 lines · Simon Glass
Hi Stephen,
Quoted 46 lines · Stephen Warren
Quoted 43 lines · Simon Glass
Hi Stephen,
Quoted 29 lines · Stephen Warren
Quoted 27 lines · Simon Glass
Quoted 25 lines · Heiko Schocher
make the sleep time and the margin configurable.
Signed-off-by Heiko Schocher <[email protected]>---
travis build:
https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225This patch is needed as I start test/py now within tbot [1]. On some configurations U-Boot is compiled on a build machine for example in munich, while the board under test is in my lab in hungary.
So the 0.25 seconds default margin is often to low because of latencies on the net.
See as an example configuration (within tbot):
https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29
[1] http://tbot.tools/modules/tc.html#u-boot-test-py
test/py/tests/test_sleep.py | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)Quoted 8 lines
Quoted 5 lines
Reviewed-by Simon Glass <[email protected]>Related, at some point we should change sandbox to fake the time movement since this test currently waits for three seconds even on sandbox.
We definitely shouldn't do that; that's the exact kind of failure this test is intended to detect.
No, we're not looking for bugs in sandbox's time handling. We are just testing the plumbing associated with delaying (timer driver, etc.).
Quoted 8 lines
Quoted 2 lines
The entire purpose of the test is to look for bugs in the backend implementation of the time handling. I should know; I wrote the test!
OK and we have
time_ut.cas well.Perhaps we should disable this test on sandbox then? It really doesn't make sense to be testing timeouts on sandbox IMO and it costs us 3 seconds on each test run.
I'd rather keep such policy out of the test framework. If you personally don't want to run the test, you can always specify '-k not sleep' (if I got the syntax right...) when running test.py. I think e.g. the Travis test configuration might already do this, as an example.
PATCH 1/1 test_sleep.py: make sleep time and margin configurable
In reply to Heiko SchocherQuoted 1 line · Heiko Schocher
make the sleep time and the margin configurable.
PATCH 1/1 test_sleep.py: make sleep time and margin configurable
In reply to Heiko SchocherQuoted 5 lines · Heiko Schocher
make the sleep time and the margin configurable.
Signed-off-by Heiko Schocher <[email protected]>Reviewed-by Simon Glass <[email protected]>Reviewed-by Stephen Warren <[email protected]>
Applied to u-boot/master, thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: not available URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200707/689dfb99/attachment.sig>