Skip to contentSkip to conversation

test_sleep.py: make sleep time and margin configurable

PATCH New Heiko Schocher 4 participants 9 messages Jun 4, 2020 – Jul 8, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 test_sleep.py: make sleep time and margin configurable

make the sleep time and the margin configurable.

---

travis build:
https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225

This patch is needed as I start test/py now within tbot [1]. On some configurations U-Boot is compiled on a build machine for example in munich, while the board under test is in my lab in hungary.

So the 0.25 seconds default margin is often to low because of latencies on the net.

See as an example configuration (within tbot):

https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29

[1] http://tbot.tools/modules/tc.html#u-boot-test-py

 test/py/tests/test_sleep.py | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)
test/py/tests/test_sleep.py +11 -3
index b69edf26ef..392af29db2 100644
@@ -11,6 +11,12 @@ change test behavior.
 # Setup env__sleep_accurate to False if time is not accurate on your platform
 env__sleep_accurate = False
 
+# Setup env__sleep_time time in seconds board is set to sleep
+env__sleep_time = 3
+
+# Setup env__sleep_margin set a margin for any system overhead
+env__sleep_margin = 0.25
+
 """
 
 def test_sleep(u_boot_console):
@@ -23,13 +29,15 @@ def test_sleep(u_boot_console):
 
     if u_boot_console.config.buildconfig.get('config_cmd_misc', 'n') != 'y':
         pytest.skip('sleep command not supported')
+
     # 3s isn't too long, but is enough to cross a few second boundaries.
-    sleep_time = 3
+    sleep_time = u_boot_console.config.env.get('env__sleep_time', 3)
+    sleep_margin = u_boot_console.config.env.get('env__sleep_margin', 0.25)
     tstart = time.time()
     u_boot_console.run_command('sleep %d' % sleep_time)
     tend = time.time()
     elapsed = tend - tstart
     assert elapsed >= (sleep_time - 0.01)
     if not u_boot_console.config.gdbserver:
-        # 0.25s margin is hopefully enough to account for any system overhead.
-        assert elapsed < (sleep_time + 0.25)
+        # margin is hopefully enough to account for any system overhead.
+        assert elapsed < (sleep_time + sleep_margin)
Signature · 1 line
2.24.1
#1 lore raw

PATCH 1/1 test_sleep.py: make sleep time and margin configurable

Quoted 17 lines · Heiko Schocher

make the sleep time and the margin configurable.

---

travis build:
https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225

This patch is needed as I start test/py now within tbot [1]. On some configurations U-Boot is compiled on a build machine for example in munich, while the board under test is in my lab in hungary.

So the 0.25 seconds default margin is often to low because of latencies on the net.

Quoted 8 lines

See as an example configuration (within tbot):

https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29

[1] http://tbot.tools/modules/tc.html#u-boot-test-py

 test/py/tests/test_sleep.py | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

Related, at some point we should change sandbox to fake the time movement since this test currently waits for three seconds even on sandbox.

#2 lore raw

PATCH 1/1 test_sleep.py: make sleep time and margin configurable

Quoted 24 lines · Simon Glass
Quoted 23 lines · Heiko Schocher

make the sleep time and the margin configurable.

---

travis build:
https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225

This patch is needed as I start test/py now within tbot [1]. On some configurations U-Boot is compiled on a build machine for example in munich, while the board under test is in my lab in hungary.

So the 0.25 seconds default margin is often to low because of latencies on the net.

See as an example configuration (within tbot):

https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29

[1] http://tbot.tools/modules/tc.html#u-boot-test-py

Quoted 8 lines
Quoted 2 lines
 test/py/tests/test_sleep.py | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

Related, at some point we should change sandbox to fake the time movement since this test currently waits for three seconds even on sandbox.

We definitely shouldn't do that; that's the exact kind of failure this test is intended to detect.

#3 lore raw

PATCH 1/1 test_sleep.py: make sleep time and margin configurable

Hi Stephen,

Quoted 29 lines · Stephen Warren
Quoted 27 lines · Simon Glass
Quoted 25 lines · Heiko Schocher

make the sleep time and the margin configurable.

---

travis build:
https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225

This patch is needed as I start test/py now within tbot [1]. On some configurations U-Boot is compiled on a build machine for example in munich, while the board under test is in my lab in hungary.

So the 0.25 seconds default margin is often to low because of latencies on the net.

See as an example configuration (within tbot):

https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29

[1] http://tbot.tools/modules/tc.html#u-boot-test-py

 test/py/tests/test_sleep.py | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)
Quoted 8 lines
Quoted 5 lines

Related, at some point we should change sandbox to fake the time movement since this test currently waits for three seconds even on sandbox.

We definitely shouldn't do that; that's the exact kind of failure this test is intended to detect.

No, we're not looking for bugs in sandbox's time handling. We are just testing the plumbing associated with delaying (timer driver, etc.).

Regards,
Simon

#4 lore raw

PATCH 1/1 test_sleep.py: make sleep time and margin configurable

Quoted 38 lines · Simon Glass

Hi Stephen,

Quoted 35 lines · Stephen Warren
Quoted 32 lines · Simon Glass
Quoted 17 lines · Heiko Schocher

make the sleep time and the margin configurable.

---

travis build:
https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225

This patch is needed as I start test/py now within tbot [1]. On some configurations U-Boot is compiled on a build machine for example in munich, while the board under test is in my lab in hungary.

So the 0.25 seconds default margin is often to low because of latencies on the net.

Quoted 8 lines

See as an example configuration (within tbot):

https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29

[1] http://tbot.tools/modules/tc.html#u-boot-test-py

 test/py/tests/test_sleep.py | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

Related, at some point we should change sandbox to fake the time movement since this test currently waits for three seconds even on sandbox.

Quoted 5 lines
Quoted 2 lines

We definitely shouldn't do that; that's the exact kind of failure this test is intended to detect.

No, we're not looking for bugs in sandbox's time handling. We are just testing the plumbing associated with delaying (timer driver, etc.).

The entire purpose of the test is to look for bugs in the backend implementation of the time handling. I should know; I wrote the test!

#5 lore raw

PATCH 1/1 test_sleep.py: make sleep time and margin configurable

Hi Stephen,

Quoted 40 lines · Stephen Warren
Quoted 38 lines · Simon Glass

Hi Stephen,

Quoted 35 lines · Stephen Warren
Quoted 32 lines · Simon Glass
Quoted 17 lines · Heiko Schocher

make the sleep time and the margin configurable.

---

travis build:
https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225

This patch is needed as I start test/py now within tbot [1]. On some configurations U-Boot is compiled on a build machine for example in munich, while the board under test is in my lab in hungary.

So the 0.25 seconds default margin is often to low because of latencies on the net.

Quoted 8 lines

See as an example configuration (within tbot):

https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29

[1] http://tbot.tools/modules/tc.html#u-boot-test-py

 test/py/tests/test_sleep.py | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

Related, at some point we should change sandbox to fake the time movement since this test currently waits for three seconds even on sandbox.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

We definitely shouldn't do that; that's the exact kind of failure this test is intended to detect.

No, we're not looking for bugs in sandbox's time handling. We are just testing the plumbing associated with delaying (timer driver, etc.).

The entire purpose of the test is to look for bugs in the backend implementation of the time handling. I should know; I wrote the test!

OK and we have time_ut.c as well.

Perhaps we should disable this test on sandbox then? It really doesn't make sense to be testing timeouts on sandbox IMO and it costs us 3 seconds on each test run.

Regards,
Simon

#6 lore raw

PATCH 1/1 test_sleep.py: make sleep time and margin configurable

Quoted 49 lines · Simon Glass

Hi Stephen,

Quoted 46 lines · Stephen Warren
Quoted 43 lines · Simon Glass

Hi Stephen,

Quoted 29 lines · Stephen Warren
Quoted 27 lines · Simon Glass
Quoted 25 lines · Heiko Schocher

make the sleep time and the margin configurable.

---

travis build:
https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225

This patch is needed as I start test/py now within tbot [1]. On some configurations U-Boot is compiled on a build machine for example in munich, while the board under test is in my lab in hungary.

So the 0.25 seconds default margin is often to low because of latencies on the net.

See as an example configuration (within tbot):

https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29

[1] http://tbot.tools/modules/tc.html#u-boot-test-py

 test/py/tests/test_sleep.py | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)
Quoted 8 lines
Quoted 5 lines

Related, at some point we should change sandbox to fake the time movement since this test currently waits for three seconds even on sandbox.

We definitely shouldn't do that; that's the exact kind of failure this test is intended to detect.

No, we're not looking for bugs in sandbox's time handling. We are just testing the plumbing associated with delaying (timer driver, etc.).

Quoted 8 lines
Quoted 2 lines

The entire purpose of the test is to look for bugs in the backend implementation of the time handling. I should know; I wrote the test!

OK and we have time_ut.c as well.

Perhaps we should disable this test on sandbox then? It really doesn't make sense to be testing timeouts on sandbox IMO and it costs us 3 seconds on each test run.

I'd rather keep such policy out of the test framework. If you personally don't want to run the test, you can always specify '-k not sleep' (if I got the syntax right...) when running test.py. I think e.g. the Travis test configuration might already do this, as an example.

#8 lore raw

PATCH 1/1 test_sleep.py: make sleep time and margin configurable

In reply to Heiko Schocher
Quoted 5 lines · Heiko Schocher

make the sleep time and the margin configurable.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200707/689dfb99/attachment.sig>