test/py: support running sandbox under gdbserver

PATCH In review Stephen Warren 3 participants 8 messages Feb 4, 2016 – Feb 8, 2016

lore

Share & tools

lore

Stephen Warren

author

2016-02-04 23:11 UTC #0 lore raw

PATCH 1/1 test/py: support running sandbox under gdbserver

From: Stephen Warren <[email protected]>

Implement command--line option --gdbserver COMM, which does two things:

a) Run the sandbox process under gdbserver, using COMM as gdbserver's communication channel.

b) Disables all timeouts, so that if U-Boot is halted under the debugger, tests don't fail. If the user gives up in the middle of a debugging session, they can simply CTRL-C the test script to abort it.

This allows easy debugging of test failures without having to manually re-create the failure conditions. Usage is:

Window 1:
./test/py/test.py --bd sandbox --gdbserver localhost:1234

Window 2:
gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234'

When using this option, it likely makes sense to use pytest's -k option to limit the set of tests that are executed.

Simply running U-Boot directly under gdb (rather than gdbserver) was also considered. However, this was rejected because:

a) gdb's output would then be processed by the test script, and likely confuse it causing false failures.

b) pytest by default hides stdout from tests, which would prevent the user from interacting with gdb.

While gdb can be told to redirect the debugee's stdio to a separate PTY, this would appear to leave gdb's stdio directed at the test scripts and the debugee's stdio directed elsewhere, which is the opposite of the desired effect. Perhaps some complicated PTY muxing and process hierarchy could invert this. However, the current scheme is simple to implement and use, so it doesn't seem worth complicating matters.

c) Using gdbserver allows arbitrary debuggers to be used, even those with a GUI. If the test scripts invoked the debugger themselves, they'd have to know how to execute arbitary applications. While the user could hide this all in a wrapper script, this feels like extra complication.

An interesting future idea might be a --gdb-screen option, which could spawn both U-Boot and gdb separately, and spawn the screen into a newly created window under screen. Similar options could be envisaged for creating a new xterm/... too.

--gdbserver currently only supports sandbox, and not real hardware. That's primarily because the test hooks are responsible for all aspects of hardware control, so there's nothing for the test scripts themselves can do to enable gdbserver on real hardware. We might consider introducing a separate --disable-timeouts option to support use of debuggers on real hardware, and having --gdbserver imply that option.

Signed-off-by Stephen Warren <[email protected]>

---
 test/py/conftest.py               |  8 ++++++++
 test/py/tests/test_sleep.py       |  7 ++++---
 test/py/u_boot_console_base.py    |  3 ++-
 test/py/u_boot_console_sandbox.py |  5 ++++-
 test/py/u_boot_spawn.py           | 12 ++++++++----
 5 files changed, 26 insertions(+), 9 deletions(-)

test/py/conftest.py +8 -0

index f55bf68686ba..bf8d735b83cb 100644
@@ -72,6 +72,9 @@ def pytest_addoption(parser):
         help='U-Boot board identity/instance')
     parser.addoption('--build', default=False, action='store_true',
         help='Compile U-Boot before running tests')
+    parser.addoption('--gdbserver', default=None,
+        help='Run sandbox under gdbserver. The argument is the channel '+
+        'over which gdbserver should communicate, e.g. localhost:1234')
 
 def pytest_configure(config):
     """pytest hook: Perform custom initialization at startup time.

@@ -111,6 +114,10 @@ def pytest_configure(config):
         persistent_data_dir = build_dir + '/persistent-data'
     mkdir_p(persistent_data_dir)
 
+    gdbserver = config.getoption('gdbserver')
+    if gdbserver and board_type != 'sandbox':
+        raise Exception('--gdbserver only supported with sandbox')
+
     import multiplexed_log
     log = multiplexed_log.Logfile(result_dir + '/test-log.html')

@@ -172,6 +179,7 @@ def pytest_configure(config):
     ubconfig.persistent_data_dir = persistent_data_dir
     ubconfig.board_type = board_type
     ubconfig.board_identity = board_identity
+    ubconfig.gdbserver = gdbserver
 
     env_vars = (
         'board_type',

test/py/tests/test_sleep.py +4 -3

index 74add891c333..5c1a2623fefa 100644
@@ -15,6 +15,7 @@ def test_sleep(u_boot_console):
     u_boot_console.run_command('sleep %d' % sleep_time)
     tend = time.time()
     elapsed = tend - tstart
-    delta_to_expected = abs(elapsed - sleep_time)
-    # 0.25s margin is hopefully enough to account for any system overhead.
-    assert delta_to_expected < 0.25
+    assert elapsed >= sleep_time
+    if not u_boot_console.config.gdbserver:
+        # 0.25s margin is hopefully enough to account for any system overhead.
+        assert elapsed < (sleep_time + 0.25)

test/py/u_boot_console_base.py +2 -1

index 392f8cb88532..cc5427335177 100644
@@ -300,7 +300,8 @@ class ConsoleBase(object):
             # text if LCD is enabled. This value may need tweaking in the
             # future, possibly per-test to be optimal. This works for 'help'
             # on board 'seaboard'.
-            self.p.timeout = 30000
+            if not self.config.gdbserver:
+                self.p.timeout = 30000
             self.p.logfile_read = self.logstream
             if self.config.buildconfig.get('CONFIG_SPL', False) == 'y':
                 m = self.p.expect([pattern_u_boot_spl_signon] + self.bad_patterns)

test/py/u_boot_console_sandbox.py +4 -1

index a7263f30b8ca..3de0fe4a3b6d 100644
@@ -39,7 +39,10 @@ class ConsoleSandbox(ConsoleBase):
             A u_boot_spawn.Spawn object that is attached to U-Boot.
         """
 
-        cmd = [
+        cmd = []
+        if self.config.gdbserver:
+            cmd += ['gdbserver', self.config.gdbserver]
+        cmd += [
             self.config.build_dir + '/u-boot',
             '-d',
             self.config.build_dir + '/arch/sandbox/dts/test.dtb'

test/py/u_boot_spawn.py +8 -4

index 0f52d3e945c7..4b9e81af3efb 100644
@@ -148,10 +148,14 @@ class Spawn(object):
                     self.buf = self.buf[posafter:]
                     return earliest_pi
                 tnow_s = time.time()
-                tdelta_ms = (tnow_s - tstart_s) * 1000
-                if tdelta_ms > self.timeout:
-                    raise Timeout()
-                events = self.poll.poll(self.timeout - tdelta_ms)
+                if self.timeout:
+                    tdelta_ms = (tnow_s - tstart_s) * 1000
+                    poll_maxwait = self.timeout - tdelta_ms
+                    if tdelta_ms > self.timeout:
+                        raise Timeout()
+                else:
+                    poll_maxwait = None
+                events = self.poll.poll(poll_maxwait)
                 if not events:
                     raise Timeout()
                 c = os.read(self.fd, 1024)

Signature · 1 line

2.7.0

Simon Glass

2016-02-06 20:30 UTC #1 lore raw

PATCH 1/1 test/py: support running sandbox under gdbserver

Hi Stephen,

Quoted 57 lines · Stephen Warren

From: Stephen Warren <[email protected]>
Implement command--line option --gdbserver COMM, which does two things:
a) Run the sandbox process under gdbserver, using COMM as gdbserver's communication channel.
b) Disables all timeouts, so that if U-Boot is halted under the debugger, tests don't fail. If the user gives up in the middle of a debugging session, they can simply CTRL-C the test script to abort it.
This allows easy debugging of test failures without having to manually re-create the failure conditions. Usage is:
Window 1:
./test/py/test.py --bd sandbox --gdbserver localhost:1234
Window 2:
gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234'
When using this option, it likely makes sense to use pytest's -k option to limit the set of tests that are executed.
Simply running U-Boot directly under gdb (rather than gdbserver) was also considered. However, this was rejected because:
a) gdb's output would then be processed by the test script, and likely confuse it causing false failures.
b) pytest by default hides stdout from tests, which would prevent the user from interacting with gdb.
While gdb can be told to redirect the debugee's stdio to a separate PTY, this would appear to leave gdb's stdio directed at the test scripts and the debugee's stdio directed elsewhere, which is the opposite of the desired effect. Perhaps some complicated PTY muxing and process hierarchy could invert this. However, the current scheme is simple to implement and use, so it doesn't seem worth complicating matters.
c) Using gdbserver allows arbitrary debuggers to be used, even those with a GUI. If the test scripts invoked the debugger themselves, they'd have to know how to execute arbitary applications. While the user could hide this all in a wrapper script, this feels like extra complication.
An interesting future idea might be a --gdb-screen option, which could spawn both U-Boot and gdb separately, and spawn the screen into a newly created window under screen. Similar options could be envisaged for creating a new xterm/... too.
--gdbserver currently only supports sandbox, and not real hardware. That's primarily because the test hooks are responsible for all aspects of hardware control, so there's nothing for the test scripts themselves can do to enable gdbserver on real hardware. We might consider introducing a separate --disable-timeouts option to support use of debuggers on real hardware, and having --gdbserver imply that option.

Quoted 8 lines

Signed-off-by Stephen Warren <[email protected]>

---
 test/py/conftest.py               |  8 ++++++++
 test/py/tests/test_sleep.py       |  7 ++++---
 test/py/u_boot_console_base.py    |  3 ++-
 test/py/u_boot_console_sandbox.py |  5 ++++-
 test/py/u_boot_spawn.py           | 12 ++++++++----
 5 files changed, 26 insertions(+), 9 deletions(-)

Can you please add info about this to the docs?

Also for me this worked up to the point where it ran the test_sandbox_exit.py test. Then the gdb process said that U-Boot exited normally. Is that test not compatible with this feature?

Regards
Simon

Stephen Warren

author

2016-02-06 20:34 UTC #2 lore raw

PATCH 1/1 test/py: support running sandbox under gdbserver

Quoted 65 lines · Simon Glass

Quoted 55 lines · Stephen Warren
Implement command--line option --gdbserver COMM, which does two things:
a) Run the sandbox process under gdbserver, using COMM as gdbserver's communication channel.
b) Disables all timeouts, so that if U-Boot is halted under the debugger, tests don't fail. If the user gives up in the middle of a debugging session, they can simply CTRL-C the test script to abort it.
This allows easy debugging of test failures without having to manually re-create the failure conditions. Usage is:
Window 1:
./test/py/test.py --bd sandbox --gdbserver localhost:1234
Window 2:
gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234'
When using this option, it likely makes sense to use pytest's -k option to limit the set of tests that are executed.
Simply running U-Boot directly under gdb (rather than gdbserver) was also considered. However, this was rejected because:
a) gdb's output would then be processed by the test script, and likely confuse it causing false failures.
b) pytest by default hides stdout from tests, which would prevent the user from interacting with gdb.
While gdb can be told to redirect the debugee's stdio to a separate PTY, this would appear to leave gdb's stdio directed at the test scripts and the debugee's stdio directed elsewhere, which is the opposite of the desired effect. Perhaps some complicated PTY muxing and process hierarchy could invert this. However, the current scheme is simple to implement and use, so it doesn't seem worth complicating matters.
c) Using gdbserver allows arbitrary debuggers to be used, even those with a GUI. If the test scripts invoked the debugger themselves, they'd have to know how to execute arbitary applications. While the user could hide this all in a wrapper script, this feels like extra complication.
An interesting future idea might be a --gdb-screen option, which could spawn both U-Boot and gdb separately, and spawn the screen into a newly created window under screen. Similar options could be envisaged for creating a new xterm/... too.
--gdbserver currently only supports sandbox, and not real hardware. That's primarily because the test hooks are responsible for all aspects of hardware control, so there's nothing for the test scripts themselves can do to enable gdbserver on real hardware. We might consider introducing a separate --disable-timeouts option to support use of debuggers on real hardware, and having --gdbserver imply that option.
Quoted 8 lines
Signed-off-by Stephen Warren <[email protected]>
---
 test/py/conftest.py               |  8 ++++++++
 test/py/tests/test_sleep.py       |  7 ++++---
 test/py/u_boot_console_base.py    |  3 ++-
 test/py/u_boot_console_sandbox.py |  5 ++++-
 test/py/u_boot_spawn.py           | 12 ++++++++----
 5 files changed, 26 insertions(+), 9 deletions(-)

Quoted 5 lines

Can you please add info about this to the docs?
Also for me this worked up to the point where it ran the test_sandbox_exit.py test. Then the gdb process said that U-Boot exited normally. Is that test not compatible with this feature?

The sandbox_exit test deliberately causes the sandbox process to exit, to make sure that the "reset" command and "typing" Ctrl-C work. To continue the test, simply re-run gdb to re-attach to the new gdbserver and U-Boot process.

Simon Glass

2016-02-06 20:39 UTC #3 lore raw

PATCH 1/1 test/py: support running sandbox under gdbserver

Hi Stephen,

Quoted 68 lines · Stephen Warren

Quoted 67 lines · Simon Glass
Quoted 55 lines · Stephen Warren
Implement command--line option --gdbserver COMM, which does two things:
a) Run the sandbox process under gdbserver, using COMM as gdbserver's communication channel.
b) Disables all timeouts, so that if U-Boot is halted under the debugger, tests don't fail. If the user gives up in the middle of a debugging session, they can simply CTRL-C the test script to abort it.
This allows easy debugging of test failures without having to manually re-create the failure conditions. Usage is:
Window 1:
./test/py/test.py --bd sandbox --gdbserver localhost:1234
Window 2:
gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234'
When using this option, it likely makes sense to use pytest's -k option to limit the set of tests that are executed.
Simply running U-Boot directly under gdb (rather than gdbserver) was also considered. However, this was rejected because:
a) gdb's output would then be processed by the test script, and likely confuse it causing false failures.
b) pytest by default hides stdout from tests, which would prevent the user from interacting with gdb.
While gdb can be told to redirect the debugee's stdio to a separate PTY, this would appear to leave gdb's stdio directed at the test scripts and the debugee's stdio directed elsewhere, which is the opposite of the desired effect. Perhaps some complicated PTY muxing and process hierarchy could invert this. However, the current scheme is simple to implement and use, so it doesn't seem worth complicating matters.
c) Using gdbserver allows arbitrary debuggers to be used, even those with a GUI. If the test scripts invoked the debugger themselves, they'd have to know how to execute arbitary applications. While the user could hide this all in a wrapper script, this feels like extra complication.
An interesting future idea might be a --gdb-screen option, which could spawn both U-Boot and gdb separately, and spawn the screen into a newly created window under screen. Similar options could be envisaged for creating a new xterm/... too.
--gdbserver currently only supports sandbox, and not real hardware. That's primarily because the test hooks are responsible for all aspects of hardware control, so there's nothing for the test scripts themselves can do to enable gdbserver on real hardware. We might consider introducing a separate --disable-timeouts option to support use of debuggers on real hardware, and having --gdbserver imply that option.
Quoted 8 lines
Signed-off-by Stephen Warren <[email protected]>
---
 test/py/conftest.py               |  8 ++++++++
 test/py/tests/test_sleep.py       |  7 ++++---
 test/py/u_boot_console_base.py    |  3 ++-
 test/py/u_boot_console_sandbox.py |  5 ++++-
 test/py/u_boot_spawn.py           | 12 ++++++++----
 5 files changed, 26 insertions(+), 9 deletions(-)
Can you please add info about this to the docs?

Quoted 8 lines

Quoted 3 lines
Also for me this worked up to the point where it ran the test_sandbox_exit.py test. Then the gdb process said that U-Boot exited normally. Is that test not compatible with this feature?
The sandbox_exit test deliberately causes the sandbox process to exit, to make sure that the "reset" command and "typing" Ctrl-C work. To continue the test, simply re-run gdb to re-attach to the new gdbserver and U-Boot process.

That's a bit annoying. Perhaps we should have a flag that disabled such tests? Or perhaps a way to specify what tests are run?

Anyway, can you add this to the docs too?

Regards,
Simon

Stephen Warren

author

2016-02-08 18:00 UTC #4 lore raw

PATCH 1/1 test/py: support running sandbox under gdbserver

Quoted 75 lines · Simon Glass

Hi Stephen,
Quoted 72 lines · Stephen Warren
Quoted 70 lines · Simon Glass
Quoted 55 lines · Stephen Warren
Implement command--line option --gdbserver COMM, which does two things:
a) Run the sandbox process under gdbserver, using COMM as gdbserver's communication channel.
b) Disables all timeouts, so that if U-Boot is halted under the debugger, tests don't fail. If the user gives up in the middle of a debugging session, they can simply CTRL-C the test script to abort it.
This allows easy debugging of test failures without having to manually re-create the failure conditions. Usage is:
Window 1:
./test/py/test.py --bd sandbox --gdbserver localhost:1234
Window 2:
gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234'
When using this option, it likely makes sense to use pytest's -k option to limit the set of tests that are executed.
Simply running U-Boot directly under gdb (rather than gdbserver) was also considered. However, this was rejected because:
a) gdb's output would then be processed by the test script, and likely confuse it causing false failures.
b) pytest by default hides stdout from tests, which would prevent the user from interacting with gdb.
While gdb can be told to redirect the debugee's stdio to a separate PTY, this would appear to leave gdb's stdio directed at the test scripts and the debugee's stdio directed elsewhere, which is the opposite of the desired effect. Perhaps some complicated PTY muxing and process hierarchy could invert this. However, the current scheme is simple to implement and use, so it doesn't seem worth complicating matters.
c) Using gdbserver allows arbitrary debuggers to be used, even those with a GUI. If the test scripts invoked the debugger themselves, they'd have to know how to execute arbitary applications. While the user could hide this all in a wrapper script, this feels like extra complication.
An interesting future idea might be a --gdb-screen option, which could spawn both U-Boot and gdb separately, and spawn the screen into a newly created window under screen. Similar options could be envisaged for creating a new xterm/... too.
--gdbserver currently only supports sandbox, and not real hardware. That's primarily because the test hooks are responsible for all aspects of hardware control, so there's nothing for the test scripts themselves can do to enable gdbserver on real hardware. We might consider introducing a separate --disable-timeouts option to support use of debuggers on real hardware, and having --gdbserver imply that option.
Quoted 8 lines
Signed-off-by Stephen Warren <[email protected]>
---
  test/py/conftest.py               |  8 ++++++++
  test/py/tests/test_sleep.py       |  7 ++++---
  test/py/u_boot_console_base.py    |  3 ++-
  test/py/u_boot_console_sandbox.py |  5 ++++-
  test/py/u_boot_spawn.py           | 12 ++++++++----
  5 files changed, 26 insertions(+), 9 deletions(-)
Can you please add info about this to the docs?
Also for me this worked up to the point where it ran the test_sandbox_exit.py test. Then the gdb process said that U-Boot exited normally. Is that test not compatible with this feature?

Quoted 7 lines

Quoted 4 lines
The sandbox_exit test deliberately causes the sandbox process to exit, to make sure that the "reset" command and "typing" Ctrl-C work. To continue the test, simply re-run gdb to re-attach to the new gdbserver and U-Boot process.
That's a bit annoying. Perhaps we should have a flag that disabled such tests? Or perhaps a way to specify what tests are run?

I would expect the user to only run the specific test that needed debugging in order to speed up the whole process. So, unless someone was debugging the reset tests, they wouldn't notice this.

There's already a standard pytest option to select which tests to run; -k.

Quoted 1 line

Anyway, can you add this to the docs too?

Sure.

Tom Rini

2016-02-08 20:49 UTC #5 lore raw

In reply to Stephen Warren

Quoted 50 lines · Stephen Warren

From: Stephen Warren <[email protected]>
Implement command--line option --gdbserver COMM, which does two things:
a) Run the sandbox process under gdbserver, using COMM as gdbserver's communication channel.
b) Disables all timeouts, so that if U-Boot is halted under the debugger, tests don't fail. If the user gives up in the middle of a debugging session, they can simply CTRL-C the test script to abort it.
This allows easy debugging of test failures without having to manually re-create the failure conditions. Usage is:
Window 1:
./test/py/test.py --bd sandbox --gdbserver localhost:1234
Window 2:
gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234'
When using this option, it likely makes sense to use pytest's -k option to limit the set of tests that are executed.
Simply running U-Boot directly under gdb (rather than gdbserver) was also considered. However, this was rejected because:
a) gdb's output would then be processed by the test script, and likely confuse it causing false failures.
b) pytest by default hides stdout from tests, which would prevent the user from interacting with gdb.
While gdb can be told to redirect the debugee's stdio to a separate PTY, this would appear to leave gdb's stdio directed at the test scripts and the debugee's stdio directed elsewhere, which is the opposite of the desired effect. Perhaps some complicated PTY muxing and process hierarchy could invert this. However, the current scheme is simple to implement and use, so it doesn't seem worth complicating matters.
c) Using gdbserver allows arbitrary debuggers to be used, even those with a GUI. If the test scripts invoked the debugger themselves, they'd have to know how to execute arbitary applications. While the user could hide this all in a wrapper script, this feels like extra complication.
An interesting future idea might be a --gdb-screen option, which could spawn both U-Boot and gdb separately, and spawn the screen into a newly created window under screen. Similar options could be envisaged for creating a new xterm/... too.

Quoted 8 lines

--gdbserver currently only supports sandbox, and not real hardware. That's primarily because the test hooks are responsible for all aspects of hardware control, so there's nothing for the test scripts themselves can do to enable gdbserver on real hardware. We might consider introducing a separate --disable-timeouts option to support use of debuggers on real hardware, and having --gdbserver imply that option.
Signed-off-by Stephen Warren <[email protected]>

Applied to u-boot/master, thanks!

Signature · 8 lines

Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160208/036ee7d9/attachment.sig>

Stephen Warren

author

2016-02-08 20:55 UTC #6 lore raw

Quoted 59 lines · Tom Rini

Quoted 57 lines · Stephen Warren
From: Stephen Warren <[email protected]>
Implement command--line option --gdbserver COMM, which does two things:
a) Run the sandbox process under gdbserver, using COMM as gdbserver's communication channel.
b) Disables all timeouts, so that if U-Boot is halted under the debugger, tests don't fail. If the user gives up in the middle of a debugging session, they can simply CTRL-C the test script to abort it.
This allows easy debugging of test failures without having to manually re-create the failure conditions. Usage is:
Window 1:
./test/py/test.py --bd sandbox --gdbserver localhost:1234
Window 2:
gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234'
When using this option, it likely makes sense to use pytest's -k option to limit the set of tests that are executed.
Simply running U-Boot directly under gdb (rather than gdbserver) was also considered. However, this was rejected because:
a) gdb's output would then be processed by the test script, and likely confuse it causing false failures.
b) pytest by default hides stdout from tests, which would prevent the user from interacting with gdb.
While gdb can be told to redirect the debugee's stdio to a separate PTY, this would appear to leave gdb's stdio directed at the test scripts and the debugee's stdio directed elsewhere, which is the opposite of the desired effect. Perhaps some complicated PTY muxing and process hierarchy could invert this. However, the current scheme is simple to implement and use, so it doesn't seem worth complicating matters.
c) Using gdbserver allows arbitrary debuggers to be used, even those with a GUI. If the test scripts invoked the debugger themselves, they'd have to know how to execute arbitary applications. While the user could hide this all in a wrapper script, this feels like extra complication.
An interesting future idea might be a --gdb-screen option, which could spawn both U-Boot and gdb separately, and spawn the screen into a newly created window under screen. Similar options could be envisaged for creating a new xterm/... too.
--gdbserver currently only supports sandbox, and not real hardware. That's primarily because the test hooks are responsible for all aspects of hardware control, so there's nothing for the test scripts themselves can do to enable gdbserver on real hardware. We might consider introducing a separate --disable-timeouts option to support use of debuggers on real hardware, and having --gdbserver imply that option.

Quoted 3 lines

Quoted 1 line
Signed-off-by Stephen Warren <[email protected]>
Applied to u-boot/master, thanks!

Oh. I was just about to send V2 to address some of Simon's comments...

Tom Rini

2016-02-08 21:23 UTC #7 lore raw

Quoted 60 lines · Stephen Warren

Quoted 59 lines · Tom Rini
Quoted 57 lines · Stephen Warren
From: Stephen Warren <[email protected]>
Implement command--line option --gdbserver COMM, which does two things:
a) Run the sandbox process under gdbserver, using COMM as gdbserver's communication channel.
b) Disables all timeouts, so that if U-Boot is halted under the debugger, tests don't fail. If the user gives up in the middle of a debugging session, they can simply CTRL-C the test script to abort it.
This allows easy debugging of test failures without having to manually re-create the failure conditions. Usage is:
Window 1:
./test/py/test.py --bd sandbox --gdbserver localhost:1234
Window 2:
gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234'
When using this option, it likely makes sense to use pytest's -k option to limit the set of tests that are executed.
Simply running U-Boot directly under gdb (rather than gdbserver) was also considered. However, this was rejected because:
a) gdb's output would then be processed by the test script, and likely confuse it causing false failures.
b) pytest by default hides stdout from tests, which would prevent the user from interacting with gdb.
While gdb can be told to redirect the debugee's stdio to a separate PTY, this would appear to leave gdb's stdio directed at the test scripts and the debugee's stdio directed elsewhere, which is the opposite of the desired effect. Perhaps some complicated PTY muxing and process hierarchy could invert this. However, the current scheme is simple to implement and use, so it doesn't seem worth complicating matters.
c) Using gdbserver allows arbitrary debuggers to be used, even those with a GUI. If the test scripts invoked the debugger themselves, they'd have to know how to execute arbitary applications. While the user could hide this all in a wrapper script, this feels like extra complication.
An interesting future idea might be a --gdb-screen option, which could spawn both U-Boot and gdb separately, and spawn the screen into a newly created window under screen. Similar options could be envisaged for creating a new xterm/... too.
--gdbserver currently only supports sandbox, and not real hardware. That's primarily because the test hooks are responsible for all aspects of hardware control, so there's nothing for the test scripts themselves can do to enable gdbserver on real hardware. We might consider introducing a separate --disable-timeouts option to support use of debuggers on real hardware, and having --gdbserver imply that option.

Quoted 5 lines

Quoted 3 lines
Quoted 1 line
Signed-off-by Stephen Warren <[email protected]>
Applied to u-boot/master, thanks!
Oh. I was just about to send V2 to address some of Simon's comments...

Feh. Please send 'em as an iterative patch then :)

Signature · 8 lines

Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160208/cc9d661d/attachment.sig>