test/py: ignore console read exceptions after test failure
PATCH v2 1/1 test/py: ignore console read exceptions after test failure
From: Stephen Warren <[email protected]>
After a test has failed, test/py drains the U-Boot console log to ensure that any relevant output is captured. At this point, we don't care about detecting any additional errors, since the test is already known to have failed, and U-Boot will be restarted. To ensure that the test cleanup code is not interrupted, and can correctly terminate the log sections for the failed test, ignore any exception that occurs while reading the U-Boot console output during this limited period of time.
---
v2:
* Combine the two except cases into one, since one was a superset of the other.
* Fix TAB-vs-spaces issue in identation.
--- test/py/u_boot_console_base.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
test/py/u_boot_console_base.py +11 -1
@@ -304,7 +304,17 @@ class ConsoleBase(object): # Wait for something U-Boot will likely never send. This will # cause the console output to be read and logged. self.p.expect(['This should never match U-Boot output']) - except u_boot_spawn.Timeout: + except: + # We expect a timeout, since U-Boot won't print what we waited + # for. Squash it when it happens. + # + # Squash any other exception too. This function is only used to + # drain (and log) the U-Boot console output after a failed test. + # The U-Boot process will be restarted, or target board reset, once + # this function returns. So, we don't care about detecting any + # additional errors, so they're squashed so that the rest of the + # post-test-failure cleanup code can continue operation, and + # correctly terminate any log sections, etc. pass finally: self.p.timeout = orig_timeout
Signature · 1 line
2.18.0
PATCH v2 1/1 test/py: ignore console read exceptions after test failure
Quoted 34 lines · Stephen Warren
From: Stephen Warren <[email protected]>
After a test has failed, test/py drains the U-Boot console log to ensure that any relevant output is captured. At this point, we don't care about detecting any additional errors, since the test is already known to have failed, and U-Boot will be restarted. To ensure that the test cleanup code is not interrupted, and can correctly terminate the log sections for the failed test, ignore any exception that occurs while reading the U-Boot console output during this limited period of time.
Signed-off-by Stephen Warren <[email protected]>---
v2:
* Combine the two except cases into one, since one was a superset of the other.
* Fix TAB-vs-spaces issue in identation.--- test/py/u_boot_console_base.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)test/py/u_boot_console_base.py +4
-1@@ -304,7 +304,17 @@ class ConsoleBase(object): # Wait for something U-Boot will likely never send. This will # cause the console output to be read and logged. self.p.expect(['This should never match U-Boot output']) - except u_boot_spawn.Timeout: + except: + # We expect a timeout, since U-Boot won't print what we waited + # for. Squash it when it happens. + #
test/py/u_boot_console_base.py class ConsoleBase(object): · line 320
Quoted 10 lines
test/py/u_boot_console_base.py +7
-0+ # Squash any other exception too. This function is only used to + # drain (and log) the U-Boot console output after a failed test. + # The U-Boot process will be restarted, or target board reset, once + # this function returns. So, we don't care about detecting any + # additional errors, so they're squashed so that the rest of the + # post-test-failure cleanup code can continue operation, and + # correctly terminate any log sections, etc. pass finally: self.p.timeout = orig_timeout
This patch only covers this single usage of the expect(). Expect() is called from many test cases directly. In all these cases we still will not catch OSErrror if the U-Boot binary terminates breaking the pipes.
That is why in my patch I put the except statement into the except() method itself.
What problem do you see in catching the exception where it is arising?
Best regards
Heinrich
PATCH v2 1/1 test/py: ignore console read exceptions after test failure
Quoted 11 lines · Heinrich Schuchardt
Quoted 10 lines · Stephen Warren
From: Stephen Warren <[email protected]>
After a test has failed, test/py drains the U-Boot console log to ensure that any relevant output is captured. At this point, we don't care about detecting any additional errors, since the test is already known to have failed, and U-Boot will be restarted. To ensure that the test cleanup code is not interrupted, and can correctly terminate the log sections for the failed test, ignore any exception that occurs while reading the U-Boot console output during this limited period of time.
Quoted 8 lines
Quoted 8 lines
Signed-off-by Stephen Warren <[email protected]>---
v2:
* Combine the two except cases into one, since one was a superset of the other.
* Fix TAB-vs-spaces issue in identation.
---
test/py/u_boot_con
sole_base.py | 12 +++++++++++-
Quoted 21 lines
Quoted 21 lines
1 file changed, 11 insertions(+), 1 deletion(-)test/py/u_boot_console_base.py +10
-1@@ -304,7 +304,17 @@ class ConsoleBase(object): # Wait for something U-Boot will likely never send. This will # cause the console output to be read and logged. self.p.expect(['This should never match U-Boot output']) - except u_boot_spawn.Timeout: + except: + # We expect a timeout, since U-Boot won't print what we waited + # for. Squash it when it happens. + # + # Squash any other exception too. This function is only used to + # drain (and log) the U-Boot console output after a failed test. + # The U-Boot process will be restarted, or target board reset, once + # this function returns. So, we don't care about detecting any + # additional errors, so they're squashed so that the rest of the + # post-test-failure cleanup code can continue operation, and
Quoted 8 lines
test/py/u_boot_console_base.py class ConsoleBase(object): · line 320
Quoted 5 lines
test/py/u_boot_console_base.py +1
-0+ # correctly terminate any log sections, etc. pass finally: self.p.timeout = orig_timeoutThis patch only covers this single usage of the
expect().Expect()is called from many test cases directly. In all these cases we still will not catch OSErrror if the U-Boot binary terminates breaking the pipes.
That is deliberate since this approach is correct.
Quoted 4 lines
That is why in my patch I put the except statement into the
except()method itself.What problem do you see in catching the exception where it is arising?
Because it ignores the exception. The correct approach is to propagate the exception, fail the test, and then clean up. Ignoring the exception and returning nothing from the expect logic may or may not actually lead to detecting the error, depending on how the individual test code ends up checking the result from the expect call. Many tests don't, because they know that if an error occurs, and exception will be thrown through the test code and fail it without the test code having to explicitly check for errors.
PATCH v2 1/1 test/py: ignore console read exceptions after test failure
Quoted 67 lines · Stephen Warren
Quoted 50 lines · Heinrich Schuchardt
Quoted 49 lines · Stephen Warren
From: Stephen Warren <[email protected]>
After a test has failed, test/py drains the U-Boot console log to ensure that any relevant output is captured. At this point, we don't care about detecting any additional errors, since the test is already known to have failed, and U-Boot will be restarted. To ensure that the test cleanup code
is not interrupted, and can correctly terminate the log sections for the failed test, ignore any exception that occurs while reading the U-Boot console output during this limited period of time.Signed-off-by Stephen Warren <[email protected]>---
v2:
* Combine the two except cases into one, since one was a superset of the other.
* Fix TAB-vs-spaces issue in identation.
---
test/py/u_boot_console_base.py| 12 +++++++++++-1 file changed, 11 insertions(+), 1 deletion(-)test/py/u_boot_console_base.py +0
-0@@ -304,7 +304,17 @@ class ConsoleBase(object): # Wait for something U-Boot will likely never send.This will
# cause the console output to be read and logged. self.p.expect(['This should never match U-Boot output']) - except u_boot_spawn.Timeout: + except: + # We expect a timeout, since U-Boot won't print what we waited + # for. Squash it when it happens. + # + # Squash any other exception too. This function is only used to + # drain (and log) the U-Boot console output after a failed test. + # The U-Boot process will be restarted, or target board reset, once + # this function returns. So, we don't care about detecting any + # additional errors, so they're squashed so that the rest of the + # post-test-failure cleanup code can continue operation, andQuoted 8 lines
Quoted 5 lines
+ # correctly terminate any log sections, etc. pass
finally:
self.p.timeout =orig_timeoutThis patch only covers this single usage of the
expect().Expect()is called from many test cases directly. In all these cases we still will not catch OSErrror if the U-Boot binary terminates breaking the pipes.That is deliberate since this approach is correct.
Quoted 4 lines
That is why in my patch I put the except statement into the
except()method itself.What problem do you see in catching the exception where it is arising?
Quoted 8 lines
Because it ignores the exception. The correct approach is to propagate the exception, fail the test, and then clean up. Ignoring the exception and returning nothing from the expect logic may or may not actually lead to detecting the error, depending on how the individual test code ends up checking the result from the expect call. Many tests don't, because they know that if an error occurs, and exception will be thrown through the test code and fail it without the test code having to explicitly check for errors.
Tom, do you plan to apply this patch? Thanks.
PATCH v2 1/1 test/py: ignore console read exceptions after test failure
In reply to Stephen WarrenQuoted 11 lines · Stephen Warren
From: Stephen Warren <[email protected]>
After a test has failed, test/py drains the U-Boot console log to ensure that any relevant output is captured. At this point, we don't care about detecting any additional errors, since the test is already known to have failed, and U-Boot will be restarted. To ensure that the test cleanup code is not interrupted, and can correctly terminate the log sections for the failed test, ignore any exception that occurs while reading the U-Boot console output during this limited period of time.
Signed-off-by Stephen Warren <[email protected]>
Applied to u-boot/master, thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://lists.denx.de/pipermail/u-boot/attachments/20181006/c469c366/attachment.sig>