Skip to contentSkip to conversation

test/py: make crash detection more robust

PATCH New Stephen Warren 2 participants 5 messages Jan 25, 2016 – Jan 29, 2016

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 test/py: make crash detection more robust

From: Stephen Warren <[email protected]>

test/py contains logic to detect the target crashing and rebooting by searching the console output for a U-Boot signon message, which will presumably be emitted when the system boots after the crash/reset.

Currently, this logic only searches for the exact signon message that was printed by the U-Boot version under test, upon the assumption that binary is written into flash, and hence will be the version booted after any reset. However, this is not a valid assumption; some test setups download the U-Boot-under-test into RAM and boot it from there, and in such a scenario an arbitrary U-Boot version may be located in flash and hence run after any reset.

Fix the reset detection logic to match any U-Boot signon message. This prevents false negatives.

---
 test/py/u_boot_console_base.py | 20 +++++++-------------
 1 file changed, 7 insertions(+), 13 deletions(-)
test/py/u_boot_console_base.py +7 -13
index 51163bc0db68..bb834b0d34ab 100644
@@ -150,12 +150,11 @@ class ConsoleBase(object):
 
         bad_patterns = []
         bad_pattern_ids = []
-        if (self.disable_check_count['spl_signon'] == 0 and
-                self.u_boot_spl_signon):
-            bad_patterns.append(self.u_boot_spl_signon_escaped)
+        if (self.disable_check_count['spl_signon'] == 0):
+            bad_patterns.append(pattern_u_boot_spl_signon)
             bad_pattern_ids.append('SPL signon')
         if self.disable_check_count['main_signon'] == 0:
-            bad_patterns.append(self.u_boot_main_signon_escaped)
+            bad_patterns.append(pattern_u_boot_main_signon)
             bad_pattern_ids.append('U-Boot main signon')
         if self.disable_check_count['unknown_command'] == 0:
             bad_patterns.append(pattern_unknown_command)
@@ -299,18 +298,13 @@ class ConsoleBase(object):
             self.p.logfile_read = self.logstream
             if self.config.buildconfig.get('CONFIG_SPL', False) == 'y':
                 self.p.expect([pattern_u_boot_spl_signon])
-                self.u_boot_spl_signon = self.p.after
-                self.u_boot_spl_signon_escaped = re.escape(self.p.after)
-            else:
-                self.u_boot_spl_signon = None
             self.p.expect([pattern_u_boot_main_signon])
-            self.u_boot_main_signon = self.p.after
-            self.u_boot_main_signon_escaped = re.escape(self.p.after)
-            build_idx = self.u_boot_main_signon.find(', Build:')
+            signon = self.p.after
+            build_idx = signon.find(', Build:')
             if build_idx == -1:
-                self.u_boot_version_string = self.u_boot_main_signon
+                self.u_boot_version_string = signon
             else:
-                self.u_boot_version_string = self.u_boot_main_signon[:build_idx]
+                self.u_boot_version_string = signon[:build_idx]
             while True:
                 match = self.p.expect([self.prompt_escaped,
                                        pattern_stop_autoboot_prompt])
Signature · 1 line
2.7.0
#1 lore raw

PATCH 1/1 test/py: make crash detection more robust

Hi Stephen,

Quoted 66 lines · Stephen Warren

From: Stephen Warren <[email protected]>

test/py contains logic to detect the target crashing and rebooting by searching the console output for a U-Boot signon message, which will presumably be emitted when the system boots after the crash/reset.

Currently, this logic only searches for the exact signon message that was printed by the U-Boot version under test, upon the assumption that binary is written into flash, and hence will be the version booted after any reset. However, this is not a valid assumption; some test setups download the U-Boot-under-test into RAM and boot it from there, and in such a scenario an arbitrary U-Boot version may be located in flash and hence run after any reset.

Fix the reset detection logic to match any U-Boot signon message. This prevents false negatives.

---
 test/py/u_boot_console_base.py | 20 +++++++-------------
 1 file changed, 7 insertions(+), 13 deletions(-)
test/py/u_boot_console_base.py +7 -14
index 51163bc0db68..bb834b0d34ab 100644
@@ -150,12 +150,11 @@ class ConsoleBase(object):
 
         bad_patterns = []
         bad_pattern_ids = []
-        if (self.disable_check_count['spl_signon'] == 0 and
-                self.u_boot_spl_signon):
-            bad_patterns.append(self.u_boot_spl_signon_escaped)
+        if (self.disable_check_count['spl_signon'] == 0):
+            bad_patterns.append(pattern_u_boot_spl_signon)
             bad_pattern_ids.append('SPL signon')
         if self.disable_check_count['main_signon'] == 0:
-            bad_patterns.append(self.u_boot_main_signon_escaped)
+            bad_patterns.append(pattern_u_boot_main_signon)
             bad_pattern_ids.append('U-Boot main signon')
         if self.disable_check_count['unknown_command'] == 0:
             bad_patterns.append(pattern_unknown_command)
@@ -299,18 +298,13 @@ class ConsoleBase(object):
             self.p.logfile_read = self.logstream
             if self.config.buildconfig.get('CONFIG_SPL', False) == 'y':
                 self.p.expect([pattern_u_boot_spl_signon])
-                self.u_boot_spl_signon = self.p.after
-                self.u_boot_spl_signon_escaped = re.escape(self.p.after)
-            else:
-                self.u_boot_spl_signon = None
             self.p.expect([pattern_u_boot_main_signon])
-            self.u_boot_main_signon = self.p.after
-            self.u_boot_main_signon_escaped = re.escape(self.p.after)
-            build_idx = self.u_boot_main_signon.find(', Build:')
+            signon = self.p.after
+            build_idx = signon.find(', Build:')
             if build_idx == -1:
-                self.u_boot_version_string = self.u_boot_main_signon
+                self.u_boot_version_string = signon
             else:
-                self.u_boot_version_string = self.u_boot_main_signon[:build_idx]
+                self.u_boot_version_string = signon[:build_idx]
             while True:
                 match = self.p.expect([self.prompt_escaped,
                                        pattern_stop_autoboot_prompt])
--
Quoted 2 lines

2.7.0

Can you add comments as to what exactly you are looking for? Just 'Build: '?

Regards,
Simon

#2 lore raw

PATCH 1/1 test/py: make crash detection more robust

Quoted 65 lines · Simon Glass

Hi Stephen,

Quoted 62 lines · Stephen Warren

From: Stephen Warren <[email protected]>

test/py contains logic to detect the target crashing and rebooting by searching the console output for a U-Boot signon message, which will presumably be emitted when the system boots after the crash/reset.

Currently, this logic only searches for the exact signon message that was printed by the U-Boot version under test, upon the assumption that binary is written into flash, and hence will be the version booted after any reset. However, this is not a valid assumption; some test setups download the U-Boot-under-test into RAM and boot it from there, and in such a scenario an arbitrary U-Boot version may be located in flash and hence run after any reset.

Fix the reset detection logic to match any U-Boot signon message. This prevents false negatives.

---
  test/py/u_boot_console_base.py | 20 +++++++-------------
  1 file changed, 7 insertions(+), 13 deletions(-)
test/py/u_boot_console_base.py +7 -13
index 51163bc0db68..bb834b0d34ab 100644
@@ -150,12 +150,11 @@ class ConsoleBase(object):
 
          bad_patterns = []
          bad_pattern_ids = []
-        if (self.disable_check_count['spl_signon'] == 0 and
-                self.u_boot_spl_signon):
-            bad_patterns.append(self.u_boot_spl_signon_escaped)
+        if (self.disable_check_count['spl_signon'] == 0):
+            bad_patterns.append(pattern_u_boot_spl_signon)
              bad_pattern_ids.append('SPL signon')
          if self.disable_check_count['main_signon'] == 0:
-            bad_patterns.append(self.u_boot_main_signon_escaped)
+            bad_patterns.append(pattern_u_boot_main_signon)
              bad_pattern_ids.append('U-Boot main signon')
          if self.disable_check_count['unknown_command'] == 0:
              bad_patterns.append(pattern_unknown_command)
@@ -299,18 +298,13 @@ class ConsoleBase(object):
              self.p.logfile_read = self.logstream
              if self.config.buildconfig.get('CONFIG_SPL', False) == 'y':
                  self.p.expect([pattern_u_boot_spl_signon])
-                self.u_boot_spl_signon = self.p.after
-                self.u_boot_spl_signon_escaped = re.escape(self.p.after)
-            else:
-                self.u_boot_spl_signon = None
              self.p.expect([pattern_u_boot_main_signon])
-            self.u_boot_main_signon = self.p.after
-            self.u_boot_main_signon_escaped = re.escape(self.p.after)
-            build_idx = self.u_boot_main_signon.find(', Build:')
+            signon = self.p.after
+            build_idx = signon.find(', Build:')
              if build_idx == -1:
-                self.u_boot_version_string = self.u_boot_main_signon
+                self.u_boot_version_string = signon
              else:
-                self.u_boot_version_string = self.u_boot_main_signon[:build_idx]
+                self.u_boot_version_string = signon[:build_idx]
Quoted 8 lines
Quoted 6 lines
test/py/u_boot_console_base.py +0 -1
              while True:
                  match = self.p.expect([self.prompt_escaped,
                                         pattern_stop_autoboot_prompt])
--

2.7.0

Can you add comments as to what exactly you are looking for? Just 'Build: '?

The pattern is at the top of the file, so very easy to find simply by reading the assignment to pattern_u_boot_main_signon. I don't think adding a comment would be terribly useful in case it got stale.

#3 lore raw

PATCH 1/1 test/py: make crash detection more robust

Hi Stephen,

Quoted 81 lines · Stephen Warren
Quoted 70 lines · Simon Glass

Hi Stephen,

Quoted 66 lines · Stephen Warren

From: Stephen Warren <[email protected]>

test/py contains logic to detect the target crashing and rebooting by searching the console output for a U-Boot signon message, which will presumably be emitted when the system boots after the crash/reset.

Currently, this logic only searches for the exact signon message that was printed by the U-Boot version under test, upon the assumption that binary is written into flash, and hence will be the version booted after any reset. However, this is not a valid assumption; some test setups download the U-Boot-under-test into RAM and boot it from there, and in such a scenario an arbitrary U-Boot version may be located in flash and hence run after any reset.

Fix the reset detection logic to match any U-Boot signon message. This prevents false negatives.

---
  test/py/u_boot_console_base.py | 20 +++++++-------------
  1 file changed, 7 insertions(+), 13 deletions(-)
test/py/u_boot_console_base.py +6 -13
index 51163bc0db68..bb834b0d34ab 100644
@@ -150,12 +150,11 @@ class ConsoleBase(object):
 
          bad_patterns = []
          bad_pattern_ids = []
-        if (self.disable_check_count['spl_signon'] == 0 and
-                self.u_boot_spl_signon):
-            bad_patterns.append(self.u_boot_spl_signon_escaped)
+        if (self.disable_check_count['spl_signon'] == 0):
+            bad_patterns.append(pattern_u_boot_spl_signon)
              bad_pattern_ids.append('SPL signon')
          if self.disable_check_count['main_signon'] == 0:
-            bad_patterns.append(self.u_boot_main_signon_escaped)
+            bad_patterns.append(pattern_u_boot_main_signon)
              bad_pattern_ids.append('U-Boot main signon')
          if self.disable_check_count['unknown_command'] == 0:
              bad_patterns.append(pattern_unknown_command)
@@ -299,18 +298,13 @@ class ConsoleBase(object):
              self.p.logfile_read = self.logstream
              if self.config.buildconfig.get('CONFIG_SPL', False) == 'y':
                  self.p.expect([pattern_u_boot_spl_signon])
-                self.u_boot_spl_signon = self.p.after
-                self.u_boot_spl_signon_escaped = re.escape(self.p.after)
-            else:
-                self.u_boot_spl_signon = None
              self.p.expect([pattern_u_boot_main_signon])
-            self.u_boot_main_signon = self.p.after
-            self.u_boot_main_signon_escaped = re.escape(self.p.after)
-            build_idx = self.u_boot_main_signon.find(', Build:')
+            signon = self.p.after
+            build_idx = signon.find(', Build:')
              if build_idx == -1:
-                self.u_boot_version_string = self.u_boot_main_signon
+                self.u_boot_version_string = signon
              else:
-                self.u_boot_version_string =

self.u_boot_main_signon[:build_idx]
+ self.u_boot_version_string = signon[:build_idx]
while True:

Quoted 8 lines
Quoted 5 lines

match = self.p.expect([self.prompt_escaped,
pattern_stop_autoboot_prompt])
--
2.7.0

Can you add comments as to what exactly you are looking for? Just 'Build:
'?

Quoted 3 lines

The pattern is at the top of the file, so very easy to find simply by reading the assignment to pattern_u_boot_main_signon. I don't think adding a comment would be terribly useful in case it got stale.

OK it's a regex. That makes sense.

Regards,
Simon

#4 lore raw

PATCH 1/1 test/py: make crash detection more robust

Quoted 30 lines · Simon Glass

Hi Stephen,

Quoted 27 lines · Stephen Warren
Quoted 26 lines · Simon Glass

Hi Stephen,

Quoted 22 lines · Stephen Warren

From: Stephen Warren <[email protected]>

test/py contains logic to detect the target crashing and rebooting by searching the console output for a U-Boot signon message, which will presumably be emitted when the system boots after the crash/reset.

Currently, this logic only searches for the exact signon message that was printed by the U-Boot version under test, upon the assumption that binary is written into flash, and hence will be the version booted after any reset. However, this is not a valid assumption; some test setups download the U-Boot-under-test into RAM and boot it from there, and in such a scenario an arbitrary U-Boot version may be located in flash and hence run after any reset.

Fix the reset detection logic to match any U-Boot signon message. This prevents false negatives.

---
  test/py/u_boot_console_base.py | 20 +++++++-------------
  1 file changed, 7 insertions(+), 13 deletions(-)

Applied to u-boot-dm, thanks!