Skip to contentSkip to conversation

qa crashme

PATCH Superseded Armin Kuster 2 participants 4 messages Nov 16, 2019 – Nov 17, 2019

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover qa crashme

Instead of reworking the ltp class at this time, here is the crashme test from Manual in an automated form.

Armin Kuster (2):
oeqa/manual/compliance-test: move crashme to runtime oeqa/manual/compliance-test: remove crashme tests

 meta/lib/oeqa/manual/compliance-test.json | 48 ---------------
 meta/lib/oeqa/runtime/cases/ltp_stress.py | 97 +++++++++++++++++++++++++++++++
 2 files changed, 97 insertions(+), 48 deletions(-)
 create mode 100644 meta/lib/oeqa/runtime/cases/ltp_stress.py
Signature · 1 line
2.7.4
#1 lore raw

PATCH v2 1/2 oeqa/manual/compliance-test: move crashme to runtime

---
 meta/lib/oeqa/runtime/cases/ltp_stress.py | 97 +++++++++++++++++++++++++++++++
 1 file changed, 97 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/cases/ltp_stress.py
meta/lib/oeqa/runtime/cases/ltp_stress.py new file +97 -0
new file mode 100644
index 0000000..98d65a7
@@ -0,0 +1,97 @@
+# LTP Stress runtime
+#
+# Copyright (c) 2019 MontaVista Software, LLC
+#
+# SPDX-License-Identifier: MIT
+#
+
+import time
+import datetime
+import pprint
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.runtime.decorator.package import OEHasPackage
+from oeqa.core.decorator.data import skipIfQemu
+from oeqa.utils.logparser import LtpParser
+
+class LtpStressBase(OERuntimeTestCase):
+
+    @classmethod
+    def setUpClass(cls):
+        cls.ltp_startup()
+
+    @classmethod
+    def tearDownClass(cls):
+        cls.ltp_finishup()
+
+    @classmethod
+    def ltp_startup(cls):
+        cls.sections = {}
+        cls.failmsg = ""
+        test_log_dir = os.path.join(cls.td.get('WORKDIR', ''), 'testimage')
+        timestamp = datetime.datetime.now().strftime('%Y%m%d%H%M%S')
+
+        cls.ltptest_log_dir_link = os.path.join(test_log_dir, 'ltpstress_log')
+        cls.ltptest_log_dir = '%s.%s' % (cls.ltptest_log_dir_link, timestamp)
+        os.makedirs(cls.ltptest_log_dir)
+
+        cls.tc.target.run("mkdir -p /opt/ltp/results")
+
+        if not hasattr(cls.tc, "extraresults"):
+            cls.tc.extraresults = {}
+        cls.extras = cls.tc.extraresults
+        cls.extras['ltpstressresult.rawlogs'] = {'log': ""}
+
+ 
+    @classmethod
+    def ltp_finishup(cls):
+        cls.extras['ltpstressresult.sections'] =  cls.sections
+
+        # update symlink to ltp_log
+        if os.path.exists(cls.ltptest_log_dir_link):
+            os.remove(cls.ltptest_log_dir_link)
+
+        os.symlink(os.path.basename(cls.ltptest_log_dir), cls.ltptest_log_dir_link)
+
+        if cls.failmsg:
+            cls.fail(cls.failmsg)
+
+class LtpStressTest(LtpStressBase):
+
+    def runltp(self, stress_group):
+            cmd = '/opt/ltp/runltp -f %s -p -q 2>@1 | tee /opt/ltp/results/%s' % (stress_group, stress_group)
+            starttime = time.time()
+            (status, output) = self.target.run(cmd)
+            endtime = time.time()
+            with open(os.path.join(self.ltptest_log_dir, "%s" % stress_group), 'w') as f:
+                f.write(output)
+
+            self.extras['ltpstressresult.rawlogs']['log'] = self.extras['ltpstressresult.rawlogs']['log'] + output
+
+            parser = LtpParser()
+            results, sections  = parser.parse(os.path.join(self.ltptest_log_dir, "%s" % stress_group))
+
+            runtime = int(endtime-starttime)
+            sections['duration'] = runtime
+            self.sections[stress_group] =  sections
+ 
+            failed_tests = {}
+            for test in results:
+                result = results[test]
+                testname = ("ltpstressresult." + stress_group + "." + test)
+                self.extras[testname] = {'status': result}
+                if result == 'FAILED':
+                    failed_tests[stress_group] = test 
+
+            if failed_tests:
+                self.failmsg = self.failmsg + "Failed ptests:\n%s" % pprint.pformat(failed_tests)
+
+    # LTP stress runtime tests
+    #
+    @skipIfQemu('qemuall', 'Test only runs on real hardware')
+
+    @OETestDepends(['ssh.SSHTest.test_ssh'])
+    @OEHasPackage(["ltp"])
+    def test_ltp_stress(self):
+        self.runltp('crashme')
Signature · 1 line
2.7.4
#2 lore raw

PATCH v2 2/2 oeqa/manual/compliance-test: remove crashme tests

In reply to Armin Kuster
---
 meta/lib/oeqa/manual/compliance-test.json | 48 -------------------------------
 1 file changed, 48 deletions(-)
meta/lib/oeqa/manual/compliance-test.json +0 -48
index 367a416..e374c5b 100644
@@ -1,52 +1,4 @@
 [
-    {
-        "test": {
-            "@alias": "compliance-test.compliance-test.stress_test_-_Genericx86-64",
-            "author": [
-                {
-                    "email": "[email protected]",
-                    "name": "[email protected]"
-                }
-            ],
-            "execution": {
-                "1": {
-                    "action": "Bootup with core-image-sato-sdk image",
-                    "expected_results": ""
-                },
-                "2": {
-                    "action": "Execute the crashme test with below command  \n\n./opt/ltp/runltp f  crashme",
-                    "expected_results": "The stress testing should not make the target crash. Check CPU usage and basic functionality of the system after the tests are over. "
-                }
-            },
-            "summary": "stress_test_-_Genericx86-64"
-        }
-    },
-     {
-    "test": {
-      "@alias": "compliance-test.compliance-test.stress_test_-_- crashme_-_-Beaglebone",
-      "author": [
-        {
-          "email": "[email protected]",
-          "name": "[email protected]"
-        }
-      ],
-      "execution": {
-        "1": {
-          "action": " Get crashme from http://people.delphiforums.com/gjc/crashme.html",
-          "expected_results": ""
-        },
-        "2": {
-          "action": "Follow the setup steps on above URL, build crashme in target",
-          "expected_results": ""
-        },
-        "3": {
-          "action": " Run crashme for 24 hours",
-          "expected_results": "Target should not crash with the program."
-        }
-      },
-      "summary": "stress_test_-_crashme_-Beaglebone"
-    }
-  },
   {
     "test": {
       "@alias": "compliance-test.compliance-test.stress_test_-_ltp_-Beaglebone",
Signature · 1 line
2.7.4
#3 lore raw

✗ patchtest: failure for qa crashme

In reply to Armin Kuster

== Series Details ==

Series: qa crashme
Revision: 1
URL : https://patchwork.openembedded.org/series/21178/
State : failure

== Summary ==

Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:

* Issue Errors in your Python code were encountered [test_pylint]
Suggested fix Correct the lines introduced by your patch
Output Please, fix the listed issues:
meta/lib/oeqa/runtime/cases/ltp_stress.py does not exist

If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).

---

Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe