Skip to contentSkip to conversation

/oeqa/manual/compliance-test: remove obsolete test

PATCH In review Armin Kuster 2 participants 8 messages Nov 17, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/7 /oeqa/manual/compliance-test: remove obsolete test

ltpstress was removed in Oct 2018

https://github.com/linux-test-project/ltp/commit/e752f7c19674d9b2f7d37aed123561a3f6410e97#diff-5231627fc8640e0adb955f9e69c3c08d

Remove LTP stress tests
ltpstress.sh runs stress.part[1-3]. But these runtest files just duplicate definitions:
* stress.part1: fs, mm, nfs
* stress.part2: ipc, math, nptl
* stress.part3: net.multicast, pty, syscalls

The definitions are outdated anyway. There is no point trying to keep them sync.

---
 meta/lib/oeqa/manual/compliance-test.json | 28 ----------------------------
 1 file changed, 28 deletions(-)
 delete mode 100644 meta/lib/oeqa/manual/compliance-test.json
meta/lib/oeqa/manual/compliance-test.json deleted +0 -28
deleted file mode 100644
index e374c5b..0000000
@@ -1,28 +0,0 @@
-[
-  {
-    "test": {
-      "@alias": "compliance-test.compliance-test.stress_test_-_ltp_-Beaglebone",
-      "author": [
-        {
-          "email": "[email protected]",
-          "name": "[email protected]"
-        }
-      ],
-      "execution": {
-        "1": {
-          "action": "Build LTP with toolchain or in sdk image",
-          "expected_results": ""
-        },
-        "2": {
-          "action": "Copy LTP folder into target, for example, /opt/ltp. Modify script,  testscripts/ltpstress.sh, set Iostat=1, NO_NETWORK=1",
-          "expected_results": ""
-        },
-        "3": {
-          "action": "cd testscripts/ && ./ltpstress.sh",
-          "expected_results": "This stress case will run for 24 hours Check the result\ntarget should not crash with the program "
-        }
-      },
-      "summary": "stress_test_-_-ltp_-Beaglebone"
-    }
-  }
-]
Signature · 1 line
2.7.4
#1 lore raw

PATCH 2/7 oeqa/manual/bsp-hw: remove rpm -ivh test

done via runtime

def test_rpm_install(self):

---
 meta/lib/oeqa/manual/bsp-hw.json | 28 +---------------------------
 1 file changed, 1 insertion(+), 27 deletions(-)
meta/lib/oeqa/manual/bsp-hw.json +1 -27
index 18cec22..2ab7dda 100644
@@ -1,32 +1,6 @@
 [
     {
         "test": {
-            "@alias": "bsps-hw.bsps-hw.rpm_-__install_dependency_package",
-            "author": [
-                {
-                    "email": "[email protected]",
-                    "name": "[email protected]"
-                }
-            ],
-            "execution": {
-                "1": {
-                    "action": "Get a not previously installed RPM package or build one on local machine, which should have run-time dependency.For example, \"mc\" (Midnight Commander, which is a visual file manager) should depend on \"ncurses-terminfo\".   \n\n$ bitbake mc  \n\n\n",
-                    "expected_results": ""
-                },
-                "2": {
-                    "action": "Copy the package into a system folder (for example /home/root/rpm_packages).  \n\n\n",
-                    "expected_results": ""
-                },
-                "3": {
-                    "action": "Run \"rpm -ivh package_name\" and check the output, for example \"rpm -ivh mc.rpm*\" should report the dependency on \"ncurses-terminfo\".\n\n\n\n",
-                    "expected_results": "3 . rpm command should report message when some RPM installation depends on other packages."
-                }
-            },
-            "summary": "rpm_-__install_dependency_package"
-        }
-    },
-    {
-        "test": {
             "@alias": "bsps-hw.bsps-hw.boot_and_install_from_USB",
             "author": [
                 {
@@ -1283,4 +1257,4 @@
             "summary": "Boot_from_JFFS2_image"
         }
     }
-]
\ No newline at end of file
+]
Signature · 1 line
2.7.4
#2 lore raw

PATCH 3/7 oeqa/runtime/boot: add reboot test

In reply to Armin Kuster
---
 meta/lib/oeqa/runtime/cases/boot.py | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/cases/boot.py
meta/lib/oeqa/runtime/cases/boot.py new file +33 -0
new file mode 100644
index 0000000..2142f40
@@ -0,0 +1,33 @@
+#
+# SPDX-License-Identifier: MIT
+#
+
+from subprocess import Popen, PIPE
+import time
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oetimeout import OETimeout
+from oeqa.core.decorator.data import skipIfQemu
+
+class BootTest(OERuntimeTestCase):
+
+    @OETimeout(120)
+    @skipIfQemu('qemuall', 'Test only runs on real hardware')
+    @OETestDepends(['ssh.SSHTest.test_ssh'])
+    def test_reboot(self):
+        output = ''
+        count = 0
+        (status, output) = self.target.run('reboot -h')
+        while count < 5:
+            time.sleep(5)
+            cmd = 'ping -c 1 %s' % self.target.ip
+            proc = Popen(cmd, shell=True, stdout=PIPE)
+            output += proc.communicate()[0].decode('utf-8')
+            if proc.poll() == 0:
+                count += 1
+            else:
+                count = 0
+        msg = ('Expected 5 consecutive, got %d.\n'
+               'ping output is:\n%s' % (count,output))
+        self.assertEqual(count, 5, msg = msg)
Signature · 1 line
2.7.4
#3 lore raw

PATCH 4/7 oeqa/manual/bsp-hw: remove reboot test

In reply to Armin Kuster
---
 meta/lib/oeqa/manual/bsp-hw.json | 22 ----------------------
 1 file changed, 22 deletions(-)
meta/lib/oeqa/manual/bsp-hw.json +0 -22
index 2ab7dda..9421c50 100644
@@ -147,28 +147,6 @@
     },
     {
         "test": {
-            "@alias": "bsps-hw.bsps-hw.reboot_system",
-            "author": [
-                {
-                    "email": "[email protected]",
-                    "name": "[email protected]"
-                }
-            ],
-            "execution": {
-                "1": {
-                    "action": "boot system",
-                    "expected_results": ""
-                },
-                "2": {
-                    "action": "launch terminal and run \"reboot\"",
-                    "expected_results": "System can reboot successfully . "
-                }
-            },
-            "summary": "reboot_system"
-        }
-    },
-    {
-        "test": {
             "@alias": "bsps-hw.bsps-hw.switch_among_multi_applications_and_desktop",
             "author": [
                 {
Signature · 1 line
2.7.4
#4 lore raw

PATCH 5/7 oeqa/manual/bsp-hw: move storage tests to runtime

In reply to Armin Kuster

move usb mount, r/w and umount tests to runtime move SD card mount, r/w and umount tests to runtim

---
 meta/lib/oeqa/runtime/cases/storage.py | 149 +++++++++++++++++++++++++++++++++
 1 file changed, 149 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/cases/storage.py
meta/lib/oeqa/runtime/cases/storage.py new file +149 -0
new file mode 100644
index 0000000..166d26b
@@ -0,0 +1,149 @@
+#
+# SPDX-License-Identifier: MIT
+#
+
+import re
+import time
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.data import skipIfQemu
+
+class StorageBase(OERuntimeTestCase):
+    def storage_mount(cls, tmo=1):
+
+        (status, output) = cls.target.run('mkdir -p %s' % cls.mount_point)
+        (status, output) = cls.target.run('mount %s %s' % (cls.device, cls.mount_point))
+        msg = ('Mount failed: %s.' % status)
+        cls.assertFalse(output, msg = msg)
+        time.sleep(tmo)
+        (status, output) = cls.target.run('cat /proc/mounts')
+        match = re.search('%s' % cls.device, output)
+        if match:
+            msg = ('Device %s not mounted.' % cls.device)
+            cls.assertTrue(match, msg = msg)
+
+        (status, output) = cls.target.run('mkdir -p %s' % cls.test_dir)
+
+        (status, output) = cls.target.run('rm -f %s/*' % cls.test_dir)
+        msg = ('Failed to cleanup files @ %s/*' % cls.test_dir)
+        cls.assertFalse(output, msg = msg)
+
+
+    def storage_basic(cls):
+        # create file on device
+        (status, output) = cls.target.run('touch %s/%s' % (cls.test_dir, cls.test_file))
+        msg = ('File %s not created on %s' % (cls.test_file, cls.device))
+        cls.assertFalse(status, msg = msg)
+        # move file
+        (status, output) = cls.target.run('mv %s/%s %s/%s1' %  
+                (cls.test_dir, cls.test_file, cls.test_dir, cls.test_file))
+        msg = ('File %s not moved to %s' % (cls.test_file, cls.device))
+        cls.assertFalse(status, msg = msg)
+        # remove file
+        (status, output) = cls.target.run('rm %s/%s1' % (cls.test_dir, cls.test_file))
+        msg = ('File %s not removed on %s' % (cls.test_file, cls.device))
+        cls.assertFalse(status, msg = msg)
+
+    def storage_read(cls):
+        # check if message is in file
+        (status, output) = cls.target.run('cat  %s/%s' % 
+                (cls.test_dir, cls.test_file))
+
+        match = re.search('%s' % cls.test_msg, output)
+        msg = ('Test message %s not in file %s.' % (cls.test_msg, cls.test_file))
+        cls.assertEqual(status, 0,  msg = msg)
+
+    def storage_write(cls):
+        # create test message in file on device
+        (status, output) = cls.target.run('echo "%s" >  %s/%s' % 
+                (cls.test_msg, cls.test_dir, cls.test_file))
+        msg = ('File %s not create test message on %s' % (cls.test_file, cls.device))
+        cls.assertEqual(status, 0,  msg = msg)
+
+    def storage_umount(cls, tmo=1):
+        time.sleep(tmo)
+        (status, output) = cls.target.run('umount %s' % cls.mount_point)
+
+        if status == 32:
+            # already unmounted, should it fail?
+            return
+        else:
+            msg = ('Device not unmount %s' % cls.mount_point)
+            cls.assertEqual(status, 0,  msg = msg)
+
+        (status, output) = cls.target.run('cat /proc/mounts')
+        match = re.search('%s' % cls.device, output)
+        if match:
+            msg = ('Device %s still mounted.' % cls.device)
+            cls.assertTrue(match, msg = msg)
+
+
+class UsbTest(StorageBase):
+    '''
+        This is to mimic the usb test previously done in manual bsp-hw.json
+    '''
+    @classmethod
+    def setUpClass(self):
+        self.test_msg = "Hello World - USB"
+        self.mount_point = "/media/usb"
+        self.device = "/dev/sda1"
+        self.test_file = "usb.tst"
+        self.test_dir = os.path.join(self.mount_point, "oeqa")
+
+    @skipIfQemu('qemuall', 'Test only runs on real hardware')
+    @OETestDepends(['ssh.SSHTest.test_ssh'])
+    def test_usb_mount(self):
+        self.storage_umount(2)
+        self.storage_mount(5)
+
+    @skipIfQemu('qemuall', 'Test only runs on real hardware')
+    @OETestDepends(['storage.UsbTest.test_usb_mount'])
+    def test_usb_basic_operations(self):
+        self.storage_basic()
+
+    @skipIfQemu('qemuall', 'Test only runs on real hardware')
+    @OETestDepends(['storage.UsbTest.test_usb_basic_operations'])
+    def test_usb_basic_rw(self):
+        self.storage_write()
+        self.storage_read()
+
+    @skipIfQemu('qemuall', 'Test only runs on real hardware')
+    @OETestDepends(['storage.UsbTest.test_usb_mount'])
+    def test_usb_umount(self):
+        self.storage_umount(2)
+
+
+class MMCTest(StorageBase):
+    '''
+        This is to mimic the usb test previously done in manual bsp-hw.json
+    '''
+    @classmethod
+    def setUpClass(self):
+        self.test_msg = "Hello World - MMC"
+        self.mount_point = "/media/mmc"
+        self.device = "/dev/mmcblk1p1"
+        self.test_file = "mmc.tst"
+        self.test_dir = os.path.join(self.mount_point, "oeqa")
+
+    @skipIfQemu('qemuall', 'Test only runs on real hardware')
+    @OETestDepends(['ssh.SSHTest.test_ssh'])
+    def test_mmc_mount(self):
+        self.storage_umount(2)
+        self.storage_mount()
+
+    @skipIfQemu('qemuall', 'Test only runs on real hardware')
+    @OETestDepends(['storage.MMCTest.test_mmc_mount'])
+    def test_mmc_basic_operations(self):
+        self.storage_basic()
+
+    @skipIfQemu('qemuall', 'Test only runs on real hardware')
+    @OETestDepends(['storage.MMCTest.test_mmc_basic_operations'])
+    def test_mmc_basic_rw(self):
+        self.storage_write()
+        self.storage_read()
+
+    @skipIfQemu('qemuall', 'Test only runs on real hardware')
+    @OETestDepends(['storage.MMCTest.test_mmc_mount'])
+    def test_mmc_umount(self):
+        self.storage_umount(2)
Signature · 1 line
2.7.4
#5 lore raw

PATCH 6/7 oeqa/manual/bsp-hw: remove usb and SDmicro tests

In reply to Armin Kuster
---
 meta/lib/oeqa/manual/bsp-hw.json | 220 ---------------------------------------
 1 file changed, 220 deletions(-)
meta/lib/oeqa/manual/bsp-hw.json +0 -220
index 9421c50..dd3504f 100644
@@ -177,114 +177,6 @@
     },
     {
         "test": {
-            "@alias": "bsps-hw.bsps-hw.USB_-_mount",
-            "author": [
-                {
-                    "email": "[email protected]",
-                    "name": "[email protected]"
-                }
-            ],
-            "execution": {
-                "1": {
-                    "action": "Boot system \n\n",
-                    "expected_results": ""
-                },
-                "2": {
-                    "action": "Plug USB stick, it should be mount in  /run/media/sd(x) If X-window system does not start and show USB device, then use the terminal to mount it, following the next steps: \na. Locate the usb stick (usually it is on /dev/sdb) \nb. Create a directory with \"mkdir stick\"  (so you will have such a path as: /home/root/stick). \nc. Run the command  \"mount  /dev/sdb  /home/root/stick\" to mount  USB device on it. \n\n",
-                    "expected_results": "USB device should be mounted in  /run/media/sd(x)  \nor in  /home/root/stick  \n\n"
-                },
-                "3": {
-                    "action": "Then you can access USB stick (/home/root/stick) via Terminal or GUI  and try various commands and actions like \"cp\", \"mv\", \"touch\" and \"rm\".  Type \"dmesg\" command and check for recent mounted devices.",
-                    "expected_results": "Basic commands work properly. The system sends a notification in \"dmesg\" command, showing that the USB stick is accessible and the device is mounted ."
-                }
-            },
-            "summary": "USB_-_mount"
-        }
-    },
-    {
-        "test": {
-            "@alias": "bsps-hw.bsps-hw.USB_-_read_files",
-            "author": [
-                {
-                    "email": "[email protected]",
-                    "name": "[email protected]"
-                }
-            ],
-            "execution": {
-                "1": {
-                    "action": "boot system",
-                    "expected_results": "view/copy successfully"
-                },
-                "2": {
-                    "action": "plug usb stick",
-                    "expected_results": ""
-                },
-                "3": {
-                    "action": "view files in usb by file browser",
-                    "expected_results": ""
-                },
-                "4": {
-                    "action": "copy some files from usb to local hardware",
-                    "expected_results": ""
-                }
-            },
-            "summary": "USB_-_read_files"
-        }
-    },
-    {
-        "test": {
-            "@alias": "bsps-hw.bsps-hw.USB_-_umount",
-            "author": [
-                {
-                    "email": "[email protected]",
-                    "name": "[email protected]"
-                }
-            ],
-            "execution": {
-                "1": {
-                    "action": "boot system",
-                    "expected_results": "usb directory in file browser automatically missed"
-                },
-                "2": {
-                    "action": "plug usb stick",
-                    "expected_results": ""
-                },
-                "3": {
-                    "action": "view files in usb by file browser \n4.unplug usb",
-                    "expected_results": ""
-                }
-            },
-            "summary": "USB_-_umount"
-        }
-    },
-    {
-        "test": {
-            "@alias": "bsps-hw.bsps-hw.USB_-_write_files",
-            "author": [
-                {
-                    "email": "[email protected]",
-                    "name": "[email protected]"
-                }
-            ],
-            "execution": {
-                "1": {
-                    "action": "boot system",
-                    "expected_results": "create/copy successfully"
-                },
-                "2": {
-                    "action": "plug usb stick",
-                    "expected_results": ""
-                },
-                "3": {
-                    "action": "create files in usb \n4.copy some files from local hardware to usb",
-                    "expected_results": ""
-                }
-            },
-            "summary": "USB_-_write_files"
-        }
-    },
-    {
-        "test": {
             "@alias": "bsps-hw.bsps-hw.ethernet_static_ip_set_in_connman",
             "author": [
                 {
@@ -583,118 +475,6 @@
     },
     {
         "test": {
-            "@alias": "bsps-hw.bsps-hw.MicroSD_-__mount",
-            "author": [
-                {
-                    "email": "[email protected]",
-                    "name": "[email protected]"
-                }
-            ],
-            "execution": {
-                "1": {
-                    "action": "boot system",
-                    "expected_results": "system notify that MicroSDis accessible"
-                },
-                "2": {
-                    "action": "plug MicroSD card",
-                    "expected_results": ""
-                }
-            },
-            "summary": "MicroSD_-__mount"
-        }
-    },
-    {
-        "test": {
-            "@alias": "bsps-hw.bsps-hw.MicroSD_-__read_files",
-            "author": [
-                {
-                    "email": "[email protected]",
-                    "name": "[email protected]"
-                }
-            ],
-            "execution": {
-                "1": {
-                    "action": "boot system",
-                    "expected_results": "view/copy successfully"
-                },
-                "2": {
-                    "action": "plug MicroSD card",
-                    "expected_results": ""
-                },
-                "3": {
-                    "action": "view files inMicroSD by file browser",
-                    "expected_results": ""
-                },
-                "4": {
-                    "action": "copy some files fromMicroSD to local hardware",
-                    "expected_results": ""
-                }
-            },
-            "summary": "MicroSD_-__read_files"
-        }
-    },
-    {
-        "test": {
-            "@alias": "bsps-hw.bsps-hw.MicroSD_-__umount",
-            "author": [
-                {
-                    "email": "[email protected]",
-                    "name": "[email protected]"
-                }
-            ],
-            "execution": {
-                "1": {
-                    "action": "boot system",
-                    "expected_results": "MicroSD in file browser automatically missed . "
-                },
-                "2": {
-                    "action": "plug MicroSD card",
-                    "expected_results": ""
-                },
-                "3": {
-                    "action": "view files in MicroSDby file browser",
-                    "expected_results": ""
-                },
-                "4": {
-                    "action": "unplug MicroSD",
-                    "expected_results": ""
-                }				
-            },
-            "summary": "MicroSD_-__umount"
-        }
-    },
-    {
-        "test": {
-            "@alias": "bsps-hw.bsps-hw.MicroSD_-__write_files",
-            "author": [
-                {
-                    "email": "[email protected]",
-                    "name": "[email protected]"
-                }
-            ],
-            "execution": {
-                "1": {
-                    "action": "boot system",
-                    "expected_results": "create/copy successfully"
-                },
-                "2": {
-                    "action": "plug MicroSD card",
-                    "expected_results": ""
-                },
-                "3": {
-                    "action": "create files in MicroSD",
-                    "expected_results": ""
-                },
-                "4": {
-                    "action": "copy some files from local hardware to MicroSD",
-                    "expected_results": ""
-                }
-            },
-            "summary": "MicroSD_-__write_files"
-        }
-    },
-    {
-        "test": {
             "@alias": "bsps-hw.bsps-hw.video_-_libva_check_(ogg_video_play)",
             "author": [
                 {
Signature · 1 line
2.7.4
#6 lore raw

PATCH 7/7 manual/bsd-hw: remove bash tests

In reply to Armin Kuster

done as part of IncompatibleLicensePerImageTests

---
 meta/lib/oeqa/manual/bsp-hw.json | 18 ------------------
 1 file changed, 18 deletions(-)
meta/lib/oeqa/manual/bsp-hw.json +0 -18
index dd3504f..5c5b9b5 100644
@@ -457,24 +457,6 @@
     },
     {
         "test": {
-            "@alias": "bsps-hw.bsps-hw.check_bash_in_image",
-            "author": [
-                {
-                    "email": "[email protected]",
-                    "name": "[email protected]"
-                }
-            ],
-            "execution": {
-                "1": {
-                    "action": "After system is up, check if bash command exists with command \"which bash\"",
-                    "expected_results": "bash command should exist in image giving something as below  \"/bin/bash\""
-                }
-            },
-            "summary": "check_bash_in_image"
-        }
-    },
-    {
-        "test": {
             "@alias": "bsps-hw.bsps-hw.video_-_libva_check_(ogg_video_play)",
             "author": [
                 {
Signature · 1 line
2.7.4
#7 lore raw

✗ patchtest: failure for "/oeqa/manual/compliance-test: ..." and 6 more

In reply to Armin Kuster

== Series Details ==

Series: "/oeqa/manual/compliance-test: ..." and 6 more
Revision: 1
URL : https://patchwork.openembedded.org/series/21180/
State : failure

== Summary ==

Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:

* Issue Series does not apply on top of target branch [test_series_merge_on_head]
Suggested fix Rebase your series on top of targeted branch
Targeted branch master (currently at dfb3b56641)

If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).

---

Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe