Skip to contentSkip to conversation

add defconfig and python unittest to

PATCH New Victor Huesca 2 participants 5 messages Aug 4, 2019

lore
Share & tools
lore
#0 lore raw

cover add defconfig and python unittest to

This patch extends support of getdeveloperlib to defconfigs and python runtime tests.

This allows to follow more files from the DEVELOPERS file.

Victor Huesca (3):
utils/getdeveloperlib: add some debug informations utils/getdeveloperlib.py: add runtime test utils/getdeveloperlib.py: add defconfigs

 utils/getdeveloperlib.py | 83 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)
Signature · 1 line
2.21.0
#1 lore raw

PATCH 1/3 utils/getdeveloperlib: add some debug informations

---
 utils/getdeveloperlib.py | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
utils/getdeveloperlib.py +16 -0
index 35c618f791..81eae7ee9b 100644
@@ -97,6 +97,22 @@ class Developer:
                 return True
         return False
 
+    def __repr__(self):
+        name = '\'' + self.name.split(' <')[0][:20] + '\''
+        things = []
+        if len(self.files):
+            things.append('{} files'.format(len(self.files)))
+        if len(self.packages):
+            things.append('{} pkgs'.format(len(self.packages)))
+        if len(self.architectures):
+            things.append('{} archs'.format(len(self.architectures)))
+        if len(self.infras):
+            things.append('{} infras'.format(len(self.infras)))
+        if things:
+            return 'Developer <{} ({})>'.format(name, ', '.join(things))
+        else:
+            return 'Developer <' + name + '>'
+
 
 def parse_developer_packages(fnames):
     """Given a list of file patterns, travel through the Buildroot source
Signature · 1 line
2.21.0
#2 lore raw

PATCH 2/3 utils/getdeveloperlib.py: add runtime test

In reply to Victor Huesca

This patch provides support to track python runtime tests.

The implementation rely on the unit-test module to list all test cases and does some manual parsing of these test-case objects to get the actual list of test-cases per test-suite

Directories are supported as file name are returns all test-cases recursively.

A global variable is used to compute the list of unittest only once.

---
 utils/getdeveloperlib.py | 56 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)
utils/getdeveloperlib.py +56 -0
index 81eae7ee9b..8c8343a137 100644
@@ -4,6 +4,7 @@ import re
 import glob
 import subprocess
 import sys
+import unittest
 
 #
 # Patch parsing functions
@@ -78,6 +79,35 @@ def analyze_patches(patches):
     return (allfiles, allinfras)
 
 
+# 
+# Unit-test parsing functions
+# 
+
+def get_all_test_cases(suite):
+    """Generate all test-cases from a given test-suite.
+    :return: (test.module, test.name)"""
+    if issubclass(type(suite), unittest.TestSuite):
+        for test in suite:
+            for res in get_all_test_cases(test):
+                yield res
+    else:
+        yield (suite.__module__, suite.__class__.__name__)
+
+
+def list_unittests(path):
+    """Use the unittest module to retreive all test cases from a given
+    directory"""
+    loader = unittest.TestLoader()
+    suite = loader.discover(path)
+    tests = {}
+    for module, test in get_all_test_cases(suite):
+        module_path = os.path.join(path, *module.split('.'))
+        tests.setdefault(module_path, []).append('%s.%s' % (module, test))
+    return tests
+
+
+unittests = {}
+
 #
 # DEVELOPERS file parsing functions
 #
@@ -89,6 +119,7 @@ class Developer:
         self.packages = parse_developer_packages(files)
         self.architectures = parse_developer_architectures(files)
         self.infras = parse_developer_infras(files)
+        self.runtime_tests = parse_developer_runtime_tests(files)
 
     def hasfile(self, f):
         f = os.path.abspath(f)
@@ -108,6 +139,8 @@ class Developer:
             things.append('{} archs'.format(len(self.architectures)))
         if len(self.infras):
             things.append('{} infras'.format(len(self.infras)))
+        if len(self.runtime_tests):
+            things.append('{} runtime'.format(len(self.runtime_tests)))
         if things:
             return 'Developer <{} ({})>'.format(name, ', '.join(things))
         else:
@@ -170,12 +203,35 @@ def parse_developer_infras(fnames):
     return infras
 
 
+def parse_developer_runtime_tests(fnames):
+    """Given a list of file names, returns the runtime tests
+    corresponding to the file."""
+    all_files = []
+    # List all files recursively
+    for fname in fnames:
+        if os.path.isdir(fname):
+            for root, _dirs, files in os.walk(fname):
+                all_files += [os.path.join(root, f) for f in files]
+        else:
+            all_files.append(fname)
+
+    # Get all runtime tests
+    runtimes = set()
+    for f in all_files:
+        name = os.path.splitext(f)[0]
+        if name in unittests:
+            runtimes |= set(unittests[name])
+    return runtimes
+
+
 def parse_developers(basepath=None):
     """Parse the DEVELOPERS file and return a list of Developer objects."""
     developers = []
     linen = 0
     if basepath is None:
         basepath = os.getcwd()
+    global unittests
+    unittests = list_unittests(os.path.join(basepath, 'support/testing'))
     with open(os.path.join(basepath, "DEVELOPERS"), "r") as f:
         files = []
         name = None
Signature · 1 line
2.21.0
#3 lore raw

PATCH 3/3 utils/getdeveloperlib.py: add defconfigs

In reply to Victor Huesca

This patch allows to follow defconfig files.

---
 utils/getdeveloperlib.py | 11 +++++++++++
 1 file changed, 11 insertions(+)
utils/getdeveloperlib.py +11 -0
index 8c8343a137..80c46b9fa8 100644
@@ -120,6 +120,7 @@ class Developer:
         self.architectures = parse_developer_architectures(files)
         self.infras = parse_developer_infras(files)
         self.runtime_tests = parse_developer_runtime_tests(files)
+        self.defconfigs = parse_developer_defconfigs(files)
 
     def hasfile(self, f):
         f = os.path.abspath(f)
@@ -141,6 +142,8 @@ class Developer:
             things.append('{} infras'.format(len(self.infras)))
         if len(self.runtime_tests):
             things.append('{} runtime'.format(len(self.runtime_tests)))
+        if len(self.defconfigs):
+            things.append('{} defconfigs'.format(len(self.defconfigs)))
         if things:
             return 'Developer <{} ({})>'.format(name, ', '.join(things))
         else:
@@ -203,6 +206,14 @@ def parse_developer_infras(fnames):
     return infras
 
 
+def parse_developer_defconfigs(fnames):
+    """Given a list of file names, returns the config names
+    corresponding to defconfigs."""
+    return {os.path.basename(fname[:-10])
+            for fname in fnames
+            if fname.endswith('_defconfig')}
+
+
 def parse_developer_runtime_tests(fnames):
     """Given a list of file names, returns the runtime tests
     corresponding to the file."""
Signature · 1 line
2.21.0
#4 lore raw

cover add defconfig and python unittest to

In reply to Victor Huesca

Hello,

Quoted 4 lines · On Sun, 4 Aug 2019 16:21:40 +0200 Victor Huesca <[email protected]>…

Victor Huesca (3):
utils/getdeveloperlib: add some debug informations utils/getdeveloperlib.py: add runtime test utils/getdeveloperlib.py: add defconfigs

Thanks, I've applied the 3 patches, with some minor adjustements, especially to the commit logs.

Looking forward to seeing the remaining changes in daily-mail to take advantage of those improvements!

Thanks,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com