Skip to contentSkip to conversation

rteval: Only process warnings if dmidecode_loaded is True

PATCH In review John Kacur 2 participants 5 messages Sep 9, 2021 – Sep 13, 2021

lore
Share & tools
lore

John Kacur

author
#0 lore raw

PATCH 1/3 rteval: Only process warnings if dmidecode_loaded is True

If python-dmidecode is not available, then you can't test hasattr or you will get a NameError. Simply return instead.

---
 rteval/sysinfo/dmi.py | 3 +++
 1 file changed, 3 insertions(+)
rteval/sysinfo/dmi.py +3 -0
index bd397360096a..80cf3c723b36 100644
@@ -42,6 +42,9 @@ except ModuleNotFoundError:
 def ProcessWarnings():
     """ Process Warnings from dmidecode """
 
+    if not dmidecode_loaded:
+        return
+
     if not hasattr(dmidecode, 'get_warnings'):
         return
 
Signature · 1 line
2.31.1
author
#1 lore raw

PATCH 2/3 rteval: Construct a 'model name' on architectures that don't have one

From: root <[email protected]>

This is based on an idea from Punit Agrawal <[email protected]>

On architectures that lack 'model name' in /proc/cpuinfo create 'model name' Unknown when creating the per core dictionaries in cpuinfo

For arm, we can construct the 'model name' from the 'CPU implementer'
'CPU architecture'
'CPU variant'
'CPU part'
'CPU revision'

---
 rteval/misc.py | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
rteval/misc.py +21 -0
index 0dd361ff19fd..016fc7c4b494 100644
@@ -77,6 +77,27 @@ def cpuinfo():
             info[core] = {}
             continue
         info[core][key] = val
+
+    for (core, dict) in info.items():
+        if not 'model name' in dict:
+            # On Arm CPU implementer is present
+            # Construct the model_name from the following fields
+            if 'CPU implementer' in dict:
+                model_name = [dict.get('CPU implementer')]
+                model_name.append(dict.get('CPU architecture'))
+                model_name.append(dict.get('CPU variant'))
+                model_name.append(dict.get('CPU part'))
+                model_name.append(dict.get('CPU revision', ''))
+
+                # If a list item is None, remove it
+                model_name = [name for name in model_name if name]
+
+                # Convert the model_name list into a string
+                model_name = " ".join(model_name)
+                info[core]['model name'] = model_name
+            else:
+                info[core]['model name'] = 'Unknown'
+
     return info
 
 if __name__ == "__main__":
Signature · 1 line
2.31.1
author
#2 lore raw

PATCH 3/3 rteval: systopology.py: Add support for systems that don't have Numa

In reply to John Kacur

From: Punit Agrawal <[email protected]>

Certain systems such as Arm v7 do not have support for Numa nodes, i.e., "/sys/devices/system/node*" does not exist. Instead of erroring out in this situation, it would be better if rteval could use alternate sources to get the system topology and memory information.

Introduce the notion of a fake Numa node (as a class) which is used when no numa nodes are found on the system. Other than the constructor, it provides the same interface as the existing NumaNode class so existing users should work without any changes.

- Renamed Fake to Sim for simulated

---
 rteval/systopology.py | 37 +++++++++++++++++++++++++++++++------
 1 file changed, 31 insertions(+), 6 deletions(-)
rteval/systopology.py +31 -6
index c61ec1a58514..3c996048f8c1 100644
@@ -191,6 +191,30 @@ class NumaNode:
         """ return list of cpus for this node """
         return self.cpus.getcpulist()
 
+class SimNumaNode(NumaNode):
+    """class representing a simulated NUMA node.
+    For systems which don't have NUMA enabled (no
+    /sys/devices/system/node) such as Arm v7
+    """
+
+    cpupath = '/sys/devices/system/cpu'
+    mempath = '/proc/meminfo'
+
+    def __init__(self):
+        self.nodeid = 0
+        self.cpus = CpuList(sysread(SimNumaNode.cpupath, "possible"))
+        self.getmeminfo()
+
+    def getmeminfo(self):
+        self.meminfo = {}
+        for l in open(SimNumaNode.mempath, "r"):
+            elements = l.split()
+            key = elements[0][0:-1]
+            val = int(elements[1])
+            if len(elements) == 3 and elements[2] == "kB":
+                val *= 1024
+            self.meminfo[key] = val
+
 #
 # Class to abstract the system topology of numa nodes and cpus
 #
@@ -238,12 +262,13 @@ class SysTopology:
 
     def getinfo(self):
         nodes = glob.glob(os.path.join(SysTopology.nodepath, 'node[0-9]*'))
-        if not nodes:
-            raise RuntimeError("No valid nodes found in %s!" % SysTopology.nodepath)
-        nodes.sort()
-        for n in nodes:
-            node = int(os.path.basename(n)[4:])
-            self.nodes[node] = NumaNode(n)
+        if nodes:
+            nodes.sort()
+            for n in nodes:
+                node = int(os.path.basename(n)[4:])
+                self.nodes[node] = NumaNode(n)
+        else:
+            self.nodes[0] = SimNumaNode()
 
     def getnodes(self):
         return list(self.nodes.keys())
Signature · 1 line
2.31.1
#3 lore raw

Re: [PATCH 3/3] rteval: systopology.py: Add support for systems that don't have Numa

Quoted 64 lines · John Kacur

From: Punit Agrawal <[email protected]>

Certain systems such as Arm v7 do not have support for Numa nodes, i.e., "/sys/devices/system/node*" does not exist. Instead of erroring out in this situation, it would be better if rteval could use alternate sources to get the system topology and memory information.

Introduce the notion of a fake Numa node (as a class) which is used when no numa nodes are found on the system. Other than the constructor, it provides the same interface as the existing NumaNode class so existing users should work without any changes.

- Renamed Fake to Sim for simulated

---
 rteval/systopology.py | 37 +++++++++++++++++++++++++++++++------
 1 file changed, 31 insertions(+), 6 deletions(-)
rteval/systopology.py +24 -6
index c61ec1a58514..3c996048f8c1 100644
@@ -191,6 +191,30 @@ class NumaNode:
         """ return list of cpus for this node """
         return self.cpus.getcpulist()
 
+class SimNumaNode(NumaNode):
+    """class representing a simulated NUMA node.
+    For systems which don't have NUMA enabled (no
+    /sys/devices/system/node) such as Arm v7
+    """
+
+    cpupath = '/sys/devices/system/cpu'
+    mempath = '/proc/meminfo'
+
+    def __init__(self):
+        self.nodeid = 0
+        self.cpus = CpuList(sysread(SimNumaNode.cpupath, "possible"))
+        self.getmeminfo()
+
+    def getmeminfo(self):
+        self.meminfo = {}
+        for l in open(SimNumaNode.mempath, "r"):
+            elements = l.split()
+            key = elements[0][0:-1]
+            val = int(elements[1])
+            if len(elements) == 3 and elements[2] == "kB":
+                val *= 1024
+            self.meminfo[key] = val
+
 #
 # Class to abstract the system topology of numa nodes and cpus
 #
@@ -238,12 +262,13 @@ class SysTopology:
 
     def getinfo(self):
         nodes = glob.glob(os.path.join(SysTopology.nodepath, 'node[0-9]*'))
-        if not nodes:
-            raise RuntimeError("No valid nodes found in %s!" % SysTopology.nodepath)
-        nodes.sort()
-        for n in nodes:
-            node = int(os.path.basename(n)[4:])
-            self.nodes[node] = NumaNode(n)

rteval/systopology.py class SysTopology: · line 274

Quoted 10 lines
rteval/systopology.py +7 -0
+        if nodes:
+            nodes.sort()
+            for n in nodes:
+                node = int(os.path.basename(n)[4:])
+                self.nodes[node] = NumaNode(n)
+        else:
+            self.nodes[0] = SimNumaNode()
 
     def getnodes(self):
         return list(self.nodes.keys())

The name changes look good and passed a test run of rteval on a system with no numa nodes.

Thanks,
Punit

Tested-by on v1 1/3
#4 lore raw
In reply to John Kacur
Quoted 13 lines · John Kacur

If python-dmidecode is not available, then you can't test hasattr or you will get a NameError. Simply return instead.

---
 rteval/sysinfo/dmi.py | 3 +++
 1 file changed, 3 insertions(+)
rteval/sysinfo/dmi.py +0 -0
index bd397360096a..80cf3c723b36 100644

rteval/sysinfo/dmi.py except ModuleNotFoundError: · line 49

Quoted 8 lines
rteval/sysinfo/dmi.py +3 -0
@@ -42,6 +42,9 @@ except ModuleNotFoundError:
 def ProcessWarnings():
     """ Process Warnings from dmidecode """
 
+    if not dmidecode_loaded:
+        return
+
     if not hasattr(dmidecode, 'get_warnings'):
         return

Tested after purging python3-dmidecode package on Debian.

Thanks,
Punit