Enable rteval on Arm based systems

PATCH In review Punit Agrawal 2 participants 19 messages Sep 1, 2021 – Sep 16, 2021

lore

Share & tools

lore

Punit Agrawal

author

2021-09-01 08:08 UTC #0 lore raw

cover Enable rteval on Arm based systems

Hi,

The following small set of patches were required to enable rteval on Arm based systems. Of these, the first two patches have been posted before[0] and have been updated to fix some issues noticed since then.

The remaining three patches are new and fix issues encountered while running rteval on a 64bit Arm platform.

All feedback welcome.

Thanks,
Punit

[0] https://lore.kernel.org/linux-rt-users/[email protected]/

Punit Agrawal (5):
rteval: systopology.py: Add support for systems that don't have Numa
rteval: cyclictest.py: Update logic to get core description
rteval: kernel.py: Add support for kthreads running with deadline
policy
rteval: hackbench.py: Enable running on a system with low memory
rteval: kcompile.py: Gracefully handle missing affinity information

 rteval/misc.py                           | 23 +++++++++++++-
 rteval/modules/loads/hackbench.py        |  3 +-
 rteval/modules/loads/kcompile.py         |  6 +++-
 rteval/modules/measurement/cyclictest.py |  8 +++--
 rteval/sysinfo/kernel.py                 |  2 +-
 rteval/systopology.py                    | 38 ++++++++++++++++++++----
 6 files changed, 66 insertions(+), 14 deletions(-)

Signature · 1 line

2.32.0

Punit Agrawal

author

2021-09-01 08:08 UTC #1 lore raw

PATCH 1/5 rteval: systopology.py: Add support for systems that don't have Numa

From: Punit Agrawal <[email protected]>

Certain systems such as Arm v7 do not have support for Numa nodes, i.e., "/sys/devices/system/node*" does not exist. Instead of erroring out in this situation, it would be better if rteval could use alternate sources to get the system topology and memory information.

Introduce the notion of a fake Numa node (as a class) which is used when no numa nodes are found on the system. Other than the constructor, it provides the same interface as the existing NumaNode class so existing users should work without any changes.

Signed-off-by Punit Agrawal <[email protected]>

---
 rteval/systopology.py | 38 ++++++++++++++++++++++++++++++++------
 1 file changed, 32 insertions(+), 6 deletions(-)

rteval/systopology.py +32 -6

index c61ec1a58514..7ce9a8c4f707 100644
@@ -191,6 +191,31 @@ class NumaNode:
         """ return list of cpus for this node """
         return self.cpus.getcpulist()
 
+class FakeNumaNode(NumaNode):
+    """class representing a fake NUMA node. The fake NUMA node is used on
+    systems which don't have NUMA enabled (no
+    /sys/devices/system/node) such as Arm v7
+
+    """
+
+    cpupath = '/sys/devices/system/cpu'
+    mempath = '/proc/meminfo'
+
+    def __init__(self):
+        self.nodeid = 0
+        self.cpus = CpuList(sysread(FakeNumaNode.cpupath, "possible"))
+        self.getmeminfo()
+
+    def getmeminfo(self):
+        self.meminfo = {}
+        for l in open(FakeNumaNode.mempath, "r"):
+            elements = l.split()
+            key = elements[0][0:-1]
+            val = int(elements[1])
+            if len(elements) == 3 and elements[2] == "kB":
+                val *= 1024
+            self.meminfo[key] = val
+
 #
 # Class to abstract the system topology of numa nodes and cpus
 #

@@ -238,12 +263,13 @@ class SysTopology:
 
     def getinfo(self):
         nodes = glob.glob(os.path.join(SysTopology.nodepath, 'node[0-9]*'))
-        if not nodes:
-            raise RuntimeError("No valid nodes found in %s!" % SysTopology.nodepath)
-        nodes.sort()
-        for n in nodes:
-            node = int(os.path.basename(n)[4:])
-            self.nodes[node] = NumaNode(n)
+        if nodes:
+            nodes.sort()
+            for n in nodes:
+                node = int(os.path.basename(n)[4:])
+                self.nodes[node] = NumaNode(n)
+        else:
+            self.nodes[0] = FakeNumaNode()
 
     def getnodes(self):
         return list(self.nodes.keys())

Signature · 1 line

2.32.0

John Kacur

2021-09-09 12:40 UTC #2 lore raw

Re: [PATCH 1/5] rteval: systopology.py: Add support for systems that don't have Numa

Quoted 77 lines · Punit Agrawal

From: Punit Agrawal <[email protected]>

Signed-off-by Punit Agrawal <[email protected]>

---
 rteval/systopology.py | 38 ++++++++++++++++++++++++++++++++------
 1 file changed, 32 insertions(+), 6 deletions(-)

rteval/systopology.py +32 -6

index c61ec1a58514..7ce9a8c4f707 100644
@@ -191,6 +191,31 @@ class NumaNode:
         """ return list of cpus for this node """
         return self.cpus.getcpulist()
 
+class FakeNumaNode(NumaNode):
+    """class representing a fake NUMA node. The fake NUMA node is used on
+    systems which don't have NUMA enabled (no
+    /sys/devices/system/node) such as Arm v7
+
+    """
+
+    cpupath = '/sys/devices/system/cpu'
+    mempath = '/proc/meminfo'
+
+    def __init__(self):
+        self.nodeid = 0
+        self.cpus = CpuList(sysread(FakeNumaNode.cpupath, "possible"))
+        self.getmeminfo()
+
+    def getmeminfo(self):
+        self.meminfo = {}
+        for l in open(FakeNumaNode.mempath, "r"):
+            elements = l.split()
+            key = elements[0][0:-1]
+            val = int(elements[1])
+            if len(elements) == 3 and elements[2] == "kB":
+                val *= 1024
+            self.meminfo[key] = val
+
 #
 # Class to abstract the system topology of numa nodes and cpus
 #

@@ -238,12 +263,13 @@ class SysTopology:
 
     def getinfo(self):
         nodes = glob.glob(os.path.join(SysTopology.nodepath, 'node[0-9]*'))
-        if not nodes:
-            raise RuntimeError("No valid nodes found in %s!" % SysTopology.nodepath)
-        nodes.sort()
-        for n in nodes:
-            node = int(os.path.basename(n)[4:])
-            self.nodes[node] = NumaNode(n)
+        if nodes:
+            nodes.sort()
+            for n in nodes:
+                node = int(os.path.basename(n)[4:])
+                self.nodes[node] = NumaNode(n)
+        else:
+            self.nodes[0] = FakeNumaNode()
 
     def getnodes(self):
         return list(self.nodes.keys())

Signature · 1 line

2.32.0

This is quite clever. I am just going to rename Fake to Sim as short for simulated, but other than that.

Signed-off-by John Kacur <[email protected]>

Punit Agrawal

author

2021-09-13 02:28 UTC #3 lore raw

Re: [PATCH 1/5] rteval: systopology.py: Add support for systems that don't have Numa

Hi John,

Quoted 83 lines · John Kacur

Quoted 77 lines · Punit Agrawal

From: Punit Agrawal <[email protected]>

Signed-off-by Punit Agrawal <[email protected]>

---
 rteval/systopology.py | 38 ++++++++++++++++++++++++++++++++------
 1 file changed, 32 insertions(+), 6 deletions(-)

rteval/systopology.py +32 -6

index c61ec1a58514..7ce9a8c4f707 100644
@@ -191,6 +191,31 @@ class NumaNode:
         """ return list of cpus for this node """
         return self.cpus.getcpulist()
 
+class FakeNumaNode(NumaNode):
+    """class representing a fake NUMA node. The fake NUMA node is used on
+    systems which don't have NUMA enabled (no
+    /sys/devices/system/node) such as Arm v7
+
+    """
+
+    cpupath = '/sys/devices/system/cpu'
+    mempath = '/proc/meminfo'
+
+    def __init__(self):
+        self.nodeid = 0
+        self.cpus = CpuList(sysread(FakeNumaNode.cpupath, "possible"))
+        self.getmeminfo()
+
+    def getmeminfo(self):
+        self.meminfo = {}
+        for l in open(FakeNumaNode.mempath, "r"):
+            elements = l.split()
+            key = elements[0][0:-1]
+            val = int(elements[1])
+            if len(elements) == 3 and elements[2] == "kB":
+                val *= 1024
+            self.meminfo[key] = val
+
 #
 # Class to abstract the system topology of numa nodes and cpus
 #

@@ -238,12 +263,13 @@ class SysTopology:
 
     def getinfo(self):
         nodes = glob.glob(os.path.join(SysTopology.nodepath, 'node[0-9]*'))
-        if not nodes:
-            raise RuntimeError("No valid nodes found in %s!" % SysTopology.nodepath)
-        nodes.sort()
-        for n in nodes:
-            node = int(os.path.basename(n)[4:])
-            self.nodes[node] = NumaNode(n)
+        if nodes:
+            nodes.sort()
+            for n in nodes:
+                node = int(os.path.basename(n)[4:])
+                self.nodes[node] = NumaNode(n)
+        else:
+            self.nodes[0] = FakeNumaNode()
 
     def getnodes(self):
         return list(self.nodes.keys())

Signature · 1 line

2.32.0

This is quite clever. I am just going to rename Fake to Sim as short for simulated, but other than that.

Quoted 1 line

Signed-off-by John Kacur <[email protected]>

Thanks for taking a look.

I'll take a look at the updates you sent and respond there as appropriate.

Regards,
Punit

Punit Agrawal

author

2021-09-01 08:08 UTC #4 lore raw

PATCH 2/5 rteval: cyclictest.py: Update logic to get core description

In reply to Punit Agrawal

From: Punit Agrawal <[email protected]>

Certain architectures such as arm and arm64 don't have a "model name" in /proc/cpuinfo but provide other identifying information such as implementer, architecture, variant, part, revision, etc..

Add a function 'get_cpumodel()' that takes the per-core dictionary constructed from /proc/cpuinfo and uses the available data to construct the CPU description. Update the users of "model name" to use the newly added function.

Signed-off-by Punit Agrawal <[email protected]>

---
 rteval/misc.py                           | 23 ++++++++++++++++++++++-
 rteval/modules/measurement/cyclictest.py |  8 +++++---
 2 files changed, 27 insertions(+), 4 deletions(-)

rteval/misc.py +22 -1

index 0dd361ff19fd..537041b53d61 100644
@@ -79,6 +79,27 @@ def cpuinfo():
         info[core][key] = val
     return info
 
+# Pass the per-core dictionary that is part of the dictionary returned
+# by cpuinfo()
+def get_cpumodel(core_info):
+    desc = core_info.get('model name', '')
+    if not desc:
+        # On Arm (both 32 and 64 bit), 'CPU Implementer' is always
+        # present. Return 'unknown' otherwise
+        if 'CPU implementer' not in core_info:
+            desc = 'unknown'
+        else:
+            implementor = core_info.get('CPU implementer', '')
+            architecture = core_info.get('CPU architecture', '')
+            variant = core_info.get('CPU variant', '')
+            part = core_info.get('CPU part', '')
+            revision = core_info.get('CPU revision', '')
+
+            desc = 'Implementor: %s Architecture: %s Variant: %s Part: %s Revision: %s' \
+                % (implementor, architecture, variant, part, revision)
+
+    return desc
+
 if __name__ == "__main__":
 
     info = cpuinfo()

@@ -86,4 +107,4 @@ if __name__ == "__main__":
     for i in idx:
         print("%s: %s" % (i, info[i]))
 
-    print("0: %s" % (info['0']['model name']))
+    print("0: %s" % (get_cpumodel(info['0'])))

rteval/modules/measurement/cyclictest.py +5 -3

index b1755d4f4421..11cb45e711dd 100644
@@ -34,7 +34,7 @@ import math
 import libxml2
 from rteval.Log import Log
 from rteval.modules import rtevalModulePrototype
-from rteval.misc import expand_cpulist, online_cpus, cpuinfo
+from rteval.misc import expand_cpulist, online_cpus, cpuinfo, get_cpumodel
 
 class RunData:
     '''class to keep instance data from a cyclictest run'''

@@ -226,13 +226,15 @@ class Cyclictest(rtevalModulePrototype):
         for core in self.__cpus:
             self.__cyclicdata[core] = RunData(core, 'core', self.__priority,
                                               logfnc=self._log)
-            self.__cyclicdata[core].description = info[core]['model name']
+            self.__cyclicdata[core].description = get_cpumodel(info[core])
+            if self.__cyclicdata[core].description == 'unknown':
+                self._log(Log.INFO, "Unknown model for core %d" % core)
 
         # Create a RunData object for the overall system
         self.__cyclicdata['system'] = RunData('system',
                                               'system', self.__priority,
                                               logfnc=self._log)
-        self.__cyclicdata['system'].description = ("(%d cores) " % self.__numcores) + info['0']['model name']
+        self.__cyclicdata['system'].description = ("(%d cores) " % self.__numcores) + get_cpumodel(info['0'])
 
         if self.__sparse:
             self._log(Log.DEBUG, "system using %d cpu cores" % self.__numcores)

Signature · 1 line

2.32.0

John Kacur

2021-09-09 12:47 UTC #5 lore raw

Re: [PATCH 2/5] rteval: cyclictest.py: Update logic to get core description

Quoted 90 lines · Punit Agrawal

From: Punit Agrawal <[email protected]>

Certain architectures such as arm and arm64 don't have a "model name" in /proc/cpuinfo but provide other identifying information such as implementer, architecture, variant, part, revision, etc..

Signed-off-by Punit Agrawal <[email protected]>

---
 rteval/misc.py                           | 23 ++++++++++++++++++++++-
 rteval/modules/measurement/cyclictest.py |  8 +++++---
 2 files changed, 27 insertions(+), 4 deletions(-)

rteval/misc.py +22 -1

index 0dd361ff19fd..537041b53d61 100644
@@ -79,6 +79,27 @@ def cpuinfo():
         info[core][key] = val
     return info
 
+# Pass the per-core dictionary that is part of the dictionary returned
+# by cpuinfo()
+def get_cpumodel(core_info):
+    desc = core_info.get('model name', '')
+    if not desc:
+        # On Arm (both 32 and 64 bit), 'CPU Implementer' is always
+        # present. Return 'unknown' otherwise
+        if 'CPU implementer' not in core_info:
+            desc = 'unknown'
+        else:
+            implementor = core_info.get('CPU implementer', '')
+            architecture = core_info.get('CPU architecture', '')
+            variant = core_info.get('CPU variant', '')
+            part = core_info.get('CPU part', '')
+            revision = core_info.get('CPU revision', '')
+
+            desc = 'Implementor: %s Architecture: %s Variant: %s Part: %s Revision: %s' \
+                % (implementor, architecture, variant, part, revision)
+
+    return desc
+
 if __name__ == "__main__":
 
     info = cpuinfo()

@@ -86,4 +107,4 @@ if __name__ == "__main__":
     for i in idx:
         print("%s: %s" % (i, info[i]))
 
-    print("0: %s" % (info['0']['model name']))
+    print("0: %s" % (get_cpumodel(info['0'])))

rteval/modules/measurement/cyclictest.py +5 -3

index b1755d4f4421..11cb45e711dd 100644
@@ -34,7 +34,7 @@ import math
 import libxml2
 from rteval.Log import Log
 from rteval.modules import rtevalModulePrototype
-from rteval.misc import expand_cpulist, online_cpus, cpuinfo
+from rteval.misc import expand_cpulist, online_cpus, cpuinfo, get_cpumodel
 
 class RunData:
     '''class to keep instance data from a cyclictest run'''

@@ -226,13 +226,15 @@ class Cyclictest(rtevalModulePrototype):
         for core in self.__cpus:
             self.__cyclicdata[core] = RunData(core, 'core', self.__priority,
                                               logfnc=self._log)
-            self.__cyclicdata[core].description = info[core]['model name']
+            self.__cyclicdata[core].description = get_cpumodel(info[core])
+            if self.__cyclicdata[core].description == 'unknown':
+                self._log(Log.INFO, "Unknown model for core %d" % core)
 
         # Create a RunData object for the overall system
         self.__cyclicdata['system'] = RunData('system',
                                               'system', self.__priority,
                                               logfnc=self._log)
-        self.__cyclicdata['system'].description = ("(%d cores) " % self.__numcores) + info['0']['model name']
+        self.__cyclicdata['system'].description = ("(%d cores) " % self.__numcores) + get_cpumodel(info['0'])
 
         if self.__sparse:
             self._log(Log.DEBUG, "system using %d cpu cores" % self.__numcores)

Signature · 1 line

2.32.0

I like the idea of constructing the 'model name' from other fields in /proc/cpuinfo available for arm.

There are a number of problems with the implementation.
This should be written in such away that the local changes in misc.py work seamlessly without changes to the code that calls it. The code should also add the 'model name' to every core.
Finally, a lesser detail, but there is no need when using get to set a default of "" when the default is already None. Just make the code work with None.

I have rewritten this from scratch, attributing the idea to you, I hope that is okay. I will send the patch to the list shortly.

John

Punit Agrawal

author

2021-09-01 08:08 UTC #6 lore raw

PATCH 3/5 rteval: kernel.py: Add support for kthreads running with deadline policy

In reply to Punit Agrawal

From: Punit Agrawal <[email protected]>

When running rteval on a system with kthreads running with "deadline" policy, an exception is encountered when parsing the output of "ps".

[DEBUG] cmd: /usr/bin/ps -eocommand,pid,policy,rtprio,comm Traceback (most recent call last):
...
File "...rteval/rteval/sysinfo/kernel.py", line 60, in kernel_get_kthreads
ret_kthreads[v[0]] = {'policy' : policies[bytes.decode(v[1])],
KeyError: 'DLN'

The kernel uses deadline policy for "schedutil" cpufreq governor threads. Fix the crash in rteval by adding support for "deadline" to the list of policies.

Signed-off-by Punit Agrawal <[email protected]>

---
 rteval/sysinfo/kernel.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

rteval/sysinfo/kernel.py +1 -1

index 97ad9402b13e..f2e9d72ac2ef 100644
@@ -44,7 +44,7 @@ class KernelInfo:
 
 
     def kernel_get_kthreads(self):
-        policies = {'FF':'fifo', 'RR':'rrobin', 'TS':'other', '?':'unknown'}
+        policies = {'DLN': 'deadline', 'FF':'fifo', 'RR':'rrobin', 'TS':'other', '?':'unknown'}
         ret_kthreads = {}
         self.__log(Log.DEBUG, "getting kthread status")
         cmd = '%s -eocommand,pid,policy,rtprio,comm' % getcmdpath('ps')

Signature · 1 line

2.32.0

John Kacur

2021-09-09 12:55 UTC #7 lore raw

Re: [PATCH 3/5] rteval: kernel.py: Add support for kthreads running with deadline policy

Quoted 38 lines · Punit Agrawal

From: Punit Agrawal <[email protected]>
When running rteval on a system with kthreads running with "deadline" policy, an exception is encountered when parsing the output of "ps".
[DEBUG] cmd: /usr/bin/ps -eocommand,pid,policy,rtprio,comm Traceback (most recent call last):
...
File "...rteval/rteval/sysinfo/kernel.py", line 60, in kernel_get_kthreads
ret_kthreads[v[0]] = {'policy' : policies[bytes.decode(v[1])],
KeyError: 'DLN'
The kernel uses deadline policy for "schedutil" cpufreq governor threads. Fix the crash in rteval by adding support for "deadline" to the list of policies.
Signed-off-by Punit Agrawal <[email protected]>
---
 rteval/sysinfo/kernel.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
rteval/sysinfo/kernel.py +1 -1
index 97ad9402b13e..f2e9d72ac2ef 100644
@@ -44,7 +44,7 @@ class KernelInfo:
 
 
     def kernel_get_kthreads(self):
-        policies = {'FF':'fifo', 'RR':'rrobin', 'TS':'other', '?':'unknown'}
+        policies = {'DLN': 'deadline', 'FF':'fifo', 'RR':'rrobin', 'TS':'other', '?':'unknown'}
         ret_kthreads = {}
         self.__log(Log.DEBUG, "getting kthread status")
         cmd = '%s -eocommand,pid,policy,rtprio,comm' % getcmdpath('ps')
Signature · 1 line
2.32.0

Signed-off-by John Kacur <[email protected]>

Punit Agrawal

author

2021-09-15 08:45 UTC #8 lore raw

Re: [PATCH 3/5] rteval: kernel.py: Add support for kthreads running with deadline policy

Hi John,

Quoted 30 lines · John Kacur

Quoted 28 lines · Punit Agrawal
From: Punit Agrawal <[email protected]>
When running rteval on a system with kthreads running with "deadline" policy, an exception is encountered when parsing the output of "ps".
[DEBUG] cmd: /usr/bin/ps -eocommand,pid,policy,rtprio,comm Traceback (most recent call last):
...
File "...rteval/rteval/sysinfo/kernel.py", line 60, in kernel_get_kthreads
ret_kthreads[v[0]] = {'policy' : policies[bytes.decode(v[1])],
KeyError: 'DLN'
The kernel uses deadline policy for "schedutil" cpufreq governor threads. Fix the crash in rteval by adding support for "deadline" to the list of policies.
Signed-off-by Punit Agrawal <[email protected]>
---
 rteval/sysinfo/kernel.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
rteval/sysinfo/kernel.py +0 -0
index 97ad9402b13e..f2e9d72ac2ef 100644
@@ -44,7 +44,7 @@ class KernelInfo:
 
 

Quoted 12 lines

Quoted 10 lines

rteval/sysinfo/kernel.py +1 -1

     def kernel_get_kthreads(self):
-        policies = {'FF':'fifo', 'RR':'rrobin', 'TS':'other', '?':'unknown'}
+        policies = {'DLN': 'deadline', 'FF':'fifo', 'RR':'rrobin', 'TS':'other', '?':'unknown'}
         ret_kthreads = {}
         self.__log(Log.DEBUG, "getting kthread status")
         cmd = '%s -eocommand,pid,policy,rtprio,comm' % getcmdpath('ps')

Signature · 1 line

2.32.0

Signed-off-by John Kacur <[email protected]>

It looks like this patch and the other one converting hackbench memory check to a warning was missed when applying the rest of the series - John, could you please pick these as well.

Thanks,
Punit

John Kacur

2021-09-15 12:17 UTC #9 lore raw

Re: [PATCH 3/5] rteval: kernel.py: Add support for kthreads running with deadline policy

Quoted 47 lines · Punit Agrawal

Hi John,
Quoted 30 lines · John Kacur
Quoted 28 lines · Punit Agrawal
From: Punit Agrawal <[email protected]>
When running rteval on a system with kthreads running with "deadline" policy, an exception is encountered when parsing the output of "ps".
[DEBUG] cmd: /usr/bin/ps -eocommand,pid,policy,rtprio,comm Traceback (most recent call last):
...
File "...rteval/rteval/sysinfo/kernel.py", line 60, in kernel_get_kthreads
ret_kthreads[v[0]] = {'policy' : policies[bytes.decode(v[1])],
KeyError: 'DLN'
The kernel uses deadline policy for "schedutil" cpufreq governor threads. Fix the crash in rteval by adding support for "deadline" to the list of policies.
Signed-off-by Punit Agrawal <[email protected]>
---
 rteval/sysinfo/kernel.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
rteval/sysinfo/kernel.py +0 -0
index 97ad9402b13e..f2e9d72ac2ef 100644
@@ -44,7 +44,7 @@ class KernelInfo:
 
 
Quoted 12 lines
Quoted 10 lines
rteval/sysinfo/kernel.py +1 -1
     def kernel_get_kthreads(self):
-        policies = {'FF':'fifo', 'RR':'rrobin', 'TS':'other', '?':'unknown'}
+        policies = {'DLN': 'deadline', 'FF':'fifo', 'RR':'rrobin', 'TS':'other', '?':'unknown'}
         ret_kthreads = {}
         self.__log(Log.DEBUG, "getting kthread status")
         cmd = '%s -eocommand,pid,policy,rtprio,comm' % getcmdpath('ps')
Signature · 1 line
2.32.0
Signed-off-by John Kacur <[email protected]>

Quoted 7 lines

It looks like this patch and the other one converting hackbench memory check to a warning was missed when applying the rest of the series - John, could you please pick these as well.
Thanks,
Punit

I added the patch to support deadline policy, the other one had already been added, I just hadn't pushed upstream yet, but I just did so, so go ahead and pull.

John

Punit Agrawal

author

2021-09-16 11:34 UTC #10 lore raw

Re: [PATCH 3/5] rteval: kernel.py: Add support for kthreads running with deadline policy

Quoted 53 lines · John Kacur

Quoted 51 lines · Punit Agrawal
Hi John,
Quoted 30 lines · John Kacur
Quoted 28 lines · Punit Agrawal
From: Punit Agrawal <[email protected]>
When running rteval on a system with kthreads running with "deadline" policy, an exception is encountered when parsing the output of "ps".
[DEBUG] cmd: /usr/bin/ps -eocommand,pid,policy,rtprio,comm Traceback (most recent call last):
...
File "...rteval/rteval/sysinfo/kernel.py", line 60, in kernel_get_kthreads
ret_kthreads[v[0]] = {'policy' : policies[bytes.decode(v[1])],
KeyError: 'DLN'
The kernel uses deadline policy for "schedutil" cpufreq governor threads. Fix the crash in rteval by adding support for "deadline" to the list of policies.
Signed-off-by Punit Agrawal <[email protected]>
---
 rteval/sysinfo/kernel.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
rteval/sysinfo/kernel.py +0 -0
index 97ad9402b13e..f2e9d72ac2ef 100644
@@ -44,7 +44,7 @@ class KernelInfo:
 
 
Quoted 12 lines
Quoted 10 lines
rteval/sysinfo/kernel.py +1 -1
     def kernel_get_kthreads(self):
-        policies = {'FF':'fifo', 'RR':'rrobin', 'TS':'other', '?':'unknown'}
+        policies = {'DLN': 'deadline', 'FF':'fifo', 'RR':'rrobin', 'TS':'other', '?':'unknown'}
         ret_kthreads = {}
         self.__log(Log.DEBUG, "getting kthread status")
         cmd = '%s -eocommand,pid,policy,rtprio,comm' % getcmdpath('ps')
Signature · 1 line
2.32.0
Signed-off-by John Kacur <[email protected]>
It looks like this patch and the other one converting hackbench memory check to a warning was missed when applying the rest of the series - John, could you please pick these as well.

Quoted 7 lines

Quoted 3 lines
Thanks,
Punit
I added the patch to support deadline policy, the other one had already been added, I just hadn't pushed upstream yet, but I just did so, so go ahead and pull.

Thanks, I see both the patches now.

Punit Agrawal

author

2021-09-01 08:08 UTC #11 lore raw

PATCH 4/5 rteval: hackbench.py: Enable running on a system with low memory

In reply to Punit Agrawal

From: Punit Agrawal <[email protected]>

The hackbench workload refues to run on RockPro64, a hexacore 64bit Arm board with 4GB memory, complaining about insufficient memory per-core.

On further investigation, it turns out that workload is using an arbitrary limit of 0.75 GB/core but will quite happily run on much lower lower memory systems.

Instead of preventing execution, convert the info message to a warning when the memory is lower than expected but continue execution. This should enable the workload to be used on a wider range of systems.

Signed-off-by Punit Agrawal <[email protected]>

---
 rteval/modules/loads/hackbench.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

rteval/modules/loads/hackbench.py +1 -2

index 3b692070e9d9..ab028c495d8b 100644
@@ -55,9 +55,8 @@ class Hackbench(CommandLineLoad):
         if ratio >= 0.75:
             mult = float(self._cfg.setdefault('jobspercore', 2))
         else:
-            self._log(Log.INFO, "Low memory system (%f GB/core)! Not running" % ratio)
+            self._log(Log.WARN, "Low memory system (%f GB/core)!" % ratio)
             mult = 0
-            self._donotrun = True
 
         sysTop = SysTopology()
         # get the number of nodes

Signature · 1 line

2.32.0

John Kacur

2021-09-09 13:43 UTC #12 lore raw

Re: [PATCH 4/5] rteval: hackbench.py: Enable running on a system with low memory

Quoted 38 lines · Punit Agrawal

From: Punit Agrawal <[email protected]>
The hackbench workload refues to run on RockPro64, a hexacore 64bit Arm board with 4GB memory, complaining about insufficient memory per-core.
On further investigation, it turns out that workload is using an arbitrary limit of 0.75 GB/core but will quite happily run on much lower lower memory systems.
Instead of preventing execution, convert the info message to a warning when the memory is lower than expected but continue execution. This should enable the workload to be used on a wider range of systems.
Signed-off-by Punit Agrawal <[email protected]>
---
 rteval/modules/loads/hackbench.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
rteval/modules/loads/hackbench.py +1 -2
index 3b692070e9d9..ab028c495d8b 100644
@@ -55,9 +55,8 @@ class Hackbench(CommandLineLoad):
         if ratio >= 0.75:
             mult = float(self._cfg.setdefault('jobspercore', 2))
         else:
-            self._log(Log.INFO, "Low memory system (%f GB/core)! Not running" % ratio)
+            self._log(Log.WARN, "Low memory system (%f GB/core)!" % ratio)
             mult = 0
-            self._donotrun = True
 
         sysTop = SysTopology()
         # get the number of nodes
Signature · 1 line
2.32.0

I'm not sure that I can accept this. The number isn't entirely arbitrary, it's based on verifying machines as realtime capable for customers, in which case I'd rather it fails early. Maybe there is some other way to indicate that the user is okay with lower memory system, such as passing an --embedded flag or something of that nature?

Punit Agrawal

author

2021-09-13 07:18 UTC #13 lore raw

Re: [PATCH 4/5] rteval: hackbench.py: Enable running on a system with low memory

Hi John,

Quoted 34 lines · John Kacur

Quoted 32 lines · Punit Agrawal
From: Punit Agrawal <[email protected]>
The hackbench workload refues to run on RockPro64, a hexacore 64bit Arm board with 4GB memory, complaining about insufficient memory per-core.
On further investigation, it turns out that workload is using an arbitrary limit of 0.75 GB/core but will quite happily run on much lower lower memory systems.
Instead of preventing execution, convert the info message to a warning when the memory is lower than expected but continue execution. This should enable the workload to be used on a wider range of systems.
Signed-off-by Punit Agrawal <[email protected]>
---
 rteval/modules/loads/hackbench.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
rteval/modules/loads/hackbench.py +1 -2
index 3b692070e9d9..ab028c495d8b 100644
@@ -55,9 +55,8 @@ class Hackbench(CommandLineLoad):
         if ratio >= 0.75:
             mult = float(self._cfg.setdefault('jobspercore', 2))
         else:
-            self._log(Log.INFO, "Low memory system (%f GB/core)! Not running" % ratio)
+            self._log(Log.WARN, "Low memory system (%f GB/core)!" % ratio)
             mult = 0
-            self._donotrun = True
 

Quoted 10 lines

Quoted 6 lines
rteval/modules/loads/hackbench.py +0 -0
         sysTop = SysTopology()
         # get the number of nodes
Signature · 1 line
2.32.0
I'm not sure that I can accept this. The number isn't entirely arbitrary, it's based on verifying machines as realtime capable for customers, in which case I'd rather it fails early.

I think there's a misunderstanding. The above check only prevents the hackbench workload from running - which takes ~2-3MB in the default configuration on the board I tested. rteval (along with cyclictest, kcompile and other workloads) executes without any issues.

In terms of memory requirements for real time systems, I am not sure there is a single number that is valid across all applications or systems. Any such requirement only manages to alienate certain class of rteval users. I thought a warning was a good compromise.

I am hoping you will reconsider the need to introduce a user option for this case.

Quoted 3 lines

Maybe there is some other way to indicate that the user is okay with lower memory system, such as passing an --embedded flag or something of that nature?

If the above doesn't convince you, I will look to adding an option. How about calling the option "--low-memory-system" to clearly state what it enables. "Embedded" is not well-defined and hard to guess what it relates to.

John Kacur

2021-09-13 12:52 UTC #14 lore raw

Re: [PATCH 4/5] rteval: hackbench.py: Enable running on a system with low memory

Quoted 62 lines · Punit Agrawal

Hi John,
Quoted 34 lines · John Kacur
Quoted 32 lines · Punit Agrawal
From: Punit Agrawal <[email protected]>
The hackbench workload refues to run on RockPro64, a hexacore 64bit Arm board with 4GB memory, complaining about insufficient memory per-core.
On further investigation, it turns out that workload is using an arbitrary limit of 0.75 GB/core but will quite happily run on much lower lower memory systems.
Instead of preventing execution, convert the info message to a warning when the memory is lower than expected but continue execution. This should enable the workload to be used on a wider range of systems.
Signed-off-by Punit Agrawal <[email protected]>
---
 rteval/modules/loads/hackbench.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
rteval/modules/loads/hackbench.py +1 -2
index 3b692070e9d9..ab028c495d8b 100644
@@ -55,9 +55,8 @@ class Hackbench(CommandLineLoad):
         if ratio >= 0.75:
             mult = float(self._cfg.setdefault('jobspercore', 2))
         else:
-            self._log(Log.INFO, "Low memory system (%f GB/core)! Not running" % ratio)
+            self._log(Log.WARN, "Low memory system (%f GB/core)!" % ratio)
             mult = 0
-            self._donotrun = True
 
Quoted 10 lines
Quoted 6 lines
rteval/modules/loads/hackbench.py +0 -0
         sysTop = SysTopology()
         # get the number of nodes
Signature · 1 line
2.32.0
I'm not sure that I can accept this. The number isn't entirely arbitrary, it's based on verifying machines as realtime capable for customers, in which case I'd rather it fails early.
I think there's a misunderstanding. The above check only prevents the hackbench workload from running - which takes ~2-3MB in the default configuration on the board I tested. rteval (along with cyclictest, kcompile and other workloads) executes without any issues.
In terms of memory requirements for real time systems, I am not sure there is a single number that is valid across all applications or systems. Any such requirement only manages to alienate certain class of rteval users. I thought a warning was a good compromise.
I am hoping you will reconsider the need to introduce a user option for this case.

Quoted 9 lines

Quoted 3 lines
Maybe there is some other way to indicate that the user is okay with lower memory system, such as passing an --embedded flag or something of that nature?
If the above doesn't convince you, I will look to adding an option. How about calling the option "--low-memory-system" to clearly state what it enables. "Embedded" is not well-defined and hard to guess what it relates to.

You're right, the original code only prevents hackbench from running, it doesn't cause rteval to fail early.

Hmmn, it almost seems like the original code was a development hack that wasn't removed. Not sure, give me a short time to think about it.

John

John Kacur

2021-09-14 18:32 UTC #15 lore raw

Re: [PATCH 4/5] rteval: hackbench.py: Enable running on a system with low memory

In reply to Punit Agrawal

Quoted 38 lines · Punit Agrawal

From: Punit Agrawal <[email protected]>
The hackbench workload refues to run on RockPro64, a hexacore 64bit Arm board with 4GB memory, complaining about insufficient memory per-core.
On further investigation, it turns out that workload is using an arbitrary limit of 0.75 GB/core but will quite happily run on much lower lower memory systems.
Instead of preventing execution, convert the info message to a warning when the memory is lower than expected but continue execution. This should enable the workload to be used on a wider range of systems.
Signed-off-by Punit Agrawal <[email protected]>
---
 rteval/modules/loads/hackbench.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
rteval/modules/loads/hackbench.py +1 -2
index 3b692070e9d9..ab028c495d8b 100644
@@ -55,9 +55,8 @@ class Hackbench(CommandLineLoad):
         if ratio >= 0.75:
             mult = float(self._cfg.setdefault('jobspercore', 2))
         else:
-            self._log(Log.INFO, "Low memory system (%f GB/core)! Not running" % ratio)
+            self._log(Log.WARN, "Low memory system (%f GB/core)!" % ratio)
             mult = 0
-            self._donotrun = True
 
         sysTop = SysTopology()
         # get the number of nodes
Signature · 1 line
2.32.0

Signed-off-by John Kacur <[email protected]>

Punit Agrawal

author

2021-09-15 01:55 UTC #16 lore raw

Re: [PATCH 4/5] rteval: hackbench.py: Enable running on a system with low memory

Quoted 34 lines · John Kacur

Quoted 32 lines · Punit Agrawal
From: Punit Agrawal <[email protected]>
The hackbench workload refues to run on RockPro64, a hexacore 64bit Arm board with 4GB memory, complaining about insufficient memory per-core.
On further investigation, it turns out that workload is using an arbitrary limit of 0.75 GB/core but will quite happily run on much lower lower memory systems.
Instead of preventing execution, convert the info message to a warning when the memory is lower than expected but continue execution. This should enable the workload to be used on a wider range of systems.
Signed-off-by Punit Agrawal <[email protected]>
---
 rteval/modules/loads/hackbench.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
rteval/modules/loads/hackbench.py +1 -2
index 3b692070e9d9..ab028c495d8b 100644
@@ -55,9 +55,8 @@ class Hackbench(CommandLineLoad):
         if ratio >= 0.75:
             mult = float(self._cfg.setdefault('jobspercore', 2))
         else:
-            self._log(Log.INFO, "Low memory system (%f GB/core)! Not running" % ratio)
+            self._log(Log.WARN, "Low memory system (%f GB/core)!" % ratio)
             mult = 0
-            self._donotrun = True
 

Quoted 7 lines

Quoted 6 lines
rteval/modules/loads/hackbench.py +0 -0
         sysTop = SysTopology()
         # get the number of nodes
Signature · 1 line
2.32.0
Signed-off-by John Kacur <[email protected]>

Thanks for applying the patches - both from here and the misc-fixes. I will address the comments on the cyclictest set and resend them.

Punit Agrawal

author

2021-09-01 08:08 UTC #17 lore raw

PATCH 5/5 rteval: kcompile.py: Gracefully handle missing affinity information

In reply to Punit Agrawal

From: Punit Agrawal <[email protected]>

kcompile either sets the affinity of the workload threads to a user specified list of cpus (using taskset) or binds the threads to the local numa node (using numactl). In the absence of both the following hard to parse error is thrown -

Exception in thread kcompile:
Traceback (most recent call last):
...
File "...rteval/rteval/modules/loads/kcompile.py", line 55, in __init__
self.binder = 'taskset -c %s' % compress_cpulist(cpulist)
File "...rteval/rteval/misc.py", line 62, in compress_cpulist
if isinstance(cpulist[0], int):
TypeError: 'NoneType' object is not subscriptable

Instead, add a warning to report the missing affinity information / tools and continue executing the workload with no affinity - relying on the affinity settings being set by the kernel.

Signed-off-by Punit Agrawal <[email protected]>

---
 rteval/modules/loads/kcompile.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

rteval/modules/loads/kcompile.py +5 -1

index d1955c7aee3d..5c83b78525e5 100644
@@ -51,8 +51,12 @@ class KBuildJob:
             os.mkdir(self.objdir)
         if os.path.exists('/usr/bin/numactl') and not cpulist:
             self.binder = 'numactl --cpunodebind %d' % int(self.node)
-        else:
+        elif cpulist is not None:
             self.binder = 'taskset -c %s' % compress_cpulist(cpulist)
+        else:
+            self.log(Log.WARN, 'Unable to set job affinity - please install "numactl" or pass "cpulist"')
+            self.log(Log.WARN, 'Running with default OS decided affinity')
+            self.binder = ''
         if cpulist:
             self.jobs = self.calc_jobs_per_cpu() * len(cpulist)
         else:

Signature · 1 line

2.32.0

John Kacur

2021-09-09 19:23 UTC #18 lore raw

Re: [PATCH 5/5] rteval: kcompile.py: Gracefully handle missing affinity information

Quoted 47 lines · Punit Agrawal

From: Punit Agrawal <[email protected]>
kcompile either sets the affinity of the workload threads to a user specified list of cpus (using taskset) or binds the threads to the local numa node (using numactl). In the absence of both the following hard to parse error is thrown -
Exception in thread kcompile:
Traceback (most recent call last):
...
File "...rteval/rteval/modules/loads/kcompile.py", line 55, in __init__
self.binder = 'taskset -c %s' % compress_cpulist(cpulist)
File "...rteval/rteval/misc.py", line 62, in compress_cpulist
if isinstance(cpulist[0], int):
TypeError: 'NoneType' object is not subscriptable
Instead, add a warning to report the missing affinity information / tools and continue executing the workload with no affinity - relying on the affinity settings being set by the kernel.
Signed-off-by Punit Agrawal <[email protected]>
---
 rteval/modules/loads/kcompile.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
rteval/modules/loads/kcompile.py +5 -1
index d1955c7aee3d..5c83b78525e5 100644
@@ -51,8 +51,12 @@ class KBuildJob:
             os.mkdir(self.objdir)
         if os.path.exists('/usr/bin/numactl') and not cpulist:
             self.binder = 'numactl --cpunodebind %d' % int(self.node)
-        else:
+        elif cpulist is not None:
             self.binder = 'taskset -c %s' % compress_cpulist(cpulist)
+        else:
+            self.log(Log.WARN, 'Unable to set job affinity - please install "numactl" or pass "cpulist"')
+            self.log(Log.WARN, 'Running with default OS decided affinity')
+            self.binder = ''
         if cpulist:
             self.jobs = self.calc_jobs_per_cpu() * len(cpulist)
         else:
Signature · 1 line
2.32.0

Wow, that's some messy code! (Not yours, the original).

I think for embedded arm distros, numactl is not always available, so we shouldn't tell users to install it, and just make the code work when it's not there. Also, it should be perfectly fine to run without passing a cpulist, so it's not something we should warn the users about.

You've got the right idea to just make the self.binder empty. I've got a version that cleans-up a few more things,
I've tested with no numactl and no cpulist, no numactl and a cpulist with numactl and no cpulist and with numactl and with a cpulist.

Could you give it a test on your arm system?
Sending in a separte mail

John