Skip to contentSkip to conversation

rteval: Add cmdline option to run measurements on isolcpus by default

PATCH In review Tomas Glozar 2 participants 8 messages Aug 11, 2023 – Sep 15, 2023

lore
Share & tools
lore
#0 lore raw

v2 cover rteval: Add cmdline option to run measurements on isolcpus by default

A common usecase for rteval is estimating the performance of workloads running on isolated CPUs. rteval does not run measurement on isolated CPUs by default (i.e. without --measurement-cpulist specified).

Add option --measurement-run-on-isolcpus that adds isolated CPUs to the default cpulist together with rteval.conf [measurement] option run-on-isolcpus with the same effect.

v2: Fixed error in rteval/modules/measurement/__init__.py that broke rteval when run-on-isolcpus not present in rteval.conf.

Tomas Glozar (3):
rteval: Allow arguments specific to module group
rteval: Add run_on_isolcpus option to measurements
rteval: Support run-on-isolcpus in cyclictest

 rteval/modules/__init__.py               | 18 ++++++++++--------
 rteval/modules/measurement/__init__.py   | 17 ++++++++++++++---
 rteval/modules/measurement/cyclictest.py | 12 ++++++++++--
 3 files changed, 34 insertions(+), 13 deletions(-)
Signature · 1 line
2.41.0
#1 lore raw

PATCH v2 1/3 rteval: Allow arguments specific to module group

Return grparser from ModuleContainer.SetupModuleOptions, which allows it to be used to setup additional options in *Modules class.

---
 rteval/modules/__init__.py | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
rteval/modules/__init__.py +10 -8
index 794135d..f4fd160 100644
@@ -312,7 +312,7 @@ the information provided by the module"""
                 # Ignore if a section is not found
                 cfg = None
 
-            grparser = parser.add_argument_group("Options for the %s module" % shortmod)
+            modgrparser = parser.add_argument_group("Options for the %s module" % shortmod)
             for (o, s) in list(opts.items()):
                 descr = 'descr' in s and s['descr'] or ""
                 metavar = 'metavar' in s and s['metavar'] or None
@@ -327,13 +327,15 @@ the information provided by the module"""
                     default = 'default' in s and s['default'] or None
 
 
-                grparser.add_argument('--%s-%s' % (shortmod, o),
-                                    dest="%s___%s" % (shortmod, o),
-                                    action='store',
-                                    help='%s%s' % (descr,
-                                                   default and ' (default: %s)' % default or ''),
-                                    default=default,
-                                    metavar=metavar)
+                modgrparser.add_argument('--%s-%s' % (shortmod, o),
+                                         dest="%s___%s" % (shortmod, o),
+                                         action='store',
+                                         help='%s%s' % (descr,
+                                                        default and ' (default: %s)' % default or ''),
+                                         default=default,
+                                         metavar=metavar)
+
+            return grparser
 
 
     def InstantiateModule(self, modname, modcfg, modroot=None):
Signature · 1 line
2.41.0
#2 lore raw

Re: [PATCH v2 1/3] rteval: Allow arguments specific to module group

Quoted 48 lines · Tomas Glozar

Return grparser from ModuleContainer.SetupModuleOptions, which allows it to be used to setup additional options in *Modules class.

---
 rteval/modules/__init__.py | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
rteval/modules/__init__.py +10 -8
index 794135d..f4fd160 100644
@@ -312,7 +312,7 @@ the information provided by the module"""
                 # Ignore if a section is not found
                 cfg = None
 
-            grparser = parser.add_argument_group("Options for the %s module" % shortmod)
+            modgrparser = parser.add_argument_group("Options for the %s module" % shortmod)
             for (o, s) in list(opts.items()):
                 descr = 'descr' in s and s['descr'] or ""
                 metavar = 'metavar' in s and s['metavar'] or None
@@ -327,13 +327,15 @@ the information provided by the module"""
                     default = 'default' in s and s['default'] or None
 
 
-                grparser.add_argument('--%s-%s' % (shortmod, o),
-                                    dest="%s___%s" % (shortmod, o),
-                                    action='store',
-                                    help='%s%s' % (descr,
-                                                   default and ' (default: %s)' % default or ''),
-                                    default=default,
-                                    metavar=metavar)
+                modgrparser.add_argument('--%s-%s' % (shortmod, o),
+                                         dest="%s___%s" % (shortmod, o),
+                                         action='store',
+                                         help='%s%s' % (descr,
+                                                        default and ' (default: %s)' % default or ''),
+                                         default=default,
+                                         metavar=metavar)
+
+            return grparser
 
 
     def InstantiateModule(self, modname, modcfg, modroot=None):
Signature · 1 line
2.41.0
#3 lore raw

PATCH v2 2/3 rteval: Add run_on_isolcpus option to measurements

In reply to Tomas Glozar

Add option --measurement-run-on-isolcpus for the user to specify to include isolcpus in the default cpulist (without --measurement-cpulist).

A default value might also be specified in rteval.conf:
[measurement]
run-on-isolcpus: true

Note that values different from true evaluate as false.

---
 rteval/modules/measurement/__init__.py | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)
rteval/modules/measurement/__init__.py +14 -3
index 0e395be..5258dd7 100644
@@ -146,7 +146,15 @@ measurement profiles, based on their characteristics"""
 
     def SetupModuleOptions(self, parser):
         "Sets up all the measurement modules' parameters for the option parser"
-        self.__container.SetupModuleOptions(parser, self.__cfg)
+        grparser = self.__container.SetupModuleOptions(parser, self.__cfg)
+
+        # Set up options specific for measurement module group
+        grparser.add_argument("--measurement-run-on-isolcpus",
+                              dest="measurement___run_on_isolcpus",
+                              action="store_true",
+                              default=self.__cfg.GetSection("measurement").setdefault("run-on-isolcpus", "false").lower()
+                                      == "true",
+                              help="Include isolated CPUs in default cpulist")
 
 
     def Setup(self, modparams):
@@ -157,9 +165,10 @@ measurement profiles, based on their characteristics"""
 
         modcfg = self.__cfg.GetSection("measurement")
         cpulist = modcfg.cpulist
+        run_on_isolcpus = modcfg.run_on_isolcpus
 
         for (modname, modtype) in modcfg:
-            if modtype.lower() == 'module':  # Only 'module' will be supported (ds)
+            if isinstance(modtype, str) and modtype.lower() == 'module':  # Only 'module' will be supported (ds)
                 # Extract the measurement modules info
                 modinfo = self.__container.ModuleInfo(modname)
 
@@ -179,6 +188,7 @@ measurement profiles, based on their characteristics"""
                 # Setup this imported module inside the appropriate measurement profile
                 self.__cfg.AppendConfig(modname, modparams)
                 self.__cfg.AppendConfig(modname, {'cpulist':cpulist})
+                self.__cfg.AppendConfig(modname, {'run-on-isolcpus':run_on_isolcpus})
                 mp.Setup(modname)
 
         del self.__container
@@ -190,11 +200,12 @@ measurement profiles, based on their characteristics"""
         # Get the reports from all meaurement modules in all measurement profiles
         rep_n = libxml2.newNode("Measurements")
         cpulist = self.__cfg.GetSection("measurement").cpulist
+        run_on_isolcpus = self.__cfg.GetSection("measurement").run_on_isolcpus
         if cpulist:
             # Convert str to list and remove offline cpus
             cpulist = CpuList(cpulist).cpulist
         else:
-            cpulist = SysTop().default_cpus()
+            cpulist = SysTop().online_cpus() if run_on_isolcpus else SysTop().default_cpus()
         rep_n.newProp("measurecpus", collapse_cpulist(cpulist))
 
         for mp in self.__measureprofiles:
Signature · 1 line
2.41.0
#4 lore raw

Re: [PATCH v2 2/3] rteval: Add run_on_isolcpus option to measurements

Quoted 73 lines · Tomas Glozar

Add option --measurement-run-on-isolcpus for the user to specify to include isolcpus in the default cpulist (without --measurement-cpulist).

A default value might also be specified in rteval.conf:
[measurement]
run-on-isolcpus: true

Note that values different from true evaluate as false.

---
 rteval/modules/measurement/__init__.py | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)
rteval/modules/measurement/__init__.py +14 -3
index 0e395be..5258dd7 100644
@@ -146,7 +146,15 @@ measurement profiles, based on their characteristics"""
 
     def SetupModuleOptions(self, parser):
         "Sets up all the measurement modules' parameters for the option parser"
-        self.__container.SetupModuleOptions(parser, self.__cfg)
+        grparser = self.__container.SetupModuleOptions(parser, self.__cfg)
+
+        # Set up options specific for measurement module group
+        grparser.add_argument("--measurement-run-on-isolcpus",
+                              dest="measurement___run_on_isolcpus",
+                              action="store_true",
+                              default=self.__cfg.GetSection("measurement").setdefault("run-on-isolcpus", "false").lower()
+                                      == "true",
+                              help="Include isolated CPUs in default cpulist")
 
 
     def Setup(self, modparams):
@@ -157,9 +165,10 @@ measurement profiles, based on their characteristics"""
 
         modcfg = self.__cfg.GetSection("measurement")
         cpulist = modcfg.cpulist
+        run_on_isolcpus = modcfg.run_on_isolcpus
 
         for (modname, modtype) in modcfg:
-            if modtype.lower() == 'module':  # Only 'module' will be supported (ds)
+            if isinstance(modtype, str) and modtype.lower() == 'module':  # Only 'module' will be supported (ds)
                 # Extract the measurement modules info
                 modinfo = self.__container.ModuleInfo(modname)
 
@@ -179,6 +188,7 @@ measurement profiles, based on their characteristics"""
                 # Setup this imported module inside the appropriate measurement profile
                 self.__cfg.AppendConfig(modname, modparams)
                 self.__cfg.AppendConfig(modname, {'cpulist':cpulist})
+                self.__cfg.AppendConfig(modname, {'run-on-isolcpus':run_on_isolcpus})
                 mp.Setup(modname)
 
         del self.__container
@@ -190,11 +200,12 @@ measurement profiles, based on their characteristics"""
         # Get the reports from all meaurement modules in all measurement profiles
         rep_n = libxml2.newNode("Measurements")
         cpulist = self.__cfg.GetSection("measurement").cpulist
+        run_on_isolcpus = self.__cfg.GetSection("measurement").run_on_isolcpus
         if cpulist:
             # Convert str to list and remove offline cpus
             cpulist = CpuList(cpulist).cpulist
         else:
-            cpulist = SysTop().default_cpus()
+            cpulist = SysTop().online_cpus() if run_on_isolcpus else SysTop().default_cpus()
         rep_n.newProp("measurecpus", collapse_cpulist(cpulist))
 
         for mp in self.__measureprofiles:
Signature · 1 line
2.41.0
#5 lore raw

Re: [PATCH v2 2/3] rteval: Add run_on_isolcpus option to measurements

In reply to Tomas Glozar
Quoted 73 lines · Tomas Glozar

Add option --measurement-run-on-isolcpus for the user to specify to include isolcpus in the default cpulist (without --measurement-cpulist).

A default value might also be specified in rteval.conf:
[measurement]
run-on-isolcpus: true

Note that values different from true evaluate as false.

---
 rteval/modules/measurement/__init__.py | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)
rteval/modules/measurement/__init__.py +14 -3
index 0e395be..5258dd7 100644
@@ -146,7 +146,15 @@ measurement profiles, based on their characteristics"""
 
     def SetupModuleOptions(self, parser):
         "Sets up all the measurement modules' parameters for the option parser"
-        self.__container.SetupModuleOptions(parser, self.__cfg)
+        grparser = self.__container.SetupModuleOptions(parser, self.__cfg)
+
+        # Set up options specific for measurement module group
+        grparser.add_argument("--measurement-run-on-isolcpus",
+                              dest="measurement___run_on_isolcpus",
+                              action="store_true",
+                              default=self.__cfg.GetSection("measurement").setdefault("run-on-isolcpus", "false").lower()
+                                      == "true",
+                              help="Include isolated CPUs in default cpulist")
 
 
     def Setup(self, modparams):
@@ -157,9 +165,10 @@ measurement profiles, based on their characteristics"""
 
         modcfg = self.__cfg.GetSection("measurement")
         cpulist = modcfg.cpulist
+        run_on_isolcpus = modcfg.run_on_isolcpus
 
         for (modname, modtype) in modcfg:
-            if modtype.lower() == 'module':  # Only 'module' will be supported (ds)
+            if isinstance(modtype, str) and modtype.lower() == 'module':  # Only 'module' will be supported (ds)
                 # Extract the measurement modules info
                 modinfo = self.__container.ModuleInfo(modname)
 
@@ -179,6 +188,7 @@ measurement profiles, based on their characteristics"""
                 # Setup this imported module inside the appropriate measurement profile
                 self.__cfg.AppendConfig(modname, modparams)
                 self.__cfg.AppendConfig(modname, {'cpulist':cpulist})
+                self.__cfg.AppendConfig(modname, {'run-on-isolcpus':run_on_isolcpus})
                 mp.Setup(modname)
 
         del self.__container
@@ -190,11 +200,12 @@ measurement profiles, based on their characteristics"""
         # Get the reports from all meaurement modules in all measurement profiles
         rep_n = libxml2.newNode("Measurements")
         cpulist = self.__cfg.GetSection("measurement").cpulist
+        run_on_isolcpus = self.__cfg.GetSection("measurement").run_on_isolcpus
         if cpulist:
             # Convert str to list and remove offline cpus
             cpulist = CpuList(cpulist).cpulist
         else:
-            cpulist = SysTop().default_cpus()
+            cpulist = SysTop().online_cpus() if run_on_isolcpus else SysTop().default_cpus()
         rep_n.newProp("measurecpus", collapse_cpulist(cpulist))
 
         for mp in self.__measureprofiles:
Signature · 1 line
2.41.0
#6 lore raw

PATCH v2 3/3 rteval: Support run-on-isolcpus in cyclictest

In reply to Tomas Glozar

If --measurement-run-on-isolcpus is enabled, add isolated CPUs to the cpumask for the case when --measurement-cpulist is not specified. If both options are specified, display a warning.

---
 rteval/modules/measurement/cyclictest.py | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
rteval/modules/measurement/cyclictest.py +10 -2
index ace8db4..4d7fcf2 100644
@@ -214,6 +214,7 @@ class Cyclictest(rtevalModulePrototype):
         self.__cpus = []
         self.__cyclicdata = {}
         self.__sparse = False
+        self.__run_on_isolcpus = bool(self.__cfg.setdefault('run-on-isolcpus', False))
 
         if self.__cfg.cpulist:
             self.__cpulist = self.__cfg.cpulist
@@ -224,14 +225,21 @@ class Cyclictest(rtevalModulePrototype):
             self.__cpulist = collapse_cpulist(self.__cpus)
             self.__cpus = [str(c) for c in self.__cpus]
             self.__sparse = True
+            if self.__run_on_isolcpus:
+                self._log(Log.WARN, "ignoring --measurement-run-on-isolcpus, since cpulist is specified")
         else:
             self.__cpus = SysTopology().online_cpus_str()
             # Get the cpuset from the environment
             cpuset = os.sched_getaffinity(0)
             # Convert the elements to strings
             cpuset = [str(c) for c in cpuset]
-            # Only include cpus that are in the cpuset
-            self.__cpus = [c for c in self.__cpus if c in cpuset]
+            # Get isolated CPU list
+            isolcpus = [str(c) for c in SysTopology().isolated_cpus()]
+            # Only include cpus that are in the cpuset and isolated CPUs if run_on_isolcpus is enabled
+            self.__cpus = [c for c in self.__cpus if c in cpuset or self.__run_on_isolcpus and c in isolcpus]
+            if self.__run_on_isolcpus:
+                self.__sparse = True
+                self.__cpulist = collapse_cpulist(self.__cpus)
 
         # Sort the list of cpus to align with the order reported by cyclictest
         self.__cpus.sort(key=int)
Signature · 1 line
2.41.0
#7 lore raw

Re: [PATCH v2 3/3] rteval: Support run-on-isolcpus in cyclictest

Quoted 49 lines · Tomas Glozar

If --measurement-run-on-isolcpus is enabled, add isolated CPUs to the cpumask for the case when --measurement-cpulist is not specified. If both options are specified, display a warning.

---
 rteval/modules/measurement/cyclictest.py | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
rteval/modules/measurement/cyclictest.py +10 -2
index ace8db4..4d7fcf2 100644
@@ -214,6 +214,7 @@ class Cyclictest(rtevalModulePrototype):
         self.__cpus = []
         self.__cyclicdata = {}
         self.__sparse = False
+        self.__run_on_isolcpus = bool(self.__cfg.setdefault('run-on-isolcpus', False))
 
         if self.__cfg.cpulist:
             self.__cpulist = self.__cfg.cpulist
@@ -224,14 +225,21 @@ class Cyclictest(rtevalModulePrototype):
             self.__cpulist = collapse_cpulist(self.__cpus)
             self.__cpus = [str(c) for c in self.__cpus]
             self.__sparse = True
+            if self.__run_on_isolcpus:
+                self._log(Log.WARN, "ignoring --measurement-run-on-isolcpus, since cpulist is specified")
         else:
             self.__cpus = SysTopology().online_cpus_str()
             # Get the cpuset from the environment
             cpuset = os.sched_getaffinity(0)
             # Convert the elements to strings
             cpuset = [str(c) for c in cpuset]
-            # Only include cpus that are in the cpuset
-            self.__cpus = [c for c in self.__cpus if c in cpuset]
+            # Get isolated CPU list
+            isolcpus = [str(c) for c in SysTopology().isolated_cpus()]
+            # Only include cpus that are in the cpuset and isolated CPUs if run_on_isolcpus is enabled
+            self.__cpus = [c for c in self.__cpus if c in cpuset or self.__run_on_isolcpus and c in isolcpus]
+            if self.__run_on_isolcpus:
+                self.__sparse = True
+                self.__cpulist = collapse_cpulist(self.__cpus)
 
         # Sort the list of cpus to align with the order reported by cyclictest
         self.__cpus.sort(key=int)
Signature · 1 line
2.41.0

--

I'm including this but we might need to finess the model. Right now this is the intersection of isolated cpus plus the mask if given.
Other possibilities would be for the mask to override the isolated cpus, but perhaps we want the user to indicate, are we adding to the isolated cpus to include more cpus, are we subtracing, etc