Skip to contentSkip to conversation

Add cpupower idle-state functionality

PATCH Superseded John B. Wyatt IV 4 participants 9 messages Mar 11, 2025 – Apr 5, 2025

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v4 cover Add cpupower idle-state functionality

This patch series adds idle-state functionality to control cpu power usage and to test idle states.

The number of cpus was needed in the cpupower file; I extracted out the previously local to tuna-cli.py functionality to a separate file so the cpu code can be used in any file in Tuna and reduce duplications. The nics code was similar so it was also extracted to reduce the number of global variables.

Sincerely,
John Wyatt
Software Engineer, Core Kernel
Red Hat

Changes v3 -> v4:
- Additional changes suggest by John Kacur around the use of @classmethods that they should've been @staticmethods and the changes needed for them.
- Changed the SPDX lines to be at the top of the files as requested by John Kacur.

Changes v2 -> v3:
- Several small improvements suggested by John Kacur off list including removing unnecessary string interpolation, renaming idle-set to idle_set, and correct placement of docstrings.

Changes v1 -> v2:
- Numerous improvements suggested by Crystal Wood including message text, output, error handling, moving a function to utils.py and structure of the code.
- Fixed a libcpupower bindings detection error that did not show on my local machine but did on a fresh install of Fedora GNOME 40 reported by John Kacur.

John B. Wyatt IV (2):
tuna: extract common cpu and nics determination code into a utils.py
file
tuna: Add idle_state control functionality

 tuna-cmd.py      |  64 +++++++++--------
 tuna/cpupower.py | 184 +++++++++++++++++++++++++++++++++++++++++++++++
 tuna/utils.py    |  28 ++++++++
 3 files changed, 246 insertions(+), 30 deletions(-)
 create mode 100755 tuna/cpupower.py
 create mode 100644 tuna/utils.py
Signature · 1 line
2.48.1
#1 lore raw

PATCH v4 1/2 tuna: extract common cpu and nics determination code into a utils.py file

Extracting the code allows these previously local (global to the file) variables and functions to be used in other files of tuna. Reducing the number of globals makes the code cleaner and reduces the size of tuna-cmd.py.

Included a suggestion by Crystal to move a function from the latter patch into utils.py and make it dependant on get_nr_cpus() (v2).

Included a request by John Kacur to place the SPDX message at the top of the file (v4).

---
 tuna-cmd.py   | 34 +++++++---------------------------
 tuna/utils.py | 28 ++++++++++++++++++++++++++++
 2 files changed, 35 insertions(+), 27 deletions(-)
 create mode 100644 tuna/utils.py
tuna-cmd.py +7 -27
index f37e286..d0323f5 100755
@@ -21,7 +21,7 @@ from functools import reduce
 import tuna.new_eth as ethtool
 import tuna.tuna_sched as tuna_sched
 import procfs
-from tuna import tuna, sysfs
+from tuna import tuna, sysfs, utils
 import logging
 import time
 import shutil
@@ -76,7 +76,6 @@ except:
 
 # FIXME: ETOOMANYGLOBALS, we need a class!
 
-nr_cpus = None
 ps = None
 irqs = None
 
@@ -233,25 +232,6 @@ def gen_parser():
     return parser
 
 
-def get_nr_cpus():
-    """ Get all cpus including disabled cpus """
-    global nr_cpus
-    if nr_cpus:
-        return nr_cpus
-    nr_cpus = os.sysconf('SC_NPROCESSORS_CONF')
-    return nr_cpus
-
-nics = None
-
-
-def get_nics():
-    global nics
-    if nics:
-        return nics
-    nics = ethtool.get_active_devices()
-    return nics
-
-
 def thread_help(tid):
     global ps
     if not ps:
@@ -277,7 +257,7 @@ def save(cpu_list, thread_list, filename):
         if (cpu_list and not set(kt.affinity).intersection(set(cpu_list))) or \
            (thread_list and kt.pid not in thread_list):
             del kthreads[name]
-    tuna.generate_rtgroups(filename, kthreads, get_nr_cpus())
+    tuna.generate_rtgroups(filename, kthreads, utils.get_nr_cpus())
 
 
 def ps_show_header(has_ctxt_switch_info, cgroups=False):
@@ -328,7 +308,7 @@ def format_affinity(affinity):
     if len(affinity) <= 4:
         return ",".join(str(a) for a in affinity)
 
-    return ",".join(str(hex(a)) for a in procfs.hexbitmask(affinity, get_nr_cpus()))
+    return ",".join(str(hex(a)) for a in procfs.hexbitmask(affinity, utils.get_nr_cpus()))
 
 def ps_show_thread(pid, affect_children, ps, has_ctxt_switch_info, sock_inodes,
                    sock_inode_re, cgroups, columns=None, compact=True):
@@ -351,7 +331,7 @@ def ps_show_thread(pid, affect_children, ps, has_ctxt_switch_info, sock_inodes,
                 irqs = procfs.interrupts()
             users = irqs[tuna.irq_thread_number(cmd)]["users"]
             for u in users:
-                if u in get_nics():
+                if u in utils.get_nics():
                     users[users.index(u)] = "%s(%s)" % (
                         u, ethtool.get_module(u))
             users = ",".join(users)
@@ -486,7 +466,7 @@ def do_ps(thread_list, cpu_list, irq_list, show_uthreads, show_kthreads,
 
 
 def find_drivers_by_users(users):
-    nics = get_nics()
+    nics = utils.get_nics()
     drivers = []
     for u in users:
         try:
@@ -689,10 +669,10 @@ def main():
         apply_config(args.profilename)
 
     elif args.command in ['include', 'I']:
-        tuna.include_cpus(args.cpu_list, get_nr_cpus())
+        tuna.include_cpus(args.cpu_list, utils.get_nr_cpus())
 
     elif args.command in ['isolate', 'i']:
-        tuna.isolate_cpus(args.cpu_list, get_nr_cpus())
+        tuna.isolate_cpus(args.cpu_list, utils.get_nr_cpus())
 
     elif args.command in ['run', 'r']:
 
tuna/utils.py new file +28 -0
new file mode 100644
index 0000000..f55432d
@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright (C) 2024 John B. Wyatt IV
+
+import os
+
+import tuna.new_eth as ethtool
+
+# Collect a few globals and functions so they can be reused in other modules
+nr_cpus = None
+nics = None
+
+def get_nr_cpus():
+    """ Get all cpus including disabled cpus """
+    global nr_cpus
+    if nr_cpus != None:
+        return nr_cpus
+    nr_cpus = os.sysconf('SC_NPROCESSORS_CONF')
+    return nr_cpus
+
+def get_all_cpu_list():
+    return list(range(get_nr_cpus()))
+
+def get_nics():
+    global nics
+    if nics != None:
+        return nics
+    nics = ethtool.get_active_devices()
+    return nics
Signature · 1 line
2.48.1
#2 lore raw

Re: [PATCH v4 1/2] tuna: extract common cpu and nics determination code into a utils.py file

Quoted 128 lines · John B. Wyatt IV

Extracting the code allows these previously local (global to the file) variables and functions to be used in other files of tuna. Reducing the number of globals makes the code cleaner and reduces the size of tuna-cmd.py.

Included a suggestion by Crystal to move a function from the latter patch into utils.py and make it dependant on get_nr_cpus() (v2).

Included a request by John Kacur to place the SPDX message at the top of the file (v4).

---
 tuna-cmd.py   | 34 +++++++---------------------------
 tuna/utils.py | 28 ++++++++++++++++++++++++++++
 2 files changed, 35 insertions(+), 27 deletions(-)
 create mode 100644 tuna/utils.py
tuna-cmd.py +7 -27
index f37e286..d0323f5 100755
@@ -21,7 +21,7 @@ from functools import reduce
 import tuna.new_eth as ethtool
 import tuna.tuna_sched as tuna_sched
 import procfs
-from tuna import tuna, sysfs
+from tuna import tuna, sysfs, utils
 import logging
 import time
 import shutil
@@ -76,7 +76,6 @@ except:
 
 # FIXME: ETOOMANYGLOBALS, we need a class!
 
-nr_cpus = None
 ps = None
 irqs = None
 
@@ -233,25 +232,6 @@ def gen_parser():
     return parser
 
 
-def get_nr_cpus():
-    """ Get all cpus including disabled cpus """
-    global nr_cpus
-    if nr_cpus:
-        return nr_cpus
-    nr_cpus = os.sysconf('SC_NPROCESSORS_CONF')
-    return nr_cpus
-
-nics = None
-
-
-def get_nics():
-    global nics
-    if nics:
-        return nics
-    nics = ethtool.get_active_devices()
-    return nics
-
-
 def thread_help(tid):
     global ps
     if not ps:
@@ -277,7 +257,7 @@ def save(cpu_list, thread_list, filename):
         if (cpu_list and not set(kt.affinity).intersection(set(cpu_list))) or \
            (thread_list and kt.pid not in thread_list):
             del kthreads[name]
-    tuna.generate_rtgroups(filename, kthreads, get_nr_cpus())
+    tuna.generate_rtgroups(filename, kthreads, utils.get_nr_cpus())
 
 
 def ps_show_header(has_ctxt_switch_info, cgroups=False):
@@ -328,7 +308,7 @@ def format_affinity(affinity):
     if len(affinity) <= 4:
         return ",".join(str(a) for a in affinity)
 
-    return ",".join(str(hex(a)) for a in procfs.hexbitmask(affinity, get_nr_cpus()))
+    return ",".join(str(hex(a)) for a in procfs.hexbitmask(affinity, utils.get_nr_cpus()))
 
 def ps_show_thread(pid, affect_children, ps, has_ctxt_switch_info, sock_inodes,
                    sock_inode_re, cgroups, columns=None, compact=True):
@@ -351,7 +331,7 @@ def ps_show_thread(pid, affect_children, ps, has_ctxt_switch_info, sock_inodes,
                 irqs = procfs.interrupts()
             users = irqs[tuna.irq_thread_number(cmd)]["users"]
             for u in users:
-                if u in get_nics():
+                if u in utils.get_nics():
                     users[users.index(u)] = "%s(%s)" % (
                         u, ethtool.get_module(u))
             users = ",".join(users)
@@ -486,7 +466,7 @@ def do_ps(thread_list, cpu_list, irq_list, show_uthreads, show_kthreads,
 
 
 def find_drivers_by_users(users):
-    nics = get_nics()
+    nics = utils.get_nics()
     drivers = []
     for u in users:
         try:
@@ -689,10 +669,10 @@ def main():
         apply_config(args.profilename)
 
     elif args.command in ['include', 'I']:
-        tuna.include_cpus(args.cpu_list, get_nr_cpus())
+        tuna.include_cpus(args.cpu_list, utils.get_nr_cpus())
 
     elif args.command in ['isolate', 'i']:
-        tuna.isolate_cpus(args.cpu_list, get_nr_cpus())
+        tuna.isolate_cpus(args.cpu_list, utils.get_nr_cpus())
 
     elif args.command in ['run', 'r']:
 
tuna/utils.py new file +5 -0
new file mode 100644
index 0000000..f55432d
@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright (C) 2024 John B. Wyatt IV
+
+import os
+

tuna/utils.py line 15

Quoted 10 lines
tuna/utils.py new file +10 -0
+import tuna.new_eth as ethtool
+
+# Collect a few globals and functions so they can be reused in other modules
+nr_cpus = None
+nics = None
+
+def get_nr_cpus():
+    """ Get all cpus including disabled cpus """
+    global nr_cpus
+    if nr_cpus != None:

Really a nit: as a general standard, objects should be compared to None using the "is" operator.

tuna/utils.py line 25

Quoted 10 lines
tuna/utils.py new file +10 -0
+        return nr_cpus
+    nr_cpus = os.sysconf('SC_NPROCESSORS_CONF')
+    return nr_cpus
+
+def get_all_cpu_list():
+    return list(range(get_nr_cpus()))
+
+def get_nics():
+    global nics
+    if nics != None:

Ditto.

Quoted 6 lines
tuna/utils.py new file +3 -0
+        return nics
+    nics = ethtool.get_active_devices()
+    return nics
Signature · 1 line
2.48.1
#3 lore raw

PATCH v4 2/2 tuna: Add idle_state control functionality

In reply to John B. Wyatt IV

Allows Tuna to control cpu idle-state functionality on the system, including querying, enabling, disabling of cpu idle-states to control power usage or to test functionality.

This requires cpupower, a utility in the Linux kernel repository and the cpupower Python bindings added in Linux 6.12 to control cpu idle-states.

This patch revision includes text snippet suggestions by Crystal Wood (v2) and small Python suggestions by John Kacur (v3 and v4).

---
 tuna-cmd.py      |  30 +++++++-
 tuna/cpupower.py | 184 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 211 insertions(+), 3 deletions(-)
 create mode 100755 tuna/cpupower.py
tuna-cmd.py +27 -3
index d0323f5..96a25a5 100755
@@ -25,6 +25,7 @@ from tuna import tuna, sysfs, utils
 import logging
 import time
 import shutil
+import tuna.cpupower as cpw
 
 def get_loglevel(level):
     if level.isdigit() and int(level) in range(0,5):
@@ -115,8 +116,12 @@ def gen_parser():
             "disable_perf": dict(action='store_true', help="Explicitly disable usage of perf in GUI for process view"),
             "refresh": dict(default=2500, metavar='MSEC', type=int, help="Refresh the GUI every MSEC milliseconds"),
             "priority": dict(default=(None, None), metavar="POLICY:RTPRIO", type=tuna.get_policy_and_rtprio, help="Set thread scheduler tunables: POLICY and RTPRIO"),
-            "background": dict(action='store_true', help="Run command as background task")
-         }
+            "background": dict(action='store_true', help="Run command as background task"),
+            "idle_info": dict(dest='idle_info', action='store_const', const=True, help='Print general idle information for the selected CPUs, including index values for IDLE-STATE.'),
+            "idle_state_disabled_status": dict(dest='idle_state_disabled_status', metavar='IDLE-STATE', type=int, help='Print whether IDLE-STATE is enabled on the selected	CPUs.'),
+            "disable_idle_state": dict(dest='disable_idle_state', metavar='IDLE-STATE', type=int, help='Disable IDLE-STATE on the selected CPUs.'),
+            "enable_idle_state": dict(dest='enable_idle_state', metavar='IDLE-STATE', type=int, help='Enable IDLE-STATE on the selected CPUs.')
+    }
 
     parser = HelpMessageParser(description="tuna - Application Tuning Program")
 
@@ -127,6 +132,9 @@ def gen_parser():
 
     subparser = parser.add_subparsers(dest='command')
 
+    idle_set = subparser.add_parser('idle_set',
+                                    description='Manage CPU idle state disabling (requires libcpupower and it\'s Python bindings)',
+                                    help='Set all idle states on a given CPU-LIST.')
     isolate = subparser.add_parser('isolate', description="Move all allowed threads and IRQs away from CPU-LIST",
                                     help="Move all allowed threads and IRQs away from CPU-LIST")
     include = subparser.add_parser('include', description="Allow all threads to run on CPU-LIST",
@@ -146,7 +154,6 @@ def gen_parser():
     show_threads = subparser.add_parser('show_threads', description='Show thread list', help='Show thread list')
     show_irqs = subparser.add_parser('show_irqs', description='Show IRQ list', help='Show IRQ list')
     show_configs = subparser.add_parser('show_configs', description='List preloaded profiles', help='List preloaded profiles')
-
     what_is = subparser.add_parser('what_is', description='Provides help about selected entities', help='Provides help about selected entities')
     gui = subparser.add_parser('gui', description="Start the GUI", help="Start the GUI")
 
@@ -218,6 +225,13 @@ def gen_parser():
     show_irqs_group.add_argument('-S', '--sockets', **MODS['sockets'])
     show_irqs.add_argument('-q', '--irqs', **MODS['irqs'])
 
+    idle_set_group = idle_set.add_mutually_exclusive_group(required=True)
+    idle_set_group.add_argument('-i', '--idle-info', **MODS['idle_info'])
+    idle_set_group.add_argument('-s', '--status', **MODS['idle_state_disabled_status'])
+    idle_set_group.add_argument('-d', '--disable', **MODS['disable_idle_state'])
+    idle_set_group.add_argument('-e', '--enable', **MODS['enable_idle_state'])
+    idle_set.add_argument('-c', '--cpus', **MODS['cpus'])
+
     what_is.add_argument('thread_list', **POS['thread_list'])
 
     gui.add_argument('-d', '--disable_perf', **MODS['disable_perf'])
@@ -647,6 +661,16 @@ def main():
             print("Valid log levels: NOTSET, DEBUG, INFO, WARNING, ERROR")
             print("Log levels may be specified numerically (0-4)\n")
 
+    if args.command == 'idle_set':
+        if not cpw.have_cpupower:
+            print(f"Error: libcpupower bindings are not detected; please install libcpupower bindings from at least kernel {cpw.cpupower_required_kernel}.")
+            sys.exit(1)
+
+        my_cpupower = cpw.Cpupower(args.cpu_list)
+        ret = my_cpupower.idle_set_handler(args)
+        if ret > 0:
+            sys.exit(ret)
+
     if 'irq_list' in vars(args):
         ps = procfs.pidstats()
         if tuna.has_threaded_irqs(ps):
tuna/cpupower.py new file +184 -0

184 lines not on this page · raw message

Signature · 1 line
2.48.1
#4 lore raw

Re: [PATCH v4 2/2] tuna: Add idle_state control functionality

Quoted 82 lines · John B. Wyatt IV

Allows Tuna to control cpu idle-state functionality on the system, including querying, enabling, disabling of cpu idle-states to control power usage or to test functionality.

This requires cpupower, a utility in the Linux kernel repository and the cpupower Python bindings added in Linux 6.12 to control cpu idle-states.

This patch revision includes text snippet suggestions by Crystal Wood (v2) and small Python suggestions by John Kacur (v3 and v4).

---
 tuna-cmd.py      |  30 +++++++-
 tuna/cpupower.py | 184 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 211 insertions(+), 3 deletions(-)
 create mode 100755 tuna/cpupower.py
tuna-cmd.py +17 -3
index d0323f5..96a25a5 100755
@@ -25,6 +25,7 @@ from tuna import tuna, sysfs, utils
 import logging
 import time
 import shutil
+import tuna.cpupower as cpw
 
 def get_loglevel(level):
     if level.isdigit() and int(level) in range(0,5):
@@ -115,8 +116,12 @@ def gen_parser():
             "disable_perf": dict(action='store_true', help="Explicitly disable usage of perf in GUI for process view"),
             "refresh": dict(default=2500, metavar='MSEC', type=int, help="Refresh the GUI every MSEC milliseconds"),
             "priority": dict(default=(None, None), metavar="POLICY:RTPRIO", type=tuna.get_policy_and_rtprio, help="Set thread scheduler tunables: POLICY and RTPRIO"),
-            "background": dict(action='store_true', help="Run command as background task")
-         }
+            "background": dict(action='store_true', help="Run command as background task"),
+            "idle_info": dict(dest='idle_info', action='store_const', const=True, help='Print general idle information for the selected CPUs, including index values for IDLE-STATE.'),
+            "idle_state_disabled_status": dict(dest='idle_state_disabled_status', metavar='IDLE-STATE', type=int, help='Print whether IDLE-STATE is enabled on the selected	CPUs.'),
+            "disable_idle_state": dict(dest='disable_idle_state', metavar='IDLE-STATE', type=int, help='Disable IDLE-STATE on the selected CPUs.'),
+            "enable_idle_state": dict(dest='enable_idle_state', metavar='IDLE-STATE', type=int, help='Enable IDLE-STATE on the selected CPUs.')
+    }
 
     parser = HelpMessageParser(description="tuna - Application Tuning Program")
 
@@ -127,6 +132,9 @@ def gen_parser():
 
     subparser = parser.add_subparsers(dest='command')
 
+    idle_set = subparser.add_parser('idle_set',
+                                    description='Manage CPU idle state disabling (requires libcpupower and it\'s Python bindings)',
+                                    help='Set all idle states on a given CPU-LIST.')
     isolate = subparser.add_parser('isolate', description="Move all allowed threads and IRQs away from CPU-LIST",
                                     help="Move all allowed threads and IRQs away from CPU-LIST")
     include = subparser.add_parser('include', description="Allow all threads to run on CPU-LIST",
@@ -146,7 +154,6 @@ def gen_parser():
     show_threads = subparser.add_parser('show_threads', description='Show thread list', help='Show thread list')
     show_irqs = subparser.add_parser('show_irqs', description='Show IRQ list', help='Show IRQ list')
     show_configs = subparser.add_parser('show_configs', description='List preloaded profiles', help='List preloaded profiles')
-
     what_is = subparser.add_parser('what_is', description='Provides help about selected entities', help='Provides help about selected entities')
     gui = subparser.add_parser('gui', description="Start the GUI", help="Start the GUI")
 
@@ -218,6 +225,13 @@ def gen_parser():
     show_irqs_group.add_argument('-S', '--sockets', **MODS['sockets'])
     show_irqs.add_argument('-q', '--irqs', **MODS['irqs'])
 
+    idle_set_group = idle_set.add_mutually_exclusive_group(required=True)
+    idle_set_group.add_argument('-i', '--idle-info', **MODS['idle_info'])
+    idle_set_group.add_argument('-s', '--status', **MODS['idle_state_disabled_status'])
+    idle_set_group.add_argument('-d', '--disable', **MODS['disable_idle_state'])
+    idle_set_group.add_argument('-e', '--enable', **MODS['enable_idle_state'])
+    idle_set.add_argument('-c', '--cpus', **MODS['cpus'])
+
     what_is.add_argument('thread_list', **POS['thread_list'])
 
     gui.add_argument('-d', '--disable_perf', **MODS['disable_perf'])

tuna-cmd.py def main(): · line 666

Quoted 6 lines
tuna-cmd.py +3 -0
@@ -647,6 +661,16 @@ def main():
             print("Valid log levels: NOTSET, DEBUG, INFO, WARNING, ERROR")
             print("Log levels may be specified numerically (0-4)\n")
 
+    if args.command == 'idle_set':
+        if not cpw.have_cpupower:
+            print(f"Error: libcpupower bindings are not detected; please install libcpupower bindings from at least kernel {cpw.cpupower_required_kernel}.")

Since it is an error message, print() should be called with "file=sys.stderr"

Quoted 20 lines
tuna-cmd.py +7 -0
+            sys.exit(1)
+
+        my_cpupower = cpw.Cpupower(args.cpu_list)
+        ret = my_cpupower.idle_set_handler(args)
+        if ret > 0:
+            sys.exit(ret)
+
     if 'irq_list' in vars(args):
         ps = procfs.pidstats()
         if tuna.has_threaded_irqs(ps):
tuna/cpupower.py new file +4 -0
new file mode 100755
index 0000000..7913027
@@ -0,0 +1,184 @@
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright (C) 2024 John B. Wyatt IV
+
+from typing import List

tuna/cpupower.py line 14

Quoted 10 lines
tuna/cpupower.py new file +10 -0
+import tuna.utils as utils
+
+cpupower_required_kernel = "6.12"
+have_cpupower = None
+
+try:
+    import raw_pylibcpupower as lcpw
+    lcpw.cpufreq_get_available_frequencies(0)
+    have_cpupower = True
+except:

Catch all statements in Python are bad IMO, because they catch syntax errors as well. We should at least print the exception.

Quoted 10 lines
tuna/cpupower.py new file +10 -0
+    lcpw = None
+    have_cpupower = False
+
+if have_cpupower:
+    class Cpupower:
+        """The Cpupower class allows you to query and change the power states of the
+        cpu.
+
+        You may query or change the cpus all at once or a list of the cpus provided to the constructor's cpulist argument.
+

tuna/cpupower.py line 34

Quoted 10 lines
tuna/cpupower.py new file +10 -0
+        The bindings must be detected on the $PYTHONPATH variable.
+
+        You must use have_cpupower variable to determine if the bindings were
+        detected in your code."""
+
+        LCPW_ERROR_TWO_CASE = 1 # enum for common error messages
+        LCPW_ERROR_THREE_CASE = 2
+
+        def __init__(self, cpu_list=None):
+            if cpu_list == None or cpu_list == []: # args.cpu_list == [] when the -c option is not used

Nit: use the "is" operator to compare against None.

Quoted 10 lines
tuna/cpupower.py new file +10 -0
+                self.__cpu_list = utils.get_all_cpu_list()
+            else:
+                self.__cpu_list = cpu_list
+
+        @staticmethod
+        def handle_common_lcpw_errors(e, error_type, idle_name):
+            match e:
+                case 0:
+                    pass
+                case -1:

tuna/cpupower.py line 54

Quoted 10 lines
tuna/cpupower.py new file +10 -0
+                    print(f"Idlestate {idle_name} not available")
+                case -2:
+                    print("Disabling is not supported by the kernel")
+                case -3:
+                    if error_type == Cpupower.LCPW_ERROR_THREE_CASE:
+                        print("No write access to disable/enable C-states: try using sudo")
+                    else:
+                        print(f"Not documented: {e}")
+                case _:
+                    print(f"Not documented: {e}")

Error message should go to stderr by adding "file=sys.stderr" to the print() statements.

Quoted 54 lines
tuna/cpupower.py new file +54 -0
+
+        @staticmethod
+        def get_idle_states(cpu):
+            """
+            Get the c-states of a cpu.
+
+            You can capture the return values with:
+            states_list, states_amt = get_idle_states()
+
+            Returns
+                List[String]: list of cstates
+                Int: amt of cstates
+            """
+            ret = []
+            for cstate in range(lcpw.cpuidle_state_count(cpu)):
+                ret.append(lcpw.cpuidle_state_name(cpu,cstate))
+            return ret, lcpw.cpuidle_state_count(cpu)
+
+        @staticmethod
+        def get_idle_info(cpu=0):
+            idle_states, idle_states_amt = Cpupower.get_idle_states(cpu)
+            idle_states_list = []
+            for idle_state in range(0, len(idle_states)):
+                idle_states_list.append(
+                    {
+                        "CPU ID": cpu,
+                        "Idle State Name": idle_states[idle_state],
+                        "Flags/Description": lcpw.cpuidle_state_desc(cpu, idle_state),
+                        "Latency": lcpw.cpuidle_state_latency(cpu, idle_state),
+                        "Usage": lcpw.cpuidle_state_usage(cpu, idle_state),
+                        "Duration": lcpw.cpuidle_state_time(cpu, idle_state)
+                    }
+                )
+            idle_info = {
+                "CPUidle-driver": lcpw.cpuidle_get_driver(),
+                "CPUidle-governor": lcpw.cpuidle_get_governor(),
+                "idle-states-count": idle_states_amt,
+                "available-idle-states": idle_states,
+                "cpu-states": idle_states_list
+            }
+            return idle_info
+
+        @staticmethod
+        def print_idle_info(cpu_list=[0]):
+            for cpu in cpu_list:
+                idle_info = Cpupower.get_idle_info(cpu)
+                print_str = f"""
+CPUidle driver: {idle_info["CPUidle-driver"]}
+CPUidle governor: {idle_info["CPUidle-governor"]}
+analyzing CPU {cpu}
+
+Number of idle states: {idle_info["idle-states-count"]}
+Available idle states: {idle_info["available-idle-states"]}
+"""

tuna/cpupower.py line 118

Quoted 10 lines
tuna/cpupower.py new file +10 -0
+                for state in idle_info["cpu-states"]:
+                    print_str += f"""{state["Idle State Name"]}
+Flags/Description: {state["Flags/Description"]}
+Latency: {state["Latency"]}
+Usage: {state["Usage"]}
+Duration: {state["Duration"]}
+"""
+                print(
+                    print_str
+                )

nit: I don't think we need three lines for this statement.

tuna/cpupower.py line 125

Quoted 7 lines
tuna/cpupower.py new file +7 -0
+
+        def idle_set_handler(self, args) -> int:
+            if args.idle_state_disabled_status != None:
+                cstate_index = args.idle_state_disabled_status
+                cstate_list, cstate_amt = self.get_idle_states(self.__cpu_list[0]) # Assuming all cpus have the same idle state
+                if cstate_index < 0 or cstate_index >= cstate_amt:
+                    print(f"Invalid idle state range. Total for this cpu is {cstate_amt}")

Nit: redirect error message to stderr.

tuna/cpupower.py line 136

Quoted 11 lines
tuna/cpupower.py new file +11 -0
+                    return 1
+                cstate_name = cstate_list[cstate_index]
+                ret = self.is_disabled_idle_state(cstate_index)
+                for i,e in enumerate(ret):
+                    if e == 1:
+                        print(f"CPU: {self.__cpu_list[i]} Idle state \"{cstate_name}\" is disabled.")
+                    elif e == 0:
+                        print(f"CPU: {self.__cpu_list[i]} Idle state \"{cstate_name}\" is enabled.")
+                    else:
+                        self.handle_common_lcpw_errors(e, self.LCPW_ERROR_TWO_CASE, cstate_name)
+            elif args.idle_info != None:

nit: Use the "is" operator to compare against None.

tuna/cpupower.py line 139

Quoted 3 lines
tuna/cpupower.py new file +3 -0
+                self.print_idle_info(self.__cpu_list)
+                return 0
+            elif args.disable_idle_state != None:

Ditto.

tuna/cpupower.py line 143

Quoted 4 lines
tuna/cpupower.py new file +4 -0
+                cstate_index = args.disable_idle_state
+                cstate_list, cstate_amt = self.get_idle_states(self.__cpu_list[0]) # Assuming all cpus have the same idle state
+                if cstate_index < 0 or cstate_index >= cstate_amt:
+                    print(f"Invalid idle state range. Total for this cpu is {cstate_amt}")

Nit: redirect error message to stderr.

tuna/cpupower.py line 149

Quoted 6 lines
tuna/cpupower.py new file +6 -0
+                    return 1
+                cstate_name = cstate_list[cstate_index]
+                ret = self.disable_idle_state(cstate_index, 1)
+                for e in ret:
+                    self.handle_common_lcpw_errors(e, self.LCPW_ERROR_THREE_CASE, cstate_name)
+            elif args.enable_idle_state != None:

nit: Use the "is" operator to compare against None.

tuna/cpupower.py line 153

Quoted 4 lines
tuna/cpupower.py new file +4 -0
+                cstate_index = args.enable_idle_state
+                cstate_list, cstate_amt = self.get_idle_states(self.__cpu_list[0]) # Assuming all cpus have the same idle state
+                if cstate_index < 0 or cstate_index >= cstate_amt:
+                    print(f"Invalid idle state range. Total for this cpu is {cstate_amt}")

nit: Use the "is" operator to compare against None.

Quoted 34 lines
tuna/cpupower.py new file +31 -0
+                    return 1
+                cstate_name = cstate_list[cstate_index]
+                ret = self.disable_idle_state(cstate_index, 0)
+                for e in ret:
+                    self.handle_common_lcpw_errors(e, self.LCPW_ERROR_THREE_CASE, cstate_name)
+            return 0
+
+        def disable_idle_state(self, state, disabled) -> List[int]:
+            """
+            Disable or enable an idle state using the object's stored list of cpus.
+
+            Args:
+                state (int): The cpu idle state index to disable or enable as an int starting from 0.
+                disabled (int): set to 1 to disable or 0 to enable. Less than 0 is an error.
+            """
+            ret = []
+            for cpu in self.__cpu_list:
+                ret.append(lcpw.cpuidle_state_disable(cpu, state, disabled))
+            return ret
+
+        def is_disabled_idle_state(self, state) -> List[int]:
+            """
+            Query the idle state.
+
+            Args:
+                state: The cpu idle state. 1 is disabled, 0 is enabled. Less than 0 is an error.
+            """
+            ret = []
+            for cpu in self.__cpu_list:
+                ret.append(lcpw.cpuidle_is_state_disabled(cpu, state))
+            return ret
Signature · 1 line
2.48.1
#5 lore raw

Re: [PATCH v4 2/2] tuna: Add idle_state control functionality

Quoted 5 lines · Wander Lairson Costa
Quoted 4 lines · John B. Wyatt IV

Allows Tuna to control cpu idle-state functionality on the system, including querying, enabling, disabling of cpu idle-states to control power usage or to test functionality.

Quoted 8 lines
Quoted 8 lines

This requires cpupower, a utility in the Linux kernel repository and the cpupower Python bindings added in Linux 6.12 to control cpu idle-states.

This patch revision includes text snippet suggestions by Crystal Wood (v2) and small Python suggestions by John Kacur (v3 and v4).

Eh, this tag makes it look like I suggested the overall idea, not that I gave feedback on the patch...

Quoted 9 lines

tuna-cmd.py def gen_parser(): · line 121

Quoted 9 lines
tuna-cmd.py +3 -2
@@ -115,8 +116,12 @@ def gen_parser():
             "disable_perf": dict(action='store_true', help="Explicitly disable usage of perf in GUI for process view"),
             "refresh": dict(default=2500, metavar='MSEC', type=int, help="Refresh the GUI every MSEC milliseconds"),
             "priority": dict(default=(None, None), metavar="POLICY:RTPRIO", type=tuna.get_policy_and_rtprio, help="Set thread scheduler tunables: POLICY and RTPRIO"),
-            "background": dict(action='store_true', help="Run command as background task")
-         }
+            "background": dict(action='store_true', help="Run command as background task"),
+            "idle_info": dict(dest='idle_info', action='store_const', const=True, help='Print general idle information for the selected CPUs, including index values for IDLE-STATE.'),
+            "idle_state_disabled_status": dict(dest='idle_state_disabled_status', metavar='IDLE-STATE', type=int, help='Print whether IDLE-STATE is enabled on the selected	CPUs.'),

You have a tab between "selected" and "CPUs".

Quoted 6 lines
Quoted 6 lines
tuna-cmd.py +3 -0
+            "disable_idle_state": dict(dest='disable_idle_state', metavar='IDLE-STATE', type=int, help='Disable IDLE-STATE on the selected CPUs.'),
+            "enable_idle_state": dict(dest='enable_idle_state', metavar='IDLE-STATE', type=int, help='Enable IDLE-STATE on the selected CPUs.')
+    }
 
     parser = HelpMessageParser(description="tuna - Application Tuning Program")
 
Quoted 7 lines

tuna-cmd.py def gen_parser(): · line 137

Quoted 7 lines
tuna-cmd.py +3 -0
@@ -127,6 +132,9 @@ def gen_parser():
 
     subparser = parser.add_subparsers(dest='command')
 
+    idle_set = subparser.add_parser('idle_set',
+                                    description='Manage CPU idle state disabling (requires libcpupower and it\'s Python bindings)',
+                                    help='Set all idle states on a given CPU-LIST.')

s/Set all idle states/Manage idle states/

I still don't like the "idle_set" name. This does more than setting, unlike the standalone utility that we're apparently reimplementing.

Quoted 15 lines
Quoted 15 lines
tuna/cpupower.py new file +9 -0
new file mode 100755
index 0000000..7913027
@@ -0,0 +1,184 @@
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright (C) 2024 John B. Wyatt IV
+
+from typing import List
+import tuna.utils as utils
+
+cpupower_required_kernel = "6.12"
+have_cpupower = None
+
Quoted 8 lines

tuna/cpupower.py line 14

Quoted 5 lines
tuna/cpupower.py new file +5 -0
+try:
+    import raw_pylibcpupower as lcpw
+    lcpw.cpufreq_get_available_frequencies(0)
+    have_cpupower = True
+except:

Catch all statements in Python are bad IMO, because they catch syntax errors as well. We should at least print the exception.

The (expected) exception is for the case where the package is not installed and thus we disable that feature. We don't want to print anything in that case (unless the user tries to use these features). But yeah, being more specific in the catch makes sense.

Quoted 4 lines

tuna/cpupower.py line 34

Quoted 2 lines
tuna/cpupower.py new file +2 -0
+        def __init__(self, cpu_list=None):
+            if cpu_list == None or cpu_list == []: # args.cpu_list == [] when the -c option is not used

Nit: use the "is" operator to compare against None.

Or just use "if cpu_list:" like the existing code does.

That's also pretty obnoxious behavior of the code that sets args.cpu_list...

Quoted 2 lines

tuna/cpupower.py line 18

Quoted 2 lines
tuna/cpupower.py new file +2 -0
+        @staticmethod
+        def print_idle_info(cpu_list=[0]):

This default is never used. cpu_list is only ever set to self.__cpu_list. Why is this a static method?

Quoted 14 lines
Quoted 14 lines
tuna/cpupower.py new file +14 -0
+            for cpu in cpu_list:
+                idle_info = Cpupower.get_idle_info(cpu)
+                print_str = f"""
+CPUidle driver: {idle_info["CPUidle-driver"]}
+CPUidle governor: {idle_info["CPUidle-governor"]}
+analyzing CPU {cpu}
+
+Number of idle states: {idle_info["idle-states-count"]}
+Available idle states: {idle_info["available-idle-states"]}
+"""
+                for state in idle_info["cpu-states"]:
+                    print_str += f"""{state["Idle State Name"]}
+Flags/Description: {state["Flags/Description"]}
+Latency: {state["Latency"]}
Quoted 8 lines

tuna/cpupower.py line 118

Quoted 6 lines
tuna/cpupower.py new file +6 -0
+Usage: {state["Usage"]}
+Duration: {state["Duration"]}
+"""
+                print(
+                    print_str
+                )

nit: I don't think we need three lines for this statement.

Why not just print each state inside the loop, instead of building one big string?

Quoted 16 lines

tuna/cpupower.py line 149

Quoted 1 line
tuna/cpupower.py new file +1 -0
+            elif args.enable_idle_state != None:

nit: Use the "is" operator to compare against None.

tuna/cpupower.py line 153

Quoted 4 lines
tuna/cpupower.py new file +4 -0
+                cstate_index = args.enable_idle_state
+                cstate_list, cstate_amt = self.get_idle_states(self.__cpu_list[0]) # Assuming all cpus have the same idle state
+                if cstate_index < 0 or cstate_index >= cstate_amt:
+                    print(f"Invalid idle state range. Total for this cpu is {cstate_amt}")

nit: Use the "is" operator to compare against None.

Quoted 5 lines
tuna/cpupower.py new file +5 -0
+                    return 1
+                cstate_name = cstate_list[cstate_index]
+                ret = self.disable_idle_state(cstate_index, 0)
+                for e in ret:
+                    self.handle_common_lcpw_errors(e, self.LCPW_ERROR_THREE_CASE, cstate_name)

Why isn't self.disable_idle_state() doing the error handling (which is still duplicated between enable and disable)? And as I mentioned before, isn't libcpupower itself doing the same range checking?

It looks like the get_idle_states() call is just for that redundant check, and to print the name in an error... You could wait until you actually get an error, and then just call lcpw.cpuidle_state_name on the particular cpu and state id. That would get rid of the assumption about all cpus having the same idle states, and be simpler.

-Crystal

#6 lore raw

Re: [PATCH v4 2/2] tuna: Add idle_state control functionality

Quoted 10 lines · Crystal Wood
Quoted 9 lines · Wander Lairson Costa
Quoted 8 lines · John B. Wyatt IV

Allows Tuna to control cpu idle-state functionality on the system, including querying, enabling, disabling of cpu idle-states to control power usage or to test functionality.

This requires cpupower, a utility in the Linux kernel repository and the cpupower Python bindings added in Linux 6.12 to control cpu idle-states.

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines

This patch revision includes text snippet suggestions by Crystal Wood (v2) and small Python suggestions by John Kacur (v3 and v4).

Eh, this tag makes it look like I suggested the overall idea, not that I gave feedback on the patch...

Good point, I can remove it.

Quoted 12 lines
Quoted 10 lines

tuna-cmd.py def gen_parser(): · line 121

Quoted 10 lines
tuna-cmd.py +3 -2
@@ -115,8 +116,12 @@ def gen_parser():
             "disable_perf": dict(action='store_true', help="Explicitly disable usage of perf in GUI for process view"),
             "refresh": dict(default=2500, metavar='MSEC', type=int, help="Refresh the GUI every MSEC milliseconds"),
             "priority": dict(default=(None, None), metavar="POLICY:RTPRIO", type=tuna.get_policy_and_rtprio, help="Set thread scheduler tunables: POLICY and RTPRIO"),
-            "background": dict(action='store_true', help="Run command as background task")
-         }
+            "background": dict(action='store_true', help="Run command as background task"),
+            "idle_info": dict(dest='idle_info', action='store_const', const=True, help='Print general idle information for the selected CPUs, including index values for IDLE-STATE.'),
+            "idle_state_disabled_status": dict(dest='idle_state_disabled_status', metavar='IDLE-STATE', type=int, help='Print whether IDLE-STATE is enabled on the selected	CPUs.'),

You have a tab between "selected" and "CPUs".

Nice catch, thanks

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
tuna-cmd.py +3 -0
+            "disable_idle_state": dict(dest='disable_idle_state', metavar='IDLE-STATE', type=int, help='Disable IDLE-STATE on the selected CPUs.'),
+            "enable_idle_state": dict(dest='enable_idle_state', metavar='IDLE-STATE', type=int, help='Enable IDLE-STATE on the selected CPUs.')
+    }
 
     parser = HelpMessageParser(description="tuna - Application Tuning Program")
 
@@ -127,6 +132,9 @@ def gen_parser():
 
     subparser = parser.add_subparsers(dest='command')
 
Quoted 8 lines
Quoted 3 lines

tuna-cmd.py def gen_parser(): · line 137

Quoted 3 lines
tuna-cmd.py +3 -0
+    idle_set = subparser.add_parser('idle_set',
+                                    description='Manage CPU idle state disabling (requires libcpupower and it\'s Python bindings)',
+                                    help='Set all idle states on a given CPU-LIST.')

s/Set all idle states/Manage idle states/

I still don't like the "idle_set" name. This does more than setting, unlike the standalone utility that we're apparently reimplementing.

I agree that idle_set is not the ideal name since if you look at the cpupower tool, this is just one function, maybe cpupower would be a better name. It's a bit unfair to say we're reimplementing the tool. The functionality is in upstream kernel code and John Wyatt created python bindings for that, and that is what he is using here. I think it is better to have all the tools for manipulating the environment through one tool and not make the user go search for multiple tools to get the job done.

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
tuna/cpupower.py new file +9 -0
new file mode 100755
index 0000000..7913027
@@ -0,0 +1,184 @@
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright (C) 2024 John B. Wyatt IV
+
+from typing import List
+import tuna.utils as utils
+
+cpupower_required_kernel = "6.12"
+have_cpupower = None
+
Quoted 8 lines
Quoted 8 lines

tuna/cpupower.py line 14

Quoted 5 lines
tuna/cpupower.py new file +5 -0
+try:
+    import raw_pylibcpupower as lcpw
+    lcpw.cpufreq_get_available_frequencies(0)
+    have_cpupower = True
+except:

Catch all statements in Python are bad IMO, because they catch syntax errors as well. We should at least print the exception.

Yes, that could be improved

Quoted 10 lines

The (expected) exception is for the case where the package is not installed and thus we disable that feature. We don't want to print anything in that case (unless the user tries to use these features). But yeah, being more specific in the catch makes sense.

Quoted 4 lines

tuna/cpupower.py line 34

Quoted 2 lines
tuna/cpupower.py new file +2 -0
+        def __init__(self, cpu_list=None):
+            if cpu_list == None or cpu_list == []: # args.cpu_list == [] when the -c option is not used

Nit: use the "is" operator to compare against None.

These nits from a previous message are valid, but of course not critical, they can be fixed in subsequent patches.

Quoted 11 lines

Or just use "if cpu_list:" like the existing code does.

That's also pretty obnoxious behavior of the code that sets args.cpu_list...

Quoted 2 lines

tuna/cpupower.py line 18

Quoted 2 lines
tuna/cpupower.py new file +2 -0
+        @staticmethod
+        def print_idle_info(cpu_list=[0]):

This default is never used. cpu_list is only ever set to self.__cpu_list. Why is this a static method?

This is a different cpu_list than the instantiated one. This is which cpus the user wants the info from. I think the name is confusing here.

Quoted 24 lines
Quoted 15 lines
Quoted 15 lines
tuna/cpupower.py new file +14 -0
+            for cpu in cpu_list:
+                idle_info = Cpupower.get_idle_info(cpu)
+                print_str = f"""
+CPUidle driver: {idle_info["CPUidle-driver"]}
+CPUidle governor: {idle_info["CPUidle-governor"]}
+analyzing CPU {cpu}
+
+Number of idle states: {idle_info["idle-states-count"]}
+Available idle states: {idle_info["available-idle-states"]}
+"""
+                for state in idle_info["cpu-states"]:
+                    print_str += f"""{state["Idle State Name"]}
+Flags/Description: {state["Flags/Description"]}
+Latency: {state["Latency"]}
Quoted 8 lines

tuna/cpupower.py line 118

Quoted 6 lines
tuna/cpupower.py new file +6 -0
+Usage: {state["Usage"]}
+Duration: {state["Duration"]}
+"""
+                print(
+                    print_str
+                )

nit: I don't think we need three lines for this statement.

Quoted 2 lines

Why not just print each state inside the loop, instead of building one big string?

valid, but I think John Wyatt was thinking the string could be useful for other purposes.

Quoted 22 lines
Quoted 17 lines

tuna/cpupower.py line 149

Quoted 2 lines
tuna/cpupower.py new file +1 -0
+            elif args.enable_idle_state != None:

nit: Use the "is" operator to compare against None.

tuna/cpupower.py line 153

Quoted 4 lines
tuna/cpupower.py new file +4 -0
+                cstate_index = args.enable_idle_state
+                cstate_list, cstate_amt = self.get_idle_states(self.__cpu_list[0]) # Assuming all cpus have the same idle state
+                if cstate_index < 0 or cstate_index >= cstate_amt:
+                    print(f"Invalid idle state range. Total for this cpu is {cstate_amt}")

nit: Use the "is" operator to compare against None.

Quoted 5 lines
tuna/cpupower.py new file +5 -0
+                    return 1
+                cstate_name = cstate_list[cstate_index]
+                ret = self.disable_idle_state(cstate_index, 0)
+                for e in ret:
+                    self.handle_common_lcpw_errors(e, self.LCPW_ERROR_THREE_CASE, cstate_name)

Why isn't self.disable_idle_state() doing the error handling (which is still duplicated between enable and disable)? And as I mentioned before, isn't libcpupower itself doing the same range checking?

Quoted 8 lines

It looks like the get_idle_states() call is just for that redundant check, and to print the name in an error... You could wait until you actually get an error, and then just call lcpw.cpuidle_state_name on the particular cpu and state id. That would get rid of the assumption about all cpus having the same idle states, and be simpler.

-Crystal

The code is functioning, it can be improved, let's fix a few more problems and let it in, and then it can be improved incrementally after that.

John Kacur

#7 lore raw

Re: [PATCH v4 2/2] tuna: Add idle_state control functionality

Quoted 11 lines · John Kacur
Quoted 8 lines · Crystal Wood
Quoted 7 lines · Wander Lairson Costa
Quoted 6 lines · John B. Wyatt IV
tuna-cmd.py +3 -0
+            "disable_idle_state": dict(dest='disable_idle_state', metavar='IDLE-STATE', type=int, help='Disable IDLE-STATE on the selected CPUs.'),
+            "enable_idle_state": dict(dest='enable_idle_state', metavar='IDLE-STATE', type=int, help='Enable IDLE-STATE on the selected CPUs.')
+    }
 
     parser = HelpMessageParser(description="tuna - Application Tuning Program")
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

tuna-cmd.py def gen_parser(): · line 137

Quoted 7 lines
tuna-cmd.py +3 -0
@@ -127,6 +132,9 @@ def gen_parser():
 
     subparser = parser.add_subparsers(dest='command')
 
+    idle_set = subparser.add_parser('idle_set',
+                                    description='Manage CPU idle state disabling (requires libcpupower and it\'s Python bindings)',
+                                    help='Set all idle states on a given CPU-LIST.')

s/it\'s/its/

Quoted 11 lines
Quoted 5 lines

s/Set all idle states/Manage idle states/

I still don't like the "idle_set" name. This does more than setting, unlike the standalone utility that we're apparently reimplementing.

I agree that idle_set is not the ideal name since if you look at the cpupower tool, this is just one function, maybe cpupower would be a better name. It's a bit unfair to say we're reimplementing the tool. The functionality is in upstream kernel code and John Wyatt created python bindings for that, and that is what he is using here.

But (as of now) this isn't using the python bindings to implement something higher-level; it's exposing them as a command line tool that has very similar functionality to the existing one. I'm not sure what being written in Python has to do with it, other than a motive for reimplementing, to make future features easier.

Quoted 3 lines

I think it is better to have all the tools for manipulating the environment through one tool and not make the user go search for multiple tools to get the job done.

"all the tools for manipulating the environment" is an extremely broad mission... :-P

So far it doesn't seem to align very well with the rest of what tuna does, which is managing processes/threads... is there a plan to tie them together so that you can have it set idle states based on what the tasks running on a given cpu need, and/or move tasks around based on that?

Quoted 8 lines
Quoted 8 lines

Or just use "if cpu_list:" like the existing code does.

That's also pretty obnoxious behavior of the code that sets args.cpu_list...

Quoted 2 lines
Quoted 2 lines
(quoted diff) +2 -0
+        @staticmethod
+        def print_idle_info(cpu_list=[0]):
Quoted 6 lines
Quoted 2 lines

This default is never used. cpu_list is only ever set to self.__cpu_list. Why is this a static method?

This is a different cpu_list than the instantiated one. This is which cpus the user wants the info from. I think the name is confusing here.

The only call (so far...) is literally "self.print_idle_info(self.__cpu_list)".

Where are you seeing a different cpu list? When (even with future patches) would the instantiated cpu list be anything other than what the user asked for, and does that say anything about whether the class design makes sense?

Quoted 23 lines
Quoted 23 lines
Quoted 22 lines
Quoted 10 lines
tuna/cpupower.py new file +10 -0
+            for cpu in cpu_list:
+                idle_info = Cpupower.get_idle_info(cpu)
+                print_str = f"""
+CPUidle driver: {idle_info["CPUidle-driver"]}
+CPUidle governor: {idle_info["CPUidle-governor"]}
+analyzing CPU {cpu}
+
+Number of idle states: {idle_info["idle-states-count"]}
+Available idle states: {idle_info["available-idle-states"]}
+"""

tuna/cpupower.py line 118

Quoted 10 lines
tuna/cpupower.py new file +10 -0
+                for state in idle_info["cpu-states"]:
+                    print_str += f"""{state["Idle State Name"]}
+Flags/Description: {state["Flags/Description"]}
+Latency: {state["Latency"]}
+Usage: {state["Usage"]}
+Duration: {state["Duration"]}
+"""
+                print(
+                    print_str
+                )

nit: I don't think we need three lines for this statement.

Quoted 5 lines
Quoted 2 lines

Why not just print each state inside the loop, instead of building one big string?

valid, but I think John Wyatt was thinking the string could be useful for other purposes.

The json stuff I assume? Is this even in the right format for that? Is there any plan to json-ize other tuna functionality?

It's really hard to review patches when there are vague or unstated notions of "this could be useful in the future"... especially when recent experience with rteval has highlighted the importance of avoiding overengineering.

-Crystal

#8 lore raw

Re: [PATCH v4 2/2] tuna: Add idle_state control functionality

Quoted 34 lines · Crystal Wood
Quoted 11 lines · John Kacur
Quoted 8 lines · Crystal Wood
Quoted 7 lines · Wander Lairson Costa
Quoted 6 lines · John B. Wyatt IV
tuna-cmd.py +3 -0
+            "disable_idle_state": dict(dest='disable_idle_state', metavar='IDLE-STATE', type=int, help='Disable IDLE-STATE on the selected CPUs.'),
+            "enable_idle_state": dict(dest='enable_idle_state', metavar='IDLE-STATE', type=int, help='Enable IDLE-STATE on the selected CPUs.')
+    }
 
     parser = HelpMessageParser(description="tuna - Application Tuning Program")
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

tuna-cmd.py def gen_parser(): · line 137

Quoted 7 lines
tuna-cmd.py +3 -0
@@ -127,6 +132,9 @@ def gen_parser():
 
     subparser = parser.add_subparsers(dest='command')
 
+    idle_set = subparser.add_parser('idle_set',
+                                    description='Manage CPU idle state disabling (requires libcpupower and it\'s Python bindings)',
+                                    help='Set all idle states on a given CPU-LIST.')

s/it\'s/its/

Quoted 11 lines
Quoted 5 lines

s/Set all idle states/Manage idle states/

I still don't like the "idle_set" name. This does more than setting, unlike the standalone utility that we're apparently reimplementing.

I agree that idle_set is not the ideal name since if you look at the cpupower tool, this is just one function, maybe cpupower would be a better name. It's a bit unfair to say we're reimplementing the tool. The functionality is in upstream kernel code and John Wyatt created python bindings for that, and that is what he is using here.

Quoted 5 lines

But (as of now) this isn't using the python bindings to implement something higher-level; it's exposing them as a command line tool that has very similar functionality to the existing one. I'm not sure what being written in Python has to do with it, other than a motive for reimplementing, to make future features easier.

People seem to be interested in testing rt with some power savings enabled, so we're trying to make it easier for people to do so with the tools they are used to using. I'm not really open to debating this.

Quoted 7 lines
Quoted 4 lines

I think it is better to have all the tools for manipulating the environment through one tool and not make the user go search for multiple tools to get the job done.

"all the tools for manipulating the environment" is an extremely broad mission... :-P

:) how about all the tools for manipulating the environment that I think are important.

Quoted 18 lines

So far it doesn't seem to align very well with the rest of what tuna does, which is managing processes/threads... is there a plan to tie them together so that you can have it set idle states based on what the tasks running on a given cpu need, and/or move tasks around based on that?

Quoted 11 lines
Quoted 10 lines

Or just use "if cpu_list:" like the existing code does.

That's also pretty obnoxious behavior of the code that sets args.cpu_list...

Quoted 2 lines

Review def gen_parser(): · line 139

Quoted 2 lines
(quoted diff) +2 -0
+        @staticmethod
+        def print_idle_info(cpu_list=[0]):

This default is never used. cpu_list is only ever set to self.__cpu_list. Why is this a static method?

Quoted 7 lines
Quoted 3 lines

This is a different cpu_list than the instantiated one. This is which cpus the user wants the info from. I think the name is confusing here.

The only call (so far...) is literally "self.print_idle_info(self.__cpu_list)".

Where are you seeing a different cpu list? When (even with future

All I meant is that self.__cpu_list is not the same as cpu_list. Since the names are confusingly similar, if a different variable really is needed, it would be best to rename the one local to the method / function.

Quoted 27 lines

patches) would the instantiated cpu list be anything other than what the user asked for, and does that say anything about whether the class design makes sense?

Quoted 23 lines
Quoted 23 lines
Quoted 22 lines
Quoted 10 lines
tuna/cpupower.py new file +10 -0
+            for cpu in cpu_list:
+                idle_info = Cpupower.get_idle_info(cpu)
+                print_str = f"""
+CPUidle driver: {idle_info["CPUidle-driver"]}
+CPUidle governor: {idle_info["CPUidle-governor"]}
+analyzing CPU {cpu}
+
+Number of idle states: {idle_info["idle-states-count"]}
+Available idle states: {idle_info["available-idle-states"]}
+"""

tuna/cpupower.py line 118

Quoted 10 lines
tuna/cpupower.py new file +10 -0
+                for state in idle_info["cpu-states"]:
+                    print_str += f"""{state["Idle State Name"]}
+Flags/Description: {state["Flags/Description"]}
+Latency: {state["Latency"]}
+Usage: {state["Usage"]}
+Duration: {state["Duration"]}
+"""
+                print(
+                    print_str
+                )

nit: I don't think we need three lines for this statement.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Why not just print each state inside the loop, instead of building one big string?

valid, but I think John Wyatt was thinking the string could be useful for other purposes.

The json stuff I assume? Is this even in the right format for that? Is there any plan to json-ize other tuna functionality?

No, there is no play to json anything in tuna. Look, agreed, the string is messy, we'll fix it, I'm just saying he probably had something else in mind when he coded it.

Quoted 5 lines

It's really hard to review patches when there are vague or unstated notions of "this could be useful in the future"... especially when recent experience with rteval has highlighted the importance of avoiding overengineering.

rteval doesn't have anything to do with this, the original coders were completely different people. However, rteval has stood the test of time. I don't like the multiple layers of inheritance, but sometimes I find suprising benefits to the original design. Anyway, you can discuss that with me separately, that doesn't have anything to do with this.

What we are talking about here, is a single file that wraps the code in a class, this is fine, it often results in cleaner code even when object oriented features aren't used a lot.

Your suggestions are fine, now, give him some time to implement them.

John Kacur