Skip to contentSkip to conversation

extend interactive telemetry script

PATCH Superseded Bruce Richardson 3 participants 20 messages May 21 – Jul 12

lore
Share & tools
lore
#0 lore raw

cover extend interactive telemetry script

To simplify interactive telemetry script for general use, i.e. not from other scripts, we can add two new features to it:

1. Support for FOREACH to allow gathering a set of output values across a list of ports or devices, e.g. ethdevs or rawdevs.
2. Support having predefined aliases in a file in the user's home directory to simplify the use of more complicated FOREACH commands.

Putting these together, we can create new commands such as "eth_names".

  bruce@host:$ cat ~/.dpdk_telemetry_aliases
  eth_names=FOREACH index /ethdev/list /ethdev/info,$index .name
  bruce@host:$ echo eth_names | ./usertools/dpdk-telemetry.py | jq
  [
    {
      "index": 0,
      "name": "0000:16:00.0"
    },
    {
      "index": 1,
      "name": "0000:16:00.1"
    }
  ]

Bruce Richardson (2):
usertools/telemetry: add a FOREACH command usertools/telemetry: support using aliases for long commands

 doc/guides/howto/telemetry.rst |  76 +++++++++++++
 usertools/dpdk-telemetry.py    | 201 ++++++++++++++++++++++++++++++++-
 2 files changed, 271 insertions(+), 6 deletions(-)

--
2.53.0

#1 lore raw

PATCH 1/2 usertools/telemetry: add a FOREACH command

To simplify querying data from multiple devices, e.g. across ethdevs, or dmadevs, add a FOREACH command to the python script, allowing you to run, e.g. /ethdev/list, and then run a second command for each item in the list, gathering the relevant output values, optionally including an index counter.

Simple examples are given in the documentation:

--> FOREACH /ethdev/list /ethdev/stats .opackets [0, 0]

--> FOREACH /ethdev/list /ethdev/stats .ipackets .opackets [{"ipackets": 0, "opackets": 0}, {"ipackets": 0, "opackets": 0}]

--> FOREACH i /ethdev/list /ethdev/info,$i .name
[{"i": 0, "name": "0000:16:00.0"}, {"i": 1, "name": "0000:16:00.1"}]

--> FOREACH i /ethdev/list /ethdev/stats,$i .ipackets .opackets [{"i": 0, "ipackets": 0, "opackets": 0}, {"i": 1, "ipackets": 0, "opackets": 0}]

---
 doc/guides/howto/telemetry.rst |  42 +++++++++++
 usertools/dpdk-telemetry.py    | 128 ++++++++++++++++++++++++++++++++-
 2 files changed, 167 insertions(+), 3 deletions(-)
doc/guides/howto/telemetry.rst +42 -0
index 0464c431fe..4bf48c635e 100644
@@ -88,6 +88,48 @@ and query information using the telemetry client python script.
        {"/help": {"/ethdev/xstats": "Returns the extended stats for a port.
        Parameters: int port_id"}}
 
+   * Run a compound query using ``FOREACH``.
+
+     The ``FOREACH`` command runs a list command, iterates each returned item,
+     runs a second command for each item, and emits combined JSON output.
+
+     Start with the simplest form (no loop variable)::
+
+        FOREACH /<list_cmd> /<iter_cmd> .<field> [.<field> ...]
+
+     To include numbered output, use a loop variable::
+
+        FOREACH <var> /<list_cmd> /<iter_cmd_with_$var> .<field> [.<field> ...]
+
+     Notes:
+
+     - Field selectors are whitespace-separated tokens, each starting with ``.``.
+     - In no-variable mode, the iter command is called as ``/<iter_cmd>,<item>``.
+     - In loop-variable mode, use ``$<var>`` in the iter command where the
+       item value should be substituted.
+
+     Examples::
+
+        --> FOREACH /ethdev/list /ethdev/stats .opackets
+        [0, 0]
+
+        --> FOREACH /ethdev/list /ethdev/stats .ipackets .opackets
+        [{"ipackets": 0, "opackets": 0}, {"ipackets": 0, "opackets": 0}]
+
+        --> FOREACH i /ethdev/list /ethdev/info,$i .name
+        [{"i": 0, "name": "0000:16:00.0"}, {"i": 1, "name": "0000:16:00.1"}]
+
+        --> FOREACH i /ethdev/list /ethdev/stats,$i .ipackets .opackets
+        [{"i": 0, "ipackets": 0, "opackets": 0}, {"i": 1, "ipackets": 0, "opackets": 0}]
+
+     Output behavior:
+
+     - Without loop variable and one field: returns an array of values.
+     - Without loop variable and multiple fields: returns an array of objects
+       containing named value fields.
+     - With loop variable: returns an array of objects containing the loop
+       variable field and requested value fields.
+
 
 Connecting to Different DPDK Processes
 --------------------------------------
usertools/dpdk-telemetry.py +125 -3

140 lines not on this page · raw message

Signature · 1 line
2.53.0
#2 lore raw

PATCH 2/2 usertools/telemetry: support using aliases for long commands

In reply to Bruce Richardson

Similarly to how shell aliases work, allow specifying of short alias commands for dpdk-telemetry.py script. The aliases are read from "$HOME/.dpdk_telemetry_aliases" at startup.

Some examples of use from the docs. Alias file contents:

# Basic shortcuts
ls=/ethdev/list
names=FOREACH i /ethdev/list /ethdev/info,$i .name q=quit

Alias use:

--> ls
{"/ethdev/list": [0, 1]}

--> names
[{"i": 0, "name": "0000:

---
 doc/guides/howto/telemetry.rst | 34 +++++++++++++++
 usertools/dpdk-telemetry.py    | 75 ++++++++++++++++++++++++++++++++--
 2 files changed, 105 insertions(+), 4 deletions(-)
doc/guides/howto/telemetry.rst +34 -0
index 4bf48c635e..072289aec8 100644
@@ -130,6 +130,40 @@ and query information using the telemetry client python script.
      - With loop variable: returns an array of objects containing the loop
        variable field and requested value fields.
 
+   * Use command aliases.
+
+     The telemetry script can load aliases at startup from::
+
+        $HOME/.dpdk_telemetry_aliases
+
+     Each alias entry must be in ``alias=command`` format.
+     Empty lines and lines starting with ``#`` are ignored.
+
+     Example alias file::
+
+        # Basic shortcuts
+        ls=/ethdev/list
+        names=FOREACH i /ethdev/list /ethdev/info,$i .name
+        q=quit
+
+     Alias behavior is intentionally similar to shell aliases:
+
+     - The first token of the entered input is checked for an alias match.
+     - If matched, that first token is replaced with its expansion.
+     - Alias expansion is recursive (aliases can expand to other aliases).
+     - Expansion has a safety limit to prevent infinite loops.
+
+     Examples::
+
+        --> ls
+        {"/ethdev/list": [0, 1]}
+
+        --> names
+        [{"i": 0, "name": "0000:16:00.0"}, {"i": 1, "name": "0000:16:00.1"}]
+
+        --> q
+        # exits the client
+
 
 Connecting to Different DPDK Processes
 --------------------------------------
usertools/dpdk-telemetry.py +71 -4

100 lines not on this page · raw message

Signature · 1 line
2.53.0
#3 lore raw
In reply to Bruce Richardson
Quoted 7 lines · Bruce Richardson

To simplify interactive telemetry script for general use, i.e. not from other scripts, we can add two new features to it:

1. Support for FOREACH to allow gathering a set of output values across a list of ports or devices, e.g. ethdevs or rawdevs.
2. Support having predefined aliases in a file in the user's home directory to simplify the use of more complicated FOREACH commands.

+1
How about add help command to introducing above FOREACH and aliases?

Quoted 29 lines

Putting these together, we can create new commands such as "eth_names".

  bruce@host:$ cat ~/.dpdk_telemetry_aliases
  eth_names=FOREACH index /ethdev/list /ethdev/info,$index .name
  bruce@host:$ echo eth_names | ./usertools/dpdk-telemetry.py | jq
  [
    {
      "index": 0,
      "name": "0000:16:00.0"
    },
    {
      "index": 1,
      "name": "0000:16:00.1"
    }
  ]

Bruce Richardson (2):
usertools/telemetry: add a FOREACH command usertools/telemetry: support using aliases for long commands

 doc/guides/howto/telemetry.rst |  76 +++++++++++++
 usertools/dpdk-telemetry.py    | 201 ++++++++++++++++++++++++++++++++-
 2 files changed, 271 insertions(+), 6 deletions(-)

--
2.53.0

#4 lore raw
Quoted 4 lines · fengchengwen
Quoted 3 lines · Bruce Richardson

To simplify interactive telemetry script for general use, i.e. not from other scripts, we can add two new features to it:

Quoted 8 lines
Quoted 4 lines

1. Support for FOREACH to allow gathering a set of output values across a list of ports or devices, e.g. ethdevs or rawdevs.
2. Support having predefined aliases in a file in the user's home directory to simplify the use of more complicated FOREACH commands.

+1
How about add help command to introducing above FOREACH and aliases?

Sure, not a bad idea if we are extending the script to make it more powerful. Will add in a V2.

/Bruce

#5 lore raw

v2 cover extend interactive telemetry script

In reply to Bruce Richardson

To simplify interactive telemetry script for general use, i.e. not from other scripts, we can add two new features to it:

1. Support for FOREACH to allow gathering a set of output values across a list of ports or devices, e.g. ethdevs or rawdevs.
2. Support having predefined aliases in a file in the user's home directory to simplify the use of more complicated FOREACH commands.

Putting these together, we can create new commands such as "eth_names".

  bruce@host:$ cat ~/.dpdk_telemetry_aliases
  eth_names=FOREACH index /ethdev/list /ethdev/info,$index .name
  bruce@host:$ echo eth_names | ./usertools/dpdk-telemetry.py | jq
  [
    {
      "index": 0,
      "name": "0000:16:00.0"
    },
    {
      "index": 1,
      "name": "0000:16:00.1"
    }
  ]

---
v2: added third patch with "help" command giving more details on
how to use the various commands.

Bruce Richardson (3):
usertools/telemetry: add a FOREACH command usertools/telemetry: support using aliases for long commands usertools/telemetry: add help support

 doc/guides/howto/telemetry.rst |  76 ++++++++++
 usertools/dpdk-telemetry.py    | 260 ++++++++++++++++++++++++++++++++-
 2 files changed, 330 insertions(+), 6 deletions(-)

--
2.53.0

#6 lore raw

PATCH v2 1/3 usertools/telemetry: add a FOREACH command

To simplify querying data from multiple devices, e.g. across ethdevs, or dmadevs, add a FOREACH command to the python script, allowing you to run, e.g. /ethdev/list, and then run a second command for each item in the list, gathering the relevant output values, optionally including an index counter.

Simple examples are given in the documentation:

--> FOREACH /ethdev/list /ethdev/stats .opackets [0, 0]

--> FOREACH /ethdev/list /ethdev/stats .ipackets .opackets [{"ipackets": 0, "opackets": 0}, {"ipackets": 0, "opackets": 0}]

--> FOREACH i /ethdev/list /ethdev/info,$i .name
[{"i": 0, "name": "0000:16:00.0"}, {"i": 1, "name": "0000:16:00.1"}]

--> FOREACH i /ethdev/list /ethdev/stats,$i .ipackets .opackets [{"i": 0, "ipackets": 0, "opackets": 0}, {"i": 1, "ipackets": 0, "opackets": 0}]

---
 doc/guides/howto/telemetry.rst |  42 +++++++++++
 usertools/dpdk-telemetry.py    | 128 ++++++++++++++++++++++++++++++++-
 2 files changed, 167 insertions(+), 3 deletions(-)
doc/guides/howto/telemetry.rst +42 -0
index 0464c431fe..4bf48c635e 100644
@@ -88,6 +88,48 @@ and query information using the telemetry client python script.
        {"/help": {"/ethdev/xstats": "Returns the extended stats for a port.
        Parameters: int port_id"}}
 
+   * Run a compound query using ``FOREACH``.
+
+     The ``FOREACH`` command runs a list command, iterates each returned item,
+     runs a second command for each item, and emits combined JSON output.
+
+     Start with the simplest form (no loop variable)::
+
+        FOREACH /<list_cmd> /<iter_cmd> .<field> [.<field> ...]
+
+     To include numbered output, use a loop variable::
+
+        FOREACH <var> /<list_cmd> /<iter_cmd_with_$var> .<field> [.<field> ...]
+
+     Notes:
+
+     - Field selectors are whitespace-separated tokens, each starting with ``.``.
+     - In no-variable mode, the iter command is called as ``/<iter_cmd>,<item>``.
+     - In loop-variable mode, use ``$<var>`` in the iter command where the
+       item value should be substituted.
+
+     Examples::
+
+        --> FOREACH /ethdev/list /ethdev/stats .opackets
+        [0, 0]
+
+        --> FOREACH /ethdev/list /ethdev/stats .ipackets .opackets
+        [{"ipackets": 0, "opackets": 0}, {"ipackets": 0, "opackets": 0}]
+
+        --> FOREACH i /ethdev/list /ethdev/info,$i .name
+        [{"i": 0, "name": "0000:16:00.0"}, {"i": 1, "name": "0000:16:00.1"}]
+
+        --> FOREACH i /ethdev/list /ethdev/stats,$i .ipackets .opackets
+        [{"i": 0, "ipackets": 0, "opackets": 0}, {"i": 1, "ipackets": 0, "opackets": 0}]
+
+     Output behavior:
+
+     - Without loop variable and one field: returns an array of values.
+     - Without loop variable and multiple fields: returns an array of objects
+       containing named value fields.
+     - With loop variable: returns an array of objects containing the loop
+       variable field and requested value fields.
+
 
 Connecting to Different DPDK Processes
 --------------------------------------
usertools/dpdk-telemetry.py +125 -3

140 lines not on this page · raw message

Signature · 1 line
2.53.0
Acked-by on v2 1/3
#7 lore raw

Re: [PATCH v2 1/3] usertools/telemetry: add a FOREACH command

Quoted 21 lines · Bruce Richardson

To simplify querying data from multiple devices, e.g. across ethdevs, or dmadevs, add a FOREACH command to the python script, allowing you to run, e.g. /ethdev/list, and then run a second command for each item in the list, gathering the relevant output values, optionally including an index counter.

Simple examples are given in the documentation:

--> FOREACH /ethdev/list /ethdev/stats .opackets [0, 0]

--> FOREACH /ethdev/list /ethdev/stats .ipackets .opackets [{"ipackets": 0, "opackets": 0}, {"ipackets": 0, "opackets": 0}]

--> FOREACH i /ethdev/list /ethdev/info,$i .name
[{"i": 0, "name": "0000:16:00.0"}, {"i": 1, "name": "0000:16:00.1"}]

--> FOREACH i /ethdev/list /ethdev/stats,$i .ipackets .opackets [{"i": 0, "ipackets": 0, "opackets": 0}, {"i": 1, "ipackets": 0, "opackets": 0}]

#8 lore raw

PATCH v2 2/3 usertools/telemetry: support using aliases for long commands

In reply to Bruce Richardson

Similarly to how shell aliases work, allow specifying of short alias commands for dpdk-telemetry.py script. The aliases are read from "$HOME/.dpdk_telemetry_aliases" at startup.

Some examples of use from the docs. Alias file contents:

# Basic shortcuts
ls=/ethdev/list
names=FOREACH i /ethdev/list /ethdev/info,$i .name q=quit

Alias use:

--> ls
{"/ethdev/list": [0, 1]}

--> names
[{"i": 0, "name": "0000:

---
 doc/guides/howto/telemetry.rst | 34 +++++++++++++++
 usertools/dpdk-telemetry.py    | 75 ++++++++++++++++++++++++++++++++--
 2 files changed, 105 insertions(+), 4 deletions(-)
doc/guides/howto/telemetry.rst +34 -0
index 4bf48c635e..072289aec8 100644
@@ -130,6 +130,40 @@ and query information using the telemetry client python script.
      - With loop variable: returns an array of objects containing the loop
        variable field and requested value fields.
 
+   * Use command aliases.
+
+     The telemetry script can load aliases at startup from::
+
+        $HOME/.dpdk_telemetry_aliases
+
+     Each alias entry must be in ``alias=command`` format.
+     Empty lines and lines starting with ``#`` are ignored.
+
+     Example alias file::
+
+        # Basic shortcuts
+        ls=/ethdev/list
+        names=FOREACH i /ethdev/list /ethdev/info,$i .name
+        q=quit
+
+     Alias behavior is intentionally similar to shell aliases:
+
+     - The first token of the entered input is checked for an alias match.
+     - If matched, that first token is replaced with its expansion.
+     - Alias expansion is recursive (aliases can expand to other aliases).
+     - Expansion has a safety limit to prevent infinite loops.
+
+     Examples::
+
+        --> ls
+        {"/ethdev/list": [0, 1]}
+
+        --> names
+        [{"i": 0, "name": "0000:16:00.0"}, {"i": 1, "name": "0000:16:00.1"}]
+
+        --> q
+        # exits the client
+
 
 Connecting to Different DPDK Processes
 --------------------------------------
usertools/dpdk-telemetry.py +71 -4

100 lines not on this page · raw message

Signature · 1 line
2.53.0
#9 lore raw

Re: [PATCH v2 2/3] usertools/telemetry: support using aliases for long commands

Quoted 30 lines · Bruce Richardson

Similarly to how shell aliases work, allow specifying of short alias commands for dpdk-telemetry.py script. The aliases are read from "$HOME/.dpdk_telemetry_aliases" at startup.

Some examples of use from the docs. Alias file contents:

# Basic shortcuts
ls=/ethdev/list
names=FOREACH i /ethdev/list /ethdev/info,$i .name q=quit

Alias use:

   --> ls

{"/ethdev/list": [0, 1]}

--> names
[{"i": 0, "name": "0000:

---
 doc/guides/howto/telemetry.rst | 34 +++++++++++++++
 usertools/dpdk-telemetry.py    | 75 ++++++++++++++++++++++++++++++++--
 2 files changed, 105 insertions(+), 4 deletions(-)
doc/guides/howto/telemetry.rst +0 -0
index 4bf48c635e..072289aec8 100644

doc/guides/howto/telemetry.rst and query information using the telemetry client python script. · line 137

Quoted 8 lines
doc/guides/howto/telemetry.rst +5 -0
@@ -130,6 +130,40 @@ and query information using the telemetry client python script.
      - With loop variable: returns an array of objects containing the loop
        variable field and requested value fields.
 
+   * Use command aliases.
+
+     The telemetry script can load aliases at startup from::
+
+        $HOME/.dpdk_telemetry_aliases

How about add one parameter to specific aliases file location

Quoted 70 lines
doc/guides/howto/telemetry.rst +29 -0
+
+     Each alias entry must be in ``alias=command`` format.
+     Empty lines and lines starting with ``#`` are ignored.
+
+     Example alias file::
+
+        # Basic shortcuts
+        ls=/ethdev/list
+        names=FOREACH i /ethdev/list /ethdev/info,$i .name
+        q=quit
+
+     Alias behavior is intentionally similar to shell aliases:
+
+     - The first token of the entered input is checked for an alias match.
+     - If matched, that first token is replaced with its expansion.
+     - Alias expansion is recursive (aliases can expand to other aliases).
+     - Expansion has a safety limit to prevent infinite loops.
+
+     Examples::
+
+        --> ls
+        {"/ethdev/list": [0, 1]}
+
+        --> names
+        [{"i": 0, "name": "0000:16:00.0"}, {"i": 1, "name": "0000:16:00.1"}]
+
+        --> q
+        # exits the client
+
 
 Connecting to Different DPDK Processes
 --------------------------------------
usertools/dpdk-telemetry.py +30 -0
index 2de10cff69..8b976160e0 100755
@@ -21,6 +21,70 @@
 SOCKET_NAME = "dpdk_telemetry.{}".format(TELEMETRY_VERSION)
 DEFAULT_PREFIX = "rte"
 CMDS = []
+ALIASES = {}
+ALIAS_FILE = ".dpdk_telemetry_aliases"
+MAX_ALIAS_EXPANSIONS = 32
+
+
+def load_aliases():
+    """Load aliases from $HOME/.dpdk_telemetry_aliases"""
+    aliases = {}
+    home = os.environ.get("HOME")
+    if not home:
+        return aliases
+
+    alias_path = os.path.join(home, ALIAS_FILE)
+    if not os.path.isfile(alias_path):
+        return aliases
+
+    try:
+        with open(alias_path) as alias_file:
+            for line_num, line in enumerate(alias_file, start=1):
+                entry = line.strip()
+                if not entry or entry.startswith("#"):
+                    continue
+                if "=" not in entry:
+                    print(
+                        "Warning: ignoring malformed alias at {}:{}".format(alias_path, line_num),
+                        file=sys.stderr,
+                    )
+                    continue
+                name, command = entry.split("=", 1)
+                name = name.strip()

usertools/dpdk-telemetry.py line 63

Quoted 10 lines
usertools/dpdk-telemetry.py +10 -0
+                command = command.strip()
+                if not name or not command:
+                    print(
+                        "Warning: ignoring malformed alias at {}:{}".format(alias_path, line_num),
+                        file=sys.stderr,
+                    )
+                    continue
+                aliases[name] = command
+    except OSError as e:
+        print("Warning: failed to read {}: {}".format(alias_path, e), file=sys.stderr)

If load OK, please add one promote at begin:

Loaded xxx command aliases, see help alias for detail

Quoted 4 lines
usertools/dpdk-telemetry.py +4 -0
+
+    return aliases
+
+
#10 lore raw

PATCH v2 3/3 usertools/telemetry: add help support

In reply to Bruce Richardson

While the telemetry infrastructure supported using "/help,/<cmd>" to get help on specific commands, with the addition of script-specific commands, we needed better help support to explain the use of FOREACH, and to allow e.g. "help /<cmd>" using space separation, which is more intuitive. This patch adds that help support.

---
 usertools/dpdk-telemetry.py | 61 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 60 insertions(+), 1 deletion(-)
usertools/dpdk-telemetry.py +60 -1
index 8b976160e0..2c88b91517 100755
@@ -25,6 +25,26 @@
 ALIAS_FILE = ".dpdk_telemetry_aliases"
 MAX_ALIAS_EXPANSIONS = 32
 
+BASIC_HELP_TEXT = """Basic usage:
+    /<command>[,<params>]      Send a telemetry command to the app
+    FOREACH ...                Run a compound query over list items
+    help                       Show this help
+    help /<command>            Show app-provided help for a command
+    help FOREACH               Show FOREACH usage and examples
+    quit                       Exit the client
+"""
+
+FOREACH_HELP_TEXT = """FOREACH usage:
+    FOREACH /<list_cmd> /<iter_cmd> .<field> [.<field> ...]
+    FOREACH <var> /<list_cmd> /<iter_cmd_with_$var> .<field> [.<field> ...]
+
+Examples:
+    FOREACH /ethdev/list /ethdev/stats .opackets
+    FOREACH /ethdev/list /ethdev/stats .ipackets .opackets
+    FOREACH i /ethdev/list /ethdev/info,$i .name
+    FOREACH i /ethdev/list /ethdev/stats,$i .ipackets .opackets
+"""
+
 
 def load_aliases():
     """Load aliases from $HOME/.dpdk_telemetry_aliases"""
@@ -203,10 +223,49 @@ def handle_foreach(sock, output_buf_len, text, pretty=False):
     print(json.dumps(output, indent=indent))
 
 
+def command_exists(cmd):
+    """Check if a telemetry command exists in the command list"""
+    return cmd in CMDS
+
+
+def app_help_command_for(target_cmd):
+    """Build a '/help,<command>' query for app-side command help"""
+    if not target_cmd:
+        return None
+    normalized = target_cmd.strip()
+    if not normalized.startswith("/"):
+        return None
+    if not command_exists(normalized):
+        print("Unknown command for help: {}".format(normalized))
+        return None
+    return "/help,{}".format(normalized)
+
+
+def handle_user_help(sock, output_buf_len, text, pretty=False):
+    """Handle local 'help' command and command-specific help lookup"""
+    parts = text.split(None, 1)
+    if len(parts) == 1:
+        print(BASIC_HELP_TEXT, end="")
+        return
+
+    help_arg = parts[1].strip()
+    if help_arg.upper() == "FOREACH":
+        print(FOREACH_HELP_TEXT, end="")
+        return
+
+    cmd = app_help_command_for(help_arg)
+    if cmd is None:
+        print("Usage: help [FOREACH|/<command>]")
+        return
+    send_command(sock, cmd, output_buf_len, echo=True, pretty=pretty)
+
+
 def handle_command(sock, output_buf_len, text, pretty=False):
     """Execute a user command if recognized"""
     if text.startswith("/"):
         send_command(sock, text, output_buf_len, echo=True, pretty=pretty)
+    elif text == "help" or text.startswith("help "):
+        handle_user_help(sock, output_buf_len, text, pretty)
     elif text.startswith("FOREACH "):
         handle_foreach(sock, output_buf_len, text, pretty)
 
@@ -340,7 +399,7 @@ def handle_socket(args, path):
 
 def readline_complete(text, state):
     """Find any matching commands from the list based on user input"""
-    all_cmds = ["quit"] + list(ALIASES.keys()) + CMDS
+    all_cmds = ["quit", "help"] + list(ALIASES.keys()) + CMDS
     if text:
         matches = [c for c in all_cmds if c.startswith(text)]
     else:
Signature · 1 line
2.53.0
#11 lore raw

Re: [PATCH v2 3/3] usertools/telemetry: add help support

Quoted 16 lines · Bruce Richardson

While the telemetry infrastructure supported using "/help,/<cmd>" to get help on specific commands, with the addition of script-specific commands, we needed better help support to explain the use of FOREACH, and to allow e.g. "help /<cmd>" using space separation, which is more intuitive. This patch adds that help support.

---
 usertools/dpdk-telemetry.py | 61 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 60 insertions(+), 1 deletion(-)
usertools/dpdk-telemetry.py +0 -0
index 8b976160e0..2c88b91517 100755

usertools/dpdk-telemetry.py line 34

Quoted 10 lines
usertools/dpdk-telemetry.py +7 -0
@@ -25,6 +25,26 @@
 ALIAS_FILE = ".dpdk_telemetry_aliases"
 MAX_ALIAS_EXPANSIONS = 32
 
+BASIC_HELP_TEXT = """Basic usage:
+    /<command>[,<params>]      Send a telemetry command to the app
+    FOREACH ...                Run a compound query over list items
+    help                       Show this help
+    help /<command>            Show app-provided help for a command
+    help FOREACH               Show FOREACH usage and examples
+    quit                       Exit the client

Please add help alias to show ALL loaded command alias

#12 lore raw

v3 cover extend interactive telemetry script

In reply to Bruce Richardson

To simplify interactive telemetry script for general use, i.e. not from other scripts, we can add two new features to it:

1. Support for FOREACH to allow gathering a set of output values across a list of ports or devices, e.g. ethdevs or rawdevs.
2. Support having predefined aliases in a file in the user's home directory to simplify the use of more complicated FOREACH commands.

Putting these together, we can create new commands such as "eth_names".

  bruce@host:$ cat ~/.dpdk_telemetry_aliases
  eth_names=FOREACH index /ethdev/list /ethdev/info,$index .name
  bruce@host:$ echo eth_names | ./usertools/dpdk-telemetry.py | jq
  [
    {
      "index": 0,
      "name": "0000:16:00.0"
    },
    {
      "index": 1,
      "name": "0000:16:00.1"
    }
  ]

---
v3: updated based on review feedback from Chengwen:
- added arg to override alias file
- printed one-line summary of alias count loaded
- improved doc for "help" command
- added "help alias" to list aliases.
v2: added third patch with "help" command giving more details on
how to use the various commands.

Bruce Richardson (3):
usertools/telemetry: add a FOREACH command usertools/telemetry: support using aliases for long commands usertools/telemetry: add help support

 doc/guides/howto/telemetry.rst | 106 ++++++++++++-
 usertools/dpdk-telemetry.py    | 278 ++++++++++++++++++++++++++++++++-
 2 files changed, 373 insertions(+), 11 deletions(-)

--
2.53.0

#13 lore raw

PATCH v3 1/3 usertools/telemetry: add a FOREACH command

To simplify querying data from multiple devices, e.g. across ethdevs, or dmadevs, add a FOREACH command to the python script, allowing you to run, e.g. /ethdev/list, and then run a second command for each item in the list, gathering the relevant output values, optionally including an index counter.

Simple examples are given in the documentation:

--> FOREACH /ethdev/list /ethdev/stats .opackets [0, 0]

--> FOREACH /ethdev/list /ethdev/stats .ipackets .opackets [{"ipackets": 0, "opackets": 0}, {"ipackets": 0, "opackets": 0}]

--> FOREACH i /ethdev/list /ethdev/info,$i .name
[{"i": 0, "name": "0000:16:00.0"}, {"i": 1, "name": "0000:16:00.1"}]

--> FOREACH i /ethdev/list /ethdev/stats,$i .ipackets .opackets [{"i": 0, "ipackets": 0, "opackets": 0}, {"i": 1, "ipackets": 0, "opackets": 0}]

---
 doc/guides/howto/telemetry.rst |  42 +++++++++++
 usertools/dpdk-telemetry.py    | 128 ++++++++++++++++++++++++++++++++-
 2 files changed, 167 insertions(+), 3 deletions(-)
doc/guides/howto/telemetry.rst +42 -0
index 0464c431fe..4bf48c635e 100644
@@ -88,6 +88,48 @@ and query information using the telemetry client python script.
        {"/help": {"/ethdev/xstats": "Returns the extended stats for a port.
        Parameters: int port_id"}}
 
+   * Run a compound query using ``FOREACH``.
+
+     The ``FOREACH`` command runs a list command, iterates each returned item,
+     runs a second command for each item, and emits combined JSON output.
+
+     Start with the simplest form (no loop variable)::
+
+        FOREACH /<list_cmd> /<iter_cmd> .<field> [.<field> ...]
+
+     To include numbered output, use a loop variable::
+
+        FOREACH <var> /<list_cmd> /<iter_cmd_with_$var> .<field> [.<field> ...]
+
+     Notes:
+
+     - Field selectors are whitespace-separated tokens, each starting with ``.``.
+     - In no-variable mode, the iter command is called as ``/<iter_cmd>,<item>``.
+     - In loop-variable mode, use ``$<var>`` in the iter command where the
+       item value should be substituted.
+
+     Examples::
+
+        --> FOREACH /ethdev/list /ethdev/stats .opackets
+        [0, 0]
+
+        --> FOREACH /ethdev/list /ethdev/stats .ipackets .opackets
+        [{"ipackets": 0, "opackets": 0}, {"ipackets": 0, "opackets": 0}]
+
+        --> FOREACH i /ethdev/list /ethdev/info,$i .name
+        [{"i": 0, "name": "0000:16:00.0"}, {"i": 1, "name": "0000:16:00.1"}]
+
+        --> FOREACH i /ethdev/list /ethdev/stats,$i .ipackets .opackets
+        [{"i": 0, "ipackets": 0, "opackets": 0}, {"i": 1, "ipackets": 0, "opackets": 0}]
+
+     Output behavior:
+
+     - Without loop variable and one field: returns an array of values.
+     - Without loop variable and multiple fields: returns an array of objects
+       containing named value fields.
+     - With loop variable: returns an array of objects containing the loop
+       variable field and requested value fields.
+
 
 Connecting to Different DPDK Processes
 --------------------------------------
usertools/dpdk-telemetry.py +125 -3

140 lines not on this page · raw message

Signature · 1 line
2.53.0
#14 lore raw

PATCH v3 2/3 usertools/telemetry: support using aliases for long commands

In reply to Bruce Richardson

Similarly to how shell aliases work, allow specifying of short alias commands for dpdk-telemetry.py script. The aliases are read from "$HOME/.dpdk_telemetry_aliases" at startup.

Some examples of use from the docs. Alias file contents:

# Basic shortcuts
ls=/ethdev/list
names=FOREACH i /ethdev/list /ethdev/info,$i .name q=quit

Alias use:

--> ls
{"/ethdev/list": [0, 1]}

--> names
[{"i": 0, "name": "0000:

---
v2: added support for providing an alias path on cmdline.
added summary printout of how many aliases were loaded.

---
 doc/guides/howto/telemetry.rst | 35 ++++++++++++++
 usertools/dpdk-telemetry.py    | 85 ++++++++++++++++++++++++++++++++--
 2 files changed, 116 insertions(+), 4 deletions(-)
doc/guides/howto/telemetry.rst +35 -0
index 4bf48c635e..bdefbdc6a6 100644
@@ -130,6 +130,41 @@ and query information using the telemetry client python script.
      - With loop variable: returns an array of objects containing the loop
        variable field and requested value fields.
 
+   * Use command aliases.
+
+     The telemetry script can load aliases at startup from::
+
+        $HOME/.dpdk_telemetry_aliases
+
+     or from a custom path provided via the ``--alias-file`` script flag.
+     Each alias entry must be in ``alias=command`` format.
+     Empty lines and lines starting with ``#`` are ignored.
+
+     Example alias file::
+
+        # Basic shortcuts
+        ls=/ethdev/list
+        names=FOREACH i /ethdev/list /ethdev/info,$i .name
+        q=quit
+
+     Alias behavior is intentionally similar to shell aliases:
+
+     - The first token of the entered input is checked for an alias match.
+     - If matched, that first token is replaced with its expansion.
+     - Alias expansion is recursive (aliases can expand to other aliases).
+     - Expansion has a safety limit to prevent infinite loops.
+
+     Examples::
+
+        --> ls
+        {"/ethdev/list": [0, 1]}
+
+        --> names
+        [{"i": 0, "name": "0000:16:00.0"}, {"i": 1, "name": "0000:16:00.1"}]
+
+        --> q
+        # exits the client
+
 
 Connecting to Different DPDK Processes
 --------------------------------------
usertools/dpdk-telemetry.py +81 -4

116 lines not on this page · raw message

Signature · 1 line
2.53.0
#15 lore raw

Re: [PATCH v3 2/3] usertools/telemetry: support using aliases for long commands

09/06/2026 18:13, Bruce Richardson:

Quoted 3 lines

Similarly to how shell aliases work, allow specifying of short alias commands for dpdk-telemetry.py script. The aliases are read from "$HOME/.dpdk_telemetry_aliases" at startup.

I don't like polluting the home directory of the user. We already have these files and directories:
~/.config/dpdk/devel.config ~/.dpdk_telemetry_aliases
~/dpdk-traces/ ~/dpdk_graph_pcap_capture_XXXXXX.pcapng /usr/share/dpdk/telemetry-endpoints

I think ~/.config/dpdk/ is the best place for our config files.

It is not blocking the series but we should work on unifying these files.

#16 lore raw

PATCH v3 3/3 usertools/telemetry: add help support

In reply to Bruce Richardson

While the telemetry infrastructure supported using "/help,/<cmd>" to get help on specific commands, with the addition of script-specific commands, we needed better help support to explain the use of FOREACH, and to allow e.g. "help /<cmd>" using space separation, which is more intuitive. This patch adds that help support.

---
v2: added "help aliases" to list defined aliases
updated docs for expanded help command

---
 doc/guides/howto/telemetry.rst | 31 ++++++++++++---
 usertools/dpdk-telemetry.py    | 69 +++++++++++++++++++++++++++++++++-
 2 files changed, 93 insertions(+), 7 deletions(-)
doc/guides/howto/telemetry.rst +25 -6
index bdefbdc6a6..00cfc1a1e1 100644
@@ -81,12 +81,31 @@ and query information using the telemetry client python script.
        ...
        "tx_priority7_xon_to_xoff_packets": 0}}
 
-   * Get the help text for a command. This will indicate what parameters are
-     required. Pass the command as a parameter::
-
-       --> /help,/ethdev/xstats
-       {"/help": {"/ethdev/xstats": "Returns the extended stats for a port.
-       Parameters: int port_id"}}
+   * Get the help text for a command.
+     This will indicate what parameters are required.
+     Use the ``help`` keyword followed by the command or keyword of interest,
+     for example::
+
+       --> help FOREACH
+       FOREACH usage:
+          FOREACH /<list_cmd> /<iter_cmd> .<field> [.<field> ...]
+          FOREACH <var> /<list_cmd> /<iter_cmd_with_$var> .<field> [.<field> ...]
+
+       Examples:
+          FOREACH /ethdev/list /ethdev/stats .opackets
+          ...
+
+       --> help /ethdev/xstats
+       {"/help": {"/ethdev/xstats": "Returns the extended stats for a port. Parameters: int port_id"}}
+
+   ..  Note::
+         For commands starting with ``/`` that are telemetry enpoints,
+         the help text can also be obtained by sending the ``/help`` command to the telemetry socket.
+         In this case, the parameter must be separated by a comma, not a space.
+         For example::
+
+           --> /help,/ethdev/xstats
+           {"/help": {"/ethdev/xstats": "Returns the extended stats for a port. Parameters: int port_id"}}
 
    * Run a compound query using ``FOREACH``.
 
usertools/dpdk-telemetry.py +68 -1

91 lines not on this page · raw message

Signature · 1 line
2.53.0
#17 lore raw

Re: [PATCH v3 3/3] usertools/telemetry: add help support

Quoted 38 lines · Bruce Richardson

While the telemetry infrastructure supported using "/help,/<cmd>" to get help on specific commands, with the addition of script-specific commands, we needed better help support to explain the use of FOREACH, and to allow e.g. "help /<cmd>" using space separation, which is more intuitive. This patch adds that help support.

---
v2: added "help aliases" to list defined aliases
updated docs for expanded help command

---
 doc/guides/howto/telemetry.rst | 31 ++++++++++++---
 usertools/dpdk-telemetry.py    | 69 +++++++++++++++++++++++++++++++++-
 2 files changed, 93 insertions(+), 7 deletions(-)
doc/guides/howto/telemetry.rst +9 -6
index bdefbdc6a6..00cfc1a1e1 100644
@@ -81,12 +81,31 @@ and query information using the telemetry client python script.
...
        "tx_priority7_xon_to_xoff_packets": 0}}
 
-   * Get the help text for a command. This will indicate what parameters are
-     required. Pass the command as a parameter::
-
-       --> /help,/ethdev/xstats
-       {"/help": {"/ethdev/xstats": "Returns the extended stats for a port.
-       Parameters: int port_id"}}
+   * Get the help text for a command.
+     This will indicate what parameters are required.
+     Use the ``help`` keyword followed by the command or keyword of interest,
+     for example::
+
+       --> help FOREACH
+       FOREACH usage:
+          FOREACH /<list_cmd> /<iter_cmd> .<field> [.<field> ...]
+          FOREACH <var> /<list_cmd> /<iter_cmd_with_$var> .<field> [.<field> ...]

doc/guides/howto/telemetry.rst and query information using the telemetry client python script. · line 102

Quoted 10 lines
doc/guides/howto/telemetry.rst +10 -0
+
+       Examples:
+          FOREACH /ethdev/list /ethdev/stats .opackets
+          ...
+
+       --> help /ethdev/xstats
+       {"/help": {"/ethdev/xstats": "Returns the extended stats for a port. Parameters: int port_id"}}
+
+   ..  Note::
+         For commands starting with ``/`` that are telemetry enpoints,

Checkpatch in CI correctly points out the typo for "endpoints" here. Please fix on apply if no v4 is otherwise necessary.

thanks,
/Bruce

#18 lore raw
In reply to Bruce Richardson
Quoted 47 lines · Bruce Richardson

To simplify interactive telemetry script for general use, i.e. not from other scripts, we can add two new features to it:

1. Support for FOREACH to allow gathering a set of output values across a list of ports or devices, e.g. ethdevs or rawdevs.
2. Support having predefined aliases in a file in the user's home directory to simplify the use of more complicated FOREACH commands.

Putting these together, we can create new commands such as "eth_names".

  bruce@host:$ cat ~/.dpdk_telemetry_aliases
  eth_names=FOREACH index /ethdev/list /ethdev/info,$index .name
  bruce@host:$ echo eth_names | ./usertools/dpdk-telemetry.py | jq
  [
    {
      "index": 0,
      "name": "0000:16:00.0"
    },
    {
      "index": 1,
      "name": "0000:16:00.1"
    }
  ]

---
v3: updated based on review feedback from Chengwen:
- added arg to override alias file
- printed one-line summary of alias count loaded
- improved doc for "help" command
- added "help alias" to list aliases.
v2: added third patch with "help" command giving more details on
how to use the various commands.

Bruce Richardson (3):
usertools/telemetry: add a FOREACH command usertools/telemetry: support using aliases for long commands usertools/telemetry: add help support

 doc/guides/howto/telemetry.rst | 106 ++++++++++++-
 usertools/dpdk-telemetry.py    | 278 ++++++++++++++++++++++++++++++++-
 2 files changed, 373 insertions(+), 11 deletions(-)

--
2.53.0

#19 lore raw
Quoted 6 lines
Quoted 4 lines

Bruce Richardson (3):
usertools/telemetry: add a FOREACH command usertools/telemetry: support using aliases for long commands usertools/telemetry: add help support

Applied, thanks.