Skip to contentSkip to conversation

app/test: rewrite telemetry test in python

PATCH Superseded Stephen Hemminger 3 participants 4 messages Jul 24 – Nov 19 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 app/test: rewrite telemetry test in python

The test_telemetry.sh was failing with test timeout.
This was caused by the overhead of spawning a fresh dpdk-telemetry.py for every command it walked. With every endpoint queried three times, that is several hundred Python interpreter startups per run.

Replace the shell script with a python test that opens the telemetry socket once and issues every command over that single connection.

Each reply is parsed as JSON and checked directly in Python. This also drops the jq dependency.

Bugzilla ID: 1972

Fixes 9da71dc4f96e ("test: add test case for scripted telemetry commands")
---
 app/test/suites/meson.build       |   2 +-
 app/test/suites/test_telemetry.py | 129 ++++++++++++++++++++++++++++++
 app/test/suites/test_telemetry.sh |  30 -------
 3 files changed, 130 insertions(+), 31 deletions(-)
 create mode 100755 app/test/suites/test_telemetry.py
 delete mode 100755 app/test/suites/test_telemetry.sh
app/test/suites/meson.build +1 -1
index 786c459c24..ec1de99154 100644
@@ -145,7 +145,7 @@ if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
         test_args += ['--vdev=rawdev_skeleton0']
     endif
     test_args += ['-a', '0000:00:00.0']
-    test('telemetry_all', find_program('test_telemetry.sh'),
+    test('telemetry_all', find_program('test_telemetry.py'),
             args: test_args,
             timeout : timeout_seconds_fast,
             is_parallel : false,
app/test/suites/test_telemetry.py new file +129 -0

129 lines not on this page · raw message

app/test/suites/test_telemetry.sh deleted +0 -30
deleted file mode 100755
index 3c5b629b63..0000000000
@@ -1,30 +0,0 @@
-#!/bin/sh -e
-# SPDX-License-Identifier: BSD-3-Clause
-# Copyright (c) 2022 Red Hat, Inc.
-
-which jq || {
-    echo "No jq available, skipping test."
-    exit 77
-}
-
-rootdir=$(readlink -f $(dirname $(readlink -f $0))/../../..)
-tmpoutput=$(mktemp -t dpdk.test_telemetry.XXXXXX)
-trap "cat $tmpoutput; rm -f $tmpoutput" EXIT
-
-call_all_telemetry() {
-    telemetry_script=$rootdir/usertools/dpdk-telemetry.py
-    echo >$tmpoutput
-    echo "Telemetry commands log:" >>$tmpoutput
-    echo / | $telemetry_script | jq -r '.["/"][]' | while read cmd
-    do
-        for input in $cmd $cmd,0 $cmd,z
-        do
-            echo Calling $input >> $tmpoutput
-            echo $input | $telemetry_script >> $tmpoutput 2>&1
-        done
-    done
-}
-
-! set -o | grep -q errtrace || set -o errtrace
-! set -o | grep -q pipefail || set -o pipefail
-(sleep 1 && call_all_telemetry && echo quit) | $@
Signature · 1 line
2.53.0
Acked-by on v1 1/1
#1 lore raw
Quoted 127 lines · Stephen Hemminger

The test_telemetry.sh was failing with test timeout.
This was caused by the overhead of spawning a fresh dpdk-telemetry.py for every command it walked. With every endpoint queried three times, that is several hundred Python interpreter startups per run.

Replace the shell script with a python test that opens the telemetry socket once and issues every command over that single connection.

Each reply is parsed as JSON and checked directly in Python. This also drops the jq dependency.

Bugzilla ID: 1972

Fixes 9da71dc4f96e ("test: add test case for scripted telemetry commands")
---
 app/test/suites/meson.build       |   2 +-
 app/test/suites/test_telemetry.py | 129 ++++++++++++++++++++++++++++++
 app/test/suites/test_telemetry.sh |  30 -------
 3 files changed, 130 insertions(+), 31 deletions(-)
 create mode 100755 app/test/suites/test_telemetry.py
 delete mode 100755 app/test/suites/test_telemetry.sh
app/test/suites/meson.build +1 -1
index 786c459c24..ec1de99154 100644
@@ -145,7 +145,7 @@ if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
         test_args += ['--vdev=rawdev_skeleton0']
     endif
     test_args += ['-a', '0000:00:00.0']
-    test('telemetry_all', find_program('test_telemetry.sh'),
+    test('telemetry_all', find_program('test_telemetry.py'),
             args: test_args,
             timeout : timeout_seconds_fast,
             is_parallel : false,
app/test/suites/test_telemetry.py new file +84 -0
new file mode 100755
index 0000000000..f37eccabcc
@@ -0,0 +1,129 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2022 Red Hat, Inc.
+
+"""Exercise every telemetry command exported by an application.
+
+Spawns the DPDK test binary (passed as arguments), waits for its telemetry
+socket to appear, then walks every command reported by "/", calling each one
+with no parameter and with dummy parameters "0" and "z". Every reply is parsed
+as JSON and checked, so a malformed, empty or missing response fails the test
+immediately and names the offending command, rather than relying on a shell
+pipeline not erroring.
+
+A single connection is reused for the whole walk: the previous shell version
+spawned a fresh dpdk-telemetry.py (Python interpreter + new connection) per
+command, which scaled with process-startup cost and timed out under load.
+"""
+
+import json
+import os
+import socket
+import subprocess
+import sys
+import time
+
+SOCKET_NAME = "dpdk_telemetry.v2"
+
+
+def runtime_dir():
+    """DPDK runtime dir for the default 'rte' file-prefix, matching EAL."""
+    run = os.environ.get("RUNTIME_DIRECTORY")
+    if not run:
+        run = (
+            "/var/run"
+            if os.getuid() == 0
+            else os.environ.get("XDG_RUNTIME_DIR", "/tmp")
+        )
+    return os.path.join(run, "dpdk", "rte")
+
+
+def wait_for_socket(path, proc, timeout=10):
+    """Wait for the telemetry socket, failing fast if the app dies first."""
+    deadline = time.time() + timeout
+    while time.time() < deadline:
+        if os.path.exists(path):
+            return
+        if proc.poll() is not None:
+            raise RuntimeError(
+                "application exited (code %d) before telemetry socket appeared"
+                % proc.returncode
+            )
+        time.sleep(0.05)
+    raise RuntimeError("timed out waiting for telemetry socket %s" % path)
+
+
+class TelemetryClient:
+    def __init__(self, path):
+        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_SEQPACKET)
+        self.sock.connect(path)
+        info = json.loads(self.sock.recv(1024))
+        self.buf_len = info["max_output_len"]
+
+    def command(self, cmd):
+        self.sock.send(cmd.encode())
+        reply = self.sock.recv(self.buf_len).decode()
+        try:
+            return json.loads(reply)
+        except json.JSONDecodeError as e:
+            raise AssertionError(
+                "invalid JSON reply for %r: %s (raw: %r)" % (cmd, e, reply)
+            )
+
+    def close(self):
+        self.sock.close()
+
+
+def check_reply(cmd, reply):
+    """A telemetry reply must be a dict keyed by the command name."""
+    if not isinstance(reply, dict) or list(reply.keys()) != [cmd.split(",")[0]]:
+        raise AssertionError("unexpected reply for %r: %r" % (cmd, reply))
+
+
+def walk(client):
+    listing = client.command("/")

app/test/suites/test_telemetry.py line 94

Quoted 10 lines
app/test/suites/test_telemetry.py new file +10 -0
+    check_reply("/", listing)
+    count = 0
+    for cmd in listing["/"]:
+        for arg in ("", ",0", ",z"):
+            full = cmd + arg
+            reply = client.command(full)
+            check_reply(full, reply)
+            count += 1
+    return count
+

Not necessary to fix in this patch, but this iteration can be improved using a few heuristics. For example, each node of type /*dev/list should take no parameters and return a list of devices that can be passed to the equivalent /*dev/info and /*dev/stats or xstats nodes. This existing simple logic can then be used as a fallback for anything not meeting that pattern.

For this whole shell to python replacement:

Quoted 74 lines
app/test/suites/test_telemetry.py new file +35 -0
+
+def main():
+    if len(sys.argv) < 2:
+        print("usage: %s <dpdk-app> [eal args...]" % sys.argv[0], file=sys.stderr)
+        return 1
+
+    sock_path = os.path.join(runtime_dir(), SOCKET_NAME)
+    proc = subprocess.Popen(sys.argv[1:], stdin=subprocess.PIPE)
+    try:
+        wait_for_socket(sock_path, proc)
+        client = TelemetryClient(sock_path)
+        try:
+            count = walk(client)
+        finally:
+            client.close()
+        print("telemetry: walked %d commands" % count)
+    finally:
+        # tell the interactive prompt to exit, then ensure the app is gone
+        try:
+            proc.stdin.write(b"quit\n")
+            proc.stdin.flush()
+            proc.stdin.close()
+        except (BrokenPipeError, OSError):
+            pass
+        try:
+            proc.wait(timeout=5)
+        except subprocess.TimeoutExpired:
+            proc.terminate()
+            proc.wait()
+
+    return 0 if proc.returncode == 0 else proc.returncode
+
+
+if __name__ == "__main__":
+    sys.exit(main())
app/test/suites/test_telemetry.sh deleted +0 -30
deleted file mode 100755
index 3c5b629b63..0000000000
@@ -1,30 +0,0 @@
-#!/bin/sh -e
-# SPDX-License-Identifier: BSD-3-Clause
-# Copyright (c) 2022 Red Hat, Inc.
-
-which jq || {
-    echo "No jq available, skipping test."
-    exit 77
-}
-
-rootdir=$(readlink -f $(dirname $(readlink -f $0))/../../..)
-tmpoutput=$(mktemp -t dpdk.test_telemetry.XXXXXX)
-trap "cat $tmpoutput; rm -f $tmpoutput" EXIT
-
-call_all_telemetry() {
-    telemetry_script=$rootdir/usertools/dpdk-telemetry.py
-    echo >$tmpoutput
-    echo "Telemetry commands log:" >>$tmpoutput
-    echo / | $telemetry_script | jq -r '.["/"][]' | while read cmd
-    do
-        for input in $cmd $cmd,0 $cmd,z
-        do
-            echo Calling $input >> $tmpoutput
-            echo $input | $telemetry_script >> $tmpoutput 2>&1
-        done
-    done
-}
-
-! set -o | grep -q errtrace || set -o errtrace
-! set -o | grep -q pipefail || set -o pipefail
-(sleep 1 && call_all_telemetry && echo quit) | $@
Signature · 1 line
2.53.0
#2 lore raw

PATCH v2 1/1 app/test: rewrite telemetry test in python

In reply to Stephen Hemminger

The test_telemetry.sh was failing with test timeout.
This was caused by the overhead of spawning a fresh dpdk-telemetry.py for every command it walked. With every endpoint queried three times, that is several hundred Python interpreter startups per run.

Replace the shell script with a python test that opens the telemetry socket once and issues every command over that single connection.

Each reply is parsed as JSON and checked directly in Python. This also drops the jq dependency.

Bugzilla ID: 1972

Fixes 9da71dc4f96e ("test: add test case for scripted telemetry commands")

---
v2 - incorporate Bruce's suggestion
- add MAINTAINERS change

 MAINTAINERS                       |   2 +-
 app/test/suites/meson.build       |   2 +-
 app/test/suites/test_telemetry.py | 168 ++++++++++++++++++++++++++++++
 app/test/suites/test_telemetry.sh |  30 ------
 4 files changed, 170 insertions(+), 32 deletions(-)
 create mode 100644 app/test/suites/test_telemetry.py
 delete mode 100755 app/test/suites/test_telemetry.sh
MAINTAINERS +1 -1
index e99a65d197..e978b068ad 100644
@@ -1912,7 +1912,7 @@ M: Bruce Richardson <[email protected]>
 F: lib/telemetry/
 F: doc/guides/prog_guide/telemetry_lib.rst
 F: app/test/test_telemetry*
-F: app/test/suites/test_telemetry.sh
+F: app/test/suites/test_telemetry.*
 F: usertools/dpdk-telemetry*
 F: doc/guides/howto/telemetry.rst
 
app/test/suites/meson.build +1 -1
index 786c459c24..ec1de99154 100644
@@ -145,7 +145,7 @@ if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
         test_args += ['--vdev=rawdev_skeleton0']
     endif
     test_args += ['-a', '0000:00:00.0']
-    test('telemetry_all', find_program('test_telemetry.sh'),
+    test('telemetry_all', find_program('test_telemetry.py'),
             args: test_args,
             timeout : timeout_seconds_fast,
             is_parallel : false,
app/test/suites/test_telemetry.py new file +168 -0

168 lines not on this page · raw message

app/test/suites/test_telemetry.sh deleted +0 -30
deleted file mode 100755
index 3c5b629b63..0000000000
@@ -1,30 +0,0 @@
-#!/bin/sh -e
-# SPDX-License-Identifier: BSD-3-Clause
-# Copyright (c) 2022 Red Hat, Inc.
-
-which jq || {
-    echo "No jq available, skipping test."
-    exit 77
-}
-
-rootdir=$(readlink -f $(dirname $(readlink -f $0))/../../..)
-tmpoutput=$(mktemp -t dpdk.test_telemetry.XXXXXX)
-trap "cat $tmpoutput; rm -f $tmpoutput" EXIT
-
-call_all_telemetry() {
-    telemetry_script=$rootdir/usertools/dpdk-telemetry.py
-    echo >$tmpoutput
-    echo "Telemetry commands log:" >>$tmpoutput
-    echo / | $telemetry_script | jq -r '.["/"][]' | while read cmd
-    do
-        for input in $cmd $cmd,0 $cmd,z
-        do
-            echo Calling $input >> $tmpoutput
-            echo $input | $telemetry_script >> $tmpoutput 2>&1
-        done
-    done
-}
-
-! set -o | grep -q errtrace || set -o errtrace
-! set -o | grep -q pipefail || set -o pipefail
-(sleep 1 && call_all_telemetry && echo quit) | $@
Signature · 1 line
2.53.0
#3 lore raw

12/08/2026 21:01, Stephen Hemminger:

Quoted 11 lines

The test_telemetry.sh was failing with test timeout.
This was caused by the overhead of spawning a fresh dpdk-telemetry.py for every command it walked. With every endpoint queried three times, that is several hundred Python interpreter startups per run.

Replace the shell script with a python test that opens the telemetry socket once and issues every command over that single connection.

Each reply is parsed as JSON and checked directly in Python. This also drops the jq dependency.

Quoted 6 lines

Bugzilla ID: 1972

Fixes 9da71dc4f96e ("test: add test case for scripted telemetry commands")

Applied, thanks.