Skip to contentSkip to conversation

perf python: Add missing infra pieces for counting

PATCH Applied Ian Rogers 3 participants 20 messages May 19, 2025 – May 26, 2025 perf events

lore
Share & tools
lore

Ian Rogers

author reviewer
#0 lore raw

v3 cover perf python: Add missing infra pieces for counting

Add the missing infra code in the perf python bindings for measuring and reading the counter values for the given perf event. Demonstrate the usage of this with counting.py - a python version of counting.c

Ian v2 -> v3:
1. Make the read API use CPUs and threads rather than indices as discussed in:
https://lore.kernel.org/linux-perf-users/[email protected]/
2. Add python cpus and threads functions to evsel so that these can be iterated to avoid reading CPUs not present on a parsed evsel.
3. Clean up bits of the perf_thread_map libperf API.
3. Allow a command line event to be specified for parsing, for example:

```
$ tools/perf/python/counting.py -e data_read
For evsel(uncore_imc_free_running_0/data_read/) val: 23062 enable: 1612523 run: 1612523
For evsel(uncore_imc_free_running_1/data_read/) val: 22423 enable: 1599354 run: 1599354
```

Gautam v1 -> v2:
1. Use the existing iteration support for evlist
2. Drop the use of next method
3. Use existing helper functions for python example

Gautam Menghani (4):
perf python: Add support for perf_counts_values to return counter data perf python: Add evsel read method
perf python: Add evlist close support
perf python: Add counting.py as example for counting perf events

Ian Rogers (3):
libperf threadmap: Don't segv for index 0 for the NULL perf_thread_map libperf threadmap: Add perf_thread_map__idx
perf python: Add evsel cpus and threads functions

 tools/lib/perf/include/perf/threadmap.h |   1 +
 tools/lib/perf/threadmap.c              |  17 +++
 tools/perf/python/counting.py           |  36 +++++
 tools/perf/util/python.c                | 178 +++++++++++++++++++++++-
 4 files changed, 231 insertions(+), 1 deletion(-)
 create mode 100755 tools/perf/python/counting.py
Signature · 1 line
2.49.0.1101.gccaa498523-goog
author reviewer
#1 lore raw

PATCH v3 1/7 libperf threadmap: Don't segv for index 0 for the NULL perf_thread_map

perf_thread_map__nr returns length 1 if the perf_thread_map is NULL, meaning index 0 is valid. When perf_thread_map__pid of index 0 is read then return the expected "any" -1 value. Assert this is only done for index 0.

---
 tools/lib/perf/threadmap.c | 5 +++++
 1 file changed, 5 insertions(+)
tools/lib/perf/threadmap.c +5 -0
index 07968f3ea093..3ca9ba4987fc 100644
@@ -97,5 +97,10 @@ int perf_thread_map__nr(struct perf_thread_map *threads)
 
 pid_t perf_thread_map__pid(struct perf_thread_map *map, int idx)
 {
+	if (!map) {
+		assert(idx == 0);
+		return -1;
+	}
+
 	return map->map[idx].pid;
 }
Signature · 1 line
2.49.0.1101.gccaa498523-goog
author reviewer
#2 lore raw

PATCH v3 2/7 libperf threadmap: Add perf_thread_map__idx

In reply to Ian Rogers

Allow computation of thread map index from a PID. Note, with a perf_cpu_map the sorted nature allows for a binary search to compute the index which isn't currently possible with a perf_thread_map as they aren't guaranteed sorted.

---
 tools/lib/perf/include/perf/threadmap.h |  1 +
 tools/lib/perf/threadmap.c              | 12 ++++++++++++
 2 files changed, 13 insertions(+)
tools/lib/perf/include/perf/threadmap.h +1 -0
index 8b40e7777cea..44deb815b817 100644
@@ -14,6 +14,7 @@ LIBPERF_API void perf_thread_map__set_pid(struct perf_thread_map *map, int idx,
 LIBPERF_API char *perf_thread_map__comm(struct perf_thread_map *map, int idx);
 LIBPERF_API int perf_thread_map__nr(struct perf_thread_map *threads);
 LIBPERF_API pid_t perf_thread_map__pid(struct perf_thread_map *map, int idx);
+LIBPERF_API int perf_thread_map__idx(struct perf_thread_map *map, pid_t pid);
 
 LIBPERF_API struct perf_thread_map *perf_thread_map__get(struct perf_thread_map *map);
 LIBPERF_API void perf_thread_map__put(struct perf_thread_map *map);
tools/lib/perf/threadmap.c +12 -0
index 3ca9ba4987fc..db431b036f57 100644
@@ -104,3 +104,15 @@ pid_t perf_thread_map__pid(struct perf_thread_map *map, int idx)
 
 	return map->map[idx].pid;
 }
+
+int perf_thread_map__idx(struct perf_thread_map *threads, pid_t pid)
+{
+	if (!threads)
+		return pid == -1 ? 0 : -1;
+
+	for (int i = 0; i < threads->nr; ++i) {
+		if (threads->map[i].pid == pid)
+			return i;
+	}
+	return -1;
+}
Signature · 1 line
2.49.0.1101.gccaa498523-goog
author reviewer
#3 lore raw

PATCH v3 3/7 perf python: Add evsel cpus and threads functions

In reply to Ian Rogers

Allow access to cpus and thread_map associated with an evsel.

---
 tools/perf/util/python.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
tools/perf/util/python.c +33 -0
index f3c05da25b4a..ead3afd2d996 100644
@@ -781,6 +781,27 @@ static PyObject *pyrf_evsel__open(struct pyrf_evsel *pevsel,
 	return Py_None;
 }
 
+static PyObject *pyrf_evsel__cpus(struct pyrf_evsel *pevsel)
+{
+	struct pyrf_cpu_map *pcpu_map = PyObject_New(struct pyrf_cpu_map, &pyrf_cpu_map__type);
+
+	if (pcpu_map)
+		pcpu_map->cpus = perf_cpu_map__get(pevsel->evsel.core.cpus);
+
+	return (PyObject *)pcpu_map;
+}
+
+static PyObject *pyrf_evsel__threads(struct pyrf_evsel *pevsel)
+{
+	struct pyrf_thread_map *pthread_map =
+		PyObject_New(struct pyrf_thread_map, &pyrf_thread_map__type);
+
+	if (pthread_map)
+		pthread_map->threads = perf_thread_map__get(pevsel->evsel.core.threads);
+
+	return (PyObject *)pthread_map;
+}
+
 static PyObject *pyrf_evsel__str(PyObject *self)
 {
 	struct pyrf_evsel *pevsel = (void *)self;
@@ -799,6 +820,18 @@ static PyMethodDef pyrf_evsel__methods[] = {
 		.ml_flags = METH_VARARGS | METH_KEYWORDS,
 		.ml_doc	  = PyDoc_STR("open the event selector file descriptor table.")
 	},
+	{
+		.ml_name  = "cpus",
+		.ml_meth  = (PyCFunction)pyrf_evsel__cpus,
+		.ml_flags = METH_NOARGS,
+		.ml_doc	  = PyDoc_STR("CPUs the event is to be used with.")
+	},
+	{
+		.ml_name  = "threads",
+		.ml_meth  = (PyCFunction)pyrf_evsel__threads,
+		.ml_flags = METH_NOARGS,
+		.ml_doc	  = PyDoc_STR("threads the event is to be used with.")
+	},
 	{ .ml_name = NULL, }
 };
 
Signature · 1 line
2.49.0.1101.gccaa498523-goog
author reviewer
#4 lore raw

PATCH v3 4/7 perf python: Add support for perf_counts_values to return counter data

In reply to Ian Rogers

From: Gautam Menghani <[email protected]>

Add support for perf_counts_values struct to enable the python bindings to read and return the counter data.

---
 tools/perf/util/python.c | 92 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 91 insertions(+), 1 deletion(-)
tools/perf/util/python.c +91 -1
index ead3afd2d996..1cbddfe77c7c 100644
@@ -626,6 +626,92 @@ static int pyrf_thread_map__setup_types(void)
 	return PyType_Ready(&pyrf_thread_map__type);
 }
 
+struct pyrf_counts_values {
+	PyObject_HEAD
+
+	struct perf_counts_values values;
+};
+
+static const char pyrf_counts_values__doc[] = PyDoc_STR("perf counts values object.");
+
+static void pyrf_counts_values__delete(struct pyrf_counts_values *pcounts_values)
+{
+	Py_TYPE(pcounts_values)->tp_free((PyObject *)pcounts_values);
+}
+
+#define counts_values_member_def(member, ptype, help) \
+	{ #member, ptype, \
+	  offsetof(struct pyrf_counts_values, values.member), \
+	  0, help }
+
+static PyMemberDef pyrf_counts_values_members[] = {
+	counts_values_member_def(val, Py_T_ULONG, "Value of event"),
+	counts_values_member_def(ena, Py_T_ULONG, "Time for which enabled"),
+	counts_values_member_def(run, Py_T_ULONG, "Time for which running"),
+	counts_values_member_def(id, Py_T_ULONG, "Unique ID for an event"),
+	counts_values_member_def(lost, Py_T_ULONG, "Num of lost samples"),
+	{NULL}
+};
+
+static PyObject *pyrf_counts_values_get_values(struct pyrf_counts_values *self, void *closure)
+{
+	PyObject *vals = PyList_New(5);
+
+	if (!vals)
+		return NULL;
+	for (int i = 0; i < 5; i++)
+		PyList_SetItem(vals, i, PyLong_FromLong(self->values.values[i]));
+
+	return vals;
+}
+
+static int pyrf_counts_values_set_values(struct pyrf_counts_values *self, PyObject *list,
+					 void *closure)
+{
+	Py_ssize_t size;
+	PyObject *item = NULL;
+
+	if (!PyList_Check(list)) {
+		PyErr_SetString(PyExc_TypeError, "Value assigned must be a list");
+		return -1;
+	}
+
+	size = PyList_Size(list);
+	for (Py_ssize_t i = 0; i < size; i++) {
+		item = PyList_GetItem(list, i);
+		if (!PyLong_Check(item)) {
+			PyErr_SetString(PyExc_TypeError, "List members should be numbers");
+			return -1;
+		}
+		self->values.values[i] = PyLong_AsLong(item);
+	}
+
+	return 0;
+}
+
+static PyGetSetDef pyrf_counts_values_getset[] = {
+	{"values", (getter)pyrf_counts_values_get_values, (setter)pyrf_counts_values_set_values,
+		"Name field", NULL},
+	{NULL}
+};
+
+static PyTypeObject pyrf_counts_values__type = {
+	PyVarObject_HEAD_INIT(NULL, 0)
+	.tp_name	= "perf.counts_values",
+	.tp_basicsize	= sizeof(struct pyrf_counts_values),
+	.tp_dealloc	= (destructor)pyrf_counts_values__delete,
+	.tp_flags	= Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE,
+	.tp_doc		= pyrf_counts_values__doc,
+	.tp_members	= pyrf_counts_values_members,
+	.tp_getset	= pyrf_counts_values_getset,
+};
+
+static int pyrf_counts_values__setup_types(void)
+{
+	pyrf_counts_values__type.tp_new = PyType_GenericNew;
+	return PyType_Ready(&pyrf_counts_values__type);
+}
+
 struct pyrf_evsel {
 	PyObject_HEAD
 
@@ -1475,7 +1561,8 @@ PyMODINIT_FUNC PyInit_perf(void)
 	    pyrf_evlist__setup_types() < 0 ||
 	    pyrf_evsel__setup_types() < 0 ||
 	    pyrf_thread_map__setup_types() < 0 ||
-	    pyrf_cpu_map__setup_types() < 0)
+	    pyrf_cpu_map__setup_types() < 0 ||
+	    pyrf_counts_values__setup_types() < 0)
 		return module;
 
 	/* The page_size is placed in util object. */
@@ -1520,6 +1607,9 @@ PyMODINIT_FUNC PyInit_perf(void)
 	Py_INCREF(&pyrf_cpu_map__type);
 	PyModule_AddObject(module, "cpu_map", (PyObject*)&pyrf_cpu_map__type);
 
+	Py_INCREF(&pyrf_counts_values__type);
+	PyModule_AddObject(module, "counts_values", (PyObject *)&pyrf_counts_values__type);
+
 	dict = PyModule_GetDict(module);
 	if (dict == NULL)
 		goto error;
Signature · 1 line
2.49.0.1101.gccaa498523-goog
#5 lore raw

Re: [PATCH v3 4/7] perf python: Add support for perf_counts_values to return counter data

Quoted 4 lines · Ian Rogers

From: Gautam Menghani <[email protected]>

Add support for perf_counts_values struct to enable the python bindings to read and return the counter data.

This (and another one in this series) are coming from Ian, that didn't modify them, so we need a Signed-off-by: Ian, as per:

Documentation/process/submitting-patches.rst

---
Any further SoBs (Signed-off-by:'s) following the author's SoB are from people handling and transporting the patch, but were not involved in its development. SoB chains should reflect the **real** route a patch took as it was propagated to the maintainers and ultimately to Linus, with the first SoB entry signalling primary authorship of a single author.
---

I'm adding them to my local branch, please ack this,

Thanks,

- Arnaldo

Quoted 124 lines
---
 tools/perf/util/python.c | 92 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 91 insertions(+), 1 deletion(-)
tools/perf/util/python.c +91 -1
index ead3afd2d996..1cbddfe77c7c 100644
@@ -626,6 +626,92 @@ static int pyrf_thread_map__setup_types(void)
 	return PyType_Ready(&pyrf_thread_map__type);
 }
 
+struct pyrf_counts_values {
+	PyObject_HEAD
+
+	struct perf_counts_values values;
+};
+
+static const char pyrf_counts_values__doc[] = PyDoc_STR("perf counts values object.");
+
+static void pyrf_counts_values__delete(struct pyrf_counts_values *pcounts_values)
+{
+	Py_TYPE(pcounts_values)->tp_free((PyObject *)pcounts_values);
+}
+
+#define counts_values_member_def(member, ptype, help) \
+	{ #member, ptype, \
+	  offsetof(struct pyrf_counts_values, values.member), \
+	  0, help }
+
+static PyMemberDef pyrf_counts_values_members[] = {
+	counts_values_member_def(val, Py_T_ULONG, "Value of event"),
+	counts_values_member_def(ena, Py_T_ULONG, "Time for which enabled"),
+	counts_values_member_def(run, Py_T_ULONG, "Time for which running"),
+	counts_values_member_def(id, Py_T_ULONG, "Unique ID for an event"),
+	counts_values_member_def(lost, Py_T_ULONG, "Num of lost samples"),
+	{NULL}
+};
+
+static PyObject *pyrf_counts_values_get_values(struct pyrf_counts_values *self, void *closure)
+{
+	PyObject *vals = PyList_New(5);
+
+	if (!vals)
+		return NULL;
+	for (int i = 0; i < 5; i++)
+		PyList_SetItem(vals, i, PyLong_FromLong(self->values.values[i]));
+
+	return vals;
+}
+
+static int pyrf_counts_values_set_values(struct pyrf_counts_values *self, PyObject *list,
+					 void *closure)
+{
+	Py_ssize_t size;
+	PyObject *item = NULL;
+
+	if (!PyList_Check(list)) {
+		PyErr_SetString(PyExc_TypeError, "Value assigned must be a list");
+		return -1;
+	}
+
+	size = PyList_Size(list);
+	for (Py_ssize_t i = 0; i < size; i++) {
+		item = PyList_GetItem(list, i);
+		if (!PyLong_Check(item)) {
+			PyErr_SetString(PyExc_TypeError, "List members should be numbers");
+			return -1;
+		}
+		self->values.values[i] = PyLong_AsLong(item);
+	}
+
+	return 0;
+}
+
+static PyGetSetDef pyrf_counts_values_getset[] = {
+	{"values", (getter)pyrf_counts_values_get_values, (setter)pyrf_counts_values_set_values,
+		"Name field", NULL},
+	{NULL}
+};
+
+static PyTypeObject pyrf_counts_values__type = {
+	PyVarObject_HEAD_INIT(NULL, 0)
+	.tp_name	= "perf.counts_values",
+	.tp_basicsize	= sizeof(struct pyrf_counts_values),
+	.tp_dealloc	= (destructor)pyrf_counts_values__delete,
+	.tp_flags	= Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE,
+	.tp_doc		= pyrf_counts_values__doc,
+	.tp_members	= pyrf_counts_values_members,
+	.tp_getset	= pyrf_counts_values_getset,
+};
+
+static int pyrf_counts_values__setup_types(void)
+{
+	pyrf_counts_values__type.tp_new = PyType_GenericNew;
+	return PyType_Ready(&pyrf_counts_values__type);
+}
+
 struct pyrf_evsel {
 	PyObject_HEAD
 
@@ -1475,7 +1561,8 @@ PyMODINIT_FUNC PyInit_perf(void)
 	    pyrf_evlist__setup_types() < 0 ||
 	    pyrf_evsel__setup_types() < 0 ||
 	    pyrf_thread_map__setup_types() < 0 ||
-	    pyrf_cpu_map__setup_types() < 0)
+	    pyrf_cpu_map__setup_types() < 0 ||
+	    pyrf_counts_values__setup_types() < 0)
 		return module;
 
 	/* The page_size is placed in util object. */
@@ -1520,6 +1607,9 @@ PyMODINIT_FUNC PyInit_perf(void)
 	Py_INCREF(&pyrf_cpu_map__type);
 	PyModule_AddObject(module, "cpu_map", (PyObject*)&pyrf_cpu_map__type);
 
+	Py_INCREF(&pyrf_counts_values__type);
+	PyModule_AddObject(module, "counts_values", (PyObject *)&pyrf_counts_values__type);
+
 	dict = PyModule_GetDict(module);
 	if (dict == NULL)
 		goto error;
Signature · 1 line
2.49.0.1101.gccaa498523-goog
author reviewer
#6 lore raw

Re: [PATCH v3 4/7] perf python: Add support for perf_counts_values to return counter data

Quoted 20 lines · Arnaldo Carvalho de Melo
Quoted 4 lines · Ian Rogers

From: Gautam Menghani <[email protected]>

Add support for perf_counts_values struct to enable the python bindings to read and return the counter data.

This (and another one in this series) are coming from Ian, that didn't modify them, so we need a Signed-off-by: Ian, as per:

Documentation/process/submitting-patches.rst

---
Any further SoBs (Signed-off-by:'s) following the author's SoB are from people handling and transporting the patch, but were not involved in its development. SoB chains should reflect the **real** route a patch took as it was propagated to the maintainers and ultimately to Linus, with the first SoB entry signalling primary authorship of a single author.
---

Quoted 1 line

I'm adding them to my local branch, please ack this,

Ack. Thanks and sorry for the work, will try to do better next time.

Ian

Quoted 128 lines

Thanks,

- Arnaldo

Quoted 124 lines
---
 tools/perf/util/python.c | 92 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 91 insertions(+), 1 deletion(-)
tools/perf/util/python.c +91 -2
index ead3afd2d996..1cbddfe77c7c 100644
@@ -626,6 +626,92 @@ static int pyrf_thread_map__setup_types(void)
      return PyType_Ready(&pyrf_thread_map__type);
 }
 
+struct pyrf_counts_values {
+     PyObject_HEAD
+
+     struct perf_counts_values values;
+};
+
+static const char pyrf_counts_values__doc[] = PyDoc_STR("perf counts values object.");
+
+static void pyrf_counts_values__delete(struct pyrf_counts_values *pcounts_values)
+{
+     Py_TYPE(pcounts_values)->tp_free((PyObject *)pcounts_values);
+}
+
+#define counts_values_member_def(member, ptype, help) \
+     { #member, ptype, \
+       offsetof(struct pyrf_counts_values, values.member), \
+       0, help }
+
+static PyMemberDef pyrf_counts_values_members[] = {
+     counts_values_member_def(val, Py_T_ULONG, "Value of event"),
+     counts_values_member_def(ena, Py_T_ULONG, "Time for which enabled"),
+     counts_values_member_def(run, Py_T_ULONG, "Time for which running"),
+     counts_values_member_def(id, Py_T_ULONG, "Unique ID for an event"),
+     counts_values_member_def(lost, Py_T_ULONG, "Num of lost samples"),
+     {NULL}
+};
+
+static PyObject *pyrf_counts_values_get_values(struct pyrf_counts_values *self, void *closure)
+{
+     PyObject *vals = PyList_New(5);
+
+     if (!vals)
+             return NULL;
+     for (int i = 0; i < 5; i++)
+             PyList_SetItem(vals, i, PyLong_FromLong(self->values.values[i]));
+
+     return vals;
+}
+
+static int pyrf_counts_values_set_values(struct pyrf_counts_values *self, PyObject *list,
+                                      void *closure)
+{
+     Py_ssize_t size;
+     PyObject *item = NULL;
+
+     if (!PyList_Check(list)) {
+             PyErr_SetString(PyExc_TypeError, "Value assigned must be a list");
+             return -1;
+     }
+
+     size = PyList_Size(list);
+     for (Py_ssize_t i = 0; i < size; i++) {
+             item = PyList_GetItem(list, i);
+             if (!PyLong_Check(item)) {
+                     PyErr_SetString(PyExc_TypeError, "List members should be numbers");
+                     return -1;
+             }
+             self->values.values[i] = PyLong_AsLong(item);
+     }
+
+     return 0;
+}
+
+static PyGetSetDef pyrf_counts_values_getset[] = {
+     {"values", (getter)pyrf_counts_values_get_values, (setter)pyrf_counts_values_set_values,
+             "Name field", NULL},
+     {NULL}
+};
+
+static PyTypeObject pyrf_counts_values__type = {
+     PyVarObject_HEAD_INIT(NULL, 0)
+     .tp_name        = "perf.counts_values",
+     .tp_basicsize   = sizeof(struct pyrf_counts_values),
+     .tp_dealloc     = (destructor)pyrf_counts_values__delete,
+     .tp_flags       = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE,
+     .tp_doc         = pyrf_counts_values__doc,
+     .tp_members     = pyrf_counts_values_members,
+     .tp_getset      = pyrf_counts_values_getset,
+};
+
+static int pyrf_counts_values__setup_types(void)
+{
+     pyrf_counts_values__type.tp_new = PyType_GenericNew;
+     return PyType_Ready(&pyrf_counts_values__type);
+}
+
 struct pyrf_evsel {
      PyObject_HEAD
 
@@ -1475,7 +1561,8 @@ PyMODINIT_FUNC PyInit_perf(void)
          pyrf_evlist__setup_types() < 0 ||
          pyrf_evsel__setup_types() < 0 ||
          pyrf_thread_map__setup_types() < 0 ||
-         pyrf_cpu_map__setup_types() < 0)
+         pyrf_cpu_map__setup_types() < 0 ||
+         pyrf_counts_values__setup_types() < 0)
              return module;
 
      /* The page_size is placed in util object. */
@@ -1520,6 +1607,9 @@ PyMODINIT_FUNC PyInit_perf(void)
      Py_INCREF(&pyrf_cpu_map__type);
      PyModule_AddObject(module, "cpu_map", (PyObject*)&pyrf_cpu_map__type);
 
+     Py_INCREF(&pyrf_counts_values__type);
+     PyModule_AddObject(module, "counts_values", (PyObject *)&pyrf_counts_values__type);
+
      dict = PyModule_GetDict(module);
      if (dict == NULL)
              goto error;
--

2.49.0.1101.gccaa498523-goog

#7 lore raw

Re: [PATCH v3 4/7] perf python: Add support for perf_counts_values to return counter data

Quoted 22 lines · Ian Rogers
Quoted 20 lines · Arnaldo Carvalho de Melo
Quoted 4 lines · Ian Rogers

From: Gautam Menghani <[email protected]>

Add support for perf_counts_values struct to enable the python bindings to read and return the counter data.

This (and another one in this series) are coming from Ian, that didn't modify them, so we need a Signed-off-by: Ian, as per:

Documentation/process/submitting-patches.rst

---
Any further SoBs (Signed-off-by:'s) following the author's SoB are from people handling and transporting the patch, but were not involved in its development. SoB chains should reflect the **real** route a patch took as it was propagated to the maintainers and ultimately to Linus, with the first SoB entry signalling primary authorship of a single author.
---

Quoted 3 lines
Quoted 1 line

I'm adding them to my local branch, please ack this,

Ack. Thanks and sorry for the work, will try to do better next time.

Thanks for confirming,

- Arnaldo

#8 lore raw

Re: [PATCH v3 4/7] perf python: Add support for perf_counts_values to return counter data

In reply to Ian Rogers
Quoted 34 lines · Ian Rogers

From: Gautam Menghani <[email protected]>

Add support for perf_counts_values struct to enable the python bindings to read and return the counter data.

---
 tools/perf/util/python.c | 92 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 91 insertions(+), 1 deletion(-)
tools/perf/util/python.c +16 -0
index ead3afd2d996..1cbddfe77c7c 100644
@@ -626,6 +626,92 @@ static int pyrf_thread_map__setup_types(void)
 	return PyType_Ready(&pyrf_thread_map__type);
 }
 
+struct pyrf_counts_values {
+	PyObject_HEAD
+
+	struct perf_counts_values values;
+};
+
+static const char pyrf_counts_values__doc[] = PyDoc_STR("perf counts values object.");
+
+static void pyrf_counts_values__delete(struct pyrf_counts_values *pcounts_values)
+{
+	Py_TYPE(pcounts_values)->tp_free((PyObject *)pcounts_values);
+}
+
+#define counts_values_member_def(member, ptype, help) \
+	{ #member, ptype, \
+	  offsetof(struct pyrf_counts_values, values.member), \

tools/perf/util/python.c static int pyrf_thread_map__setup_types(void) · line 654

Quoted 10 lines
tools/perf/util/python.c +10 -0
+	  0, help }
+
+static PyMemberDef pyrf_counts_values_members[] = {
+	counts_values_member_def(val, Py_T_ULONG, "Value of event"),
+	counts_values_member_def(ena, Py_T_ULONG, "Time for which enabled"),
+	counts_values_member_def(run, Py_T_ULONG, "Time for which running"),
+	counts_values_member_def(id, Py_T_ULONG, "Unique ID for an event"),
+	counts_values_member_def(lost, Py_T_ULONG, "Num of lost samples"),
+	{NULL}
+};

So the above is failing on a aarch64 debian (rpi5):

acme@raspberrypi:~/git/perf-tools-next $ dpkg -S /usr/include/python3.11/structmember.h libpython3.11-dev:arm64: /usr/include/python3.11/structmember.h acme@raspberrypi:~/git/perf-tools-next $

Where it only has:

acme@raspberrypi:~/git/perf-tools-next $ grep -r Py_T_ULONG /usr/include/ acme@raspberrypi:~/git/perf-tools-next $ grep -rw Py_T_ULONG /usr/include/ acme@raspberrypi:~/git/perf-tools-next $ grep -rw T_ULONG /usr/include/
/usr/include/python3.11/structmember.h:#define T_ULONG 12
acme@raspberrypi:~/git/perf-tools-next $

while on fedora 42 x86_64:

⬢ [acme@toolbx perf-tools-next]$ grep -rw Py_T_ULONG /usr/include/
/usr/include/python3.13/descrobject.h:#define Py_T_ULONG 12
/usr/include/python3.13/structmember.h:#define T_ULONG Py_T_ULONG
⬢ [acme@toolbx perf-tools-next]$

So I'm making it use the T_ULONG and others as all the other PyMemberDef arrays in tools/perf/util/python.c, ok?

- Arnaldo

Quoted 84 lines
tools/perf/util/python.c +64 -1
+static PyObject *pyrf_counts_values_get_values(struct pyrf_counts_values *self, void *closure)
+{
+	PyObject *vals = PyList_New(5);
+
+	if (!vals)
+		return NULL;
+	for (int i = 0; i < 5; i++)
+		PyList_SetItem(vals, i, PyLong_FromLong(self->values.values[i]));
+
+	return vals;
+}
+
+static int pyrf_counts_values_set_values(struct pyrf_counts_values *self, PyObject *list,
+					 void *closure)
+{
+	Py_ssize_t size;
+	PyObject *item = NULL;
+
+	if (!PyList_Check(list)) {
+		PyErr_SetString(PyExc_TypeError, "Value assigned must be a list");
+		return -1;
+	}
+
+	size = PyList_Size(list);
+	for (Py_ssize_t i = 0; i < size; i++) {
+		item = PyList_GetItem(list, i);
+		if (!PyLong_Check(item)) {
+			PyErr_SetString(PyExc_TypeError, "List members should be numbers");
+			return -1;
+		}
+		self->values.values[i] = PyLong_AsLong(item);
+	}
+
+	return 0;
+}
+
+static PyGetSetDef pyrf_counts_values_getset[] = {
+	{"values", (getter)pyrf_counts_values_get_values, (setter)pyrf_counts_values_set_values,
+		"Name field", NULL},
+	{NULL}
+};
+
+static PyTypeObject pyrf_counts_values__type = {
+	PyVarObject_HEAD_INIT(NULL, 0)
+	.tp_name	= "perf.counts_values",
+	.tp_basicsize	= sizeof(struct pyrf_counts_values),
+	.tp_dealloc	= (destructor)pyrf_counts_values__delete,
+	.tp_flags	= Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE,
+	.tp_doc		= pyrf_counts_values__doc,
+	.tp_members	= pyrf_counts_values_members,
+	.tp_getset	= pyrf_counts_values_getset,
+};
+
+static int pyrf_counts_values__setup_types(void)
+{
+	pyrf_counts_values__type.tp_new = PyType_GenericNew;
+	return PyType_Ready(&pyrf_counts_values__type);
+}
+
 struct pyrf_evsel {
 	PyObject_HEAD
 
@@ -1475,7 +1561,8 @@ PyMODINIT_FUNC PyInit_perf(void)
 	    pyrf_evlist__setup_types() < 0 ||
 	    pyrf_evsel__setup_types() < 0 ||
 	    pyrf_thread_map__setup_types() < 0 ||
-	    pyrf_cpu_map__setup_types() < 0)
+	    pyrf_cpu_map__setup_types() < 0 ||
+	    pyrf_counts_values__setup_types() < 0)
 		return module;
 
 	/* The page_size is placed in util object. */
@@ -1520,6 +1607,9 @@ PyMODINIT_FUNC PyInit_perf(void)
 	Py_INCREF(&pyrf_cpu_map__type);
 	PyModule_AddObject(module, "cpu_map", (PyObject*)&pyrf_cpu_map__type);
 
+	Py_INCREF(&pyrf_counts_values__type);
+	PyModule_AddObject(module, "counts_values", (PyObject *)&pyrf_counts_values__type);
+
 	dict = PyModule_GetDict(module);
 	if (dict == NULL)
 		goto error;
Signature · 1 line
2.49.0.1101.gccaa498523-goog
author reviewer
#9 lore raw

Re: [PATCH v3 4/7] perf python: Add support for perf_counts_values to return counter data

Quoted 63 lines · Arnaldo Carvalho de Melo
Quoted 34 lines · Ian Rogers

From: Gautam Menghani <[email protected]>

Add support for perf_counts_values struct to enable the python bindings to read and return the counter data.

---
 tools/perf/util/python.c | 92 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 91 insertions(+), 1 deletion(-)
tools/perf/util/python.c +16 -0
index ead3afd2d996..1cbddfe77c7c 100644
@@ -626,6 +626,92 @@ static int pyrf_thread_map__setup_types(void)
      return PyType_Ready(&pyrf_thread_map__type);
 }
 
+struct pyrf_counts_values {
+     PyObject_HEAD
+
+     struct perf_counts_values values;
+};
+
+static const char pyrf_counts_values__doc[] = PyDoc_STR("perf counts values object.");
+
+static void pyrf_counts_values__delete(struct pyrf_counts_values *pcounts_values)
+{
+     Py_TYPE(pcounts_values)->tp_free((PyObject *)pcounts_values);
+}
+
+#define counts_values_member_def(member, ptype, help) \
+     { #member, ptype, \
+       offsetof(struct pyrf_counts_values, values.member), \

tools/perf/util/python.c static int pyrf_thread_map__setup_types(void) · line 654

Quoted 10 lines
tools/perf/util/python.c +10 -0
+       0, help }
+
+static PyMemberDef pyrf_counts_values_members[] = {
+     counts_values_member_def(val, Py_T_ULONG, "Value of event"),
+     counts_values_member_def(ena, Py_T_ULONG, "Time for which enabled"),
+     counts_values_member_def(run, Py_T_ULONG, "Time for which running"),
+     counts_values_member_def(id, Py_T_ULONG, "Unique ID for an event"),
+     counts_values_member_def(lost, Py_T_ULONG, "Num of lost samples"),
+     {NULL}
+};

So the above is failing on a aarch64 debian (rpi5):

acme@raspberrypi:~/git/perf-tools-next $ dpkg -S /usr/include/python3.11/structmember.h libpython3.11-dev:arm64: /usr/include/python3.11/structmember.h acme@raspberrypi:~/git/perf-tools-next $

Where it only has:

acme@raspberrypi:~/git/perf-tools-next $ grep -r Py_T_ULONG /usr/include/ acme@raspberrypi:~/git/perf-tools-next $ grep -rw Py_T_ULONG /usr/include/ acme@raspberrypi:~/git/perf-tools-next $ grep -rw T_ULONG /usr/include/
/usr/include/python3.11/structmember.h:#define T_ULONG 12
acme@raspberrypi:~/git/perf-tools-next $

while on fedora 42 x86_64:

Quoted 7 lines

⬢ [acme@toolbx perf-tools-next]$ grep -rw Py_T_ULONG /usr/include/
/usr/include/python3.13/descrobject.h:#define Py_T_ULONG 12
/usr/include/python3.13/structmember.h:#define T_ULONG Py_T_ULONG
⬢ [acme@toolbx perf-tools-next]$

So I'm making it use the T_ULONG and others as all the other PyMemberDef arrays in tools/perf/util/python.c, ok?

The fix makes sense to me. Checking the documentation it seems Py_T_ULONG is preferred:
https://docs.python.org/3/c-api/structures.html#member-types
perhaps we can add:

```
#ifndef Py_T_ULONG
#define Py_T_ULONG T_ULONG
#endif
```

so that we use the approach matching the documentation while fixing the RaspberryPi issue.

Thanks,
Ian

Quoted 86 lines

- Arnaldo

Quoted 84 lines
tools/perf/util/python.c +64 -2
+static PyObject *pyrf_counts_values_get_values(struct pyrf_counts_values *self, void *closure)
+{
+     PyObject *vals = PyList_New(5);
+
+     if (!vals)
+             return NULL;
+     for (int i = 0; i < 5; i++)
+             PyList_SetItem(vals, i, PyLong_FromLong(self->values.values[i]));
+
+     return vals;
+}
+
+static int pyrf_counts_values_set_values(struct pyrf_counts_values *self, PyObject *list,
+                                      void *closure)
+{
+     Py_ssize_t size;
+     PyObject *item = NULL;
+
+     if (!PyList_Check(list)) {
+             PyErr_SetString(PyExc_TypeError, "Value assigned must be a list");
+             return -1;
+     }
+
+     size = PyList_Size(list);
+     for (Py_ssize_t i = 0; i < size; i++) {
+             item = PyList_GetItem(list, i);
+             if (!PyLong_Check(item)) {
+                     PyErr_SetString(PyExc_TypeError, "List members should be numbers");
+                     return -1;
+             }
+             self->values.values[i] = PyLong_AsLong(item);
+     }
+
+     return 0;
+}
+
+static PyGetSetDef pyrf_counts_values_getset[] = {
+     {"values", (getter)pyrf_counts_values_get_values, (setter)pyrf_counts_values_set_values,
+             "Name field", NULL},
+     {NULL}
+};
+
+static PyTypeObject pyrf_counts_values__type = {
+     PyVarObject_HEAD_INIT(NULL, 0)
+     .tp_name        = "perf.counts_values",
+     .tp_basicsize   = sizeof(struct pyrf_counts_values),
+     .tp_dealloc     = (destructor)pyrf_counts_values__delete,
+     .tp_flags       = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE,
+     .tp_doc         = pyrf_counts_values__doc,
+     .tp_members     = pyrf_counts_values_members,
+     .tp_getset      = pyrf_counts_values_getset,
+};
+
+static int pyrf_counts_values__setup_types(void)
+{
+     pyrf_counts_values__type.tp_new = PyType_GenericNew;
+     return PyType_Ready(&pyrf_counts_values__type);
+}
+
 struct pyrf_evsel {
      PyObject_HEAD
 
@@ -1475,7 +1561,8 @@ PyMODINIT_FUNC PyInit_perf(void)
          pyrf_evlist__setup_types() < 0 ||
          pyrf_evsel__setup_types() < 0 ||
          pyrf_thread_map__setup_types() < 0 ||
-         pyrf_cpu_map__setup_types() < 0)
+         pyrf_cpu_map__setup_types() < 0 ||
+         pyrf_counts_values__setup_types() < 0)
              return module;
 
      /* The page_size is placed in util object. */
@@ -1520,6 +1607,9 @@ PyMODINIT_FUNC PyInit_perf(void)
      Py_INCREF(&pyrf_cpu_map__type);
      PyModule_AddObject(module, "cpu_map", (PyObject*)&pyrf_cpu_map__type);
 
+     Py_INCREF(&pyrf_counts_values__type);
+     PyModule_AddObject(module, "counts_values", (PyObject *)&pyrf_counts_values__type);
+
      dict = PyModule_GetDict(module);
      if (dict == NULL)
              goto error;
--

2.49.0.1101.gccaa498523-goog

#10 lore raw

Re: [PATCH v3 4/7] perf python: Add support for perf_counts_values to return counter data

Quoted 77 lines · Ian Rogers
Quoted 63 lines · Arnaldo Carvalho de Melo
Quoted 34 lines · Ian Rogers

From: Gautam Menghani <[email protected]>

Add support for perf_counts_values struct to enable the python bindings to read and return the counter data.

---
 tools/perf/util/python.c | 92 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 91 insertions(+), 1 deletion(-)
tools/perf/util/python.c +16 -0
index ead3afd2d996..1cbddfe77c7c 100644
@@ -626,6 +626,92 @@ static int pyrf_thread_map__setup_types(void)
      return PyType_Ready(&pyrf_thread_map__type);
 }
 
+struct pyrf_counts_values {
+     PyObject_HEAD
+
+     struct perf_counts_values values;
+};
+
+static const char pyrf_counts_values__doc[] = PyDoc_STR("perf counts values object.");
+
+static void pyrf_counts_values__delete(struct pyrf_counts_values *pcounts_values)
+{
+     Py_TYPE(pcounts_values)->tp_free((PyObject *)pcounts_values);
+}
+
+#define counts_values_member_def(member, ptype, help) \
+     { #member, ptype, \
+       offsetof(struct pyrf_counts_values, values.member), \

tools/perf/util/python.c static int pyrf_thread_map__setup_types(void) · line 654

Quoted 10 lines
tools/perf/util/python.c +10 -0
+       0, help }
+
+static PyMemberDef pyrf_counts_values_members[] = {
+     counts_values_member_def(val, Py_T_ULONG, "Value of event"),
+     counts_values_member_def(ena, Py_T_ULONG, "Time for which enabled"),
+     counts_values_member_def(run, Py_T_ULONG, "Time for which running"),
+     counts_values_member_def(id, Py_T_ULONG, "Unique ID for an event"),
+     counts_values_member_def(lost, Py_T_ULONG, "Num of lost samples"),
+     {NULL}
+};

So the above is failing on a aarch64 debian (rpi5):

acme@raspberrypi:~/git/perf-tools-next $ dpkg -S /usr/include/python3.11/structmember.h libpython3.11-dev:arm64: /usr/include/python3.11/structmember.h acme@raspberrypi:~/git/perf-tools-next $

Where it only has:

acme@raspberrypi:~/git/perf-tools-next $ grep -r Py_T_ULONG /usr/include/ acme@raspberrypi:~/git/perf-tools-next $ grep -rw Py_T_ULONG /usr/include/ acme@raspberrypi:~/git/perf-tools-next $ grep -rw T_ULONG /usr/include/
/usr/include/python3.11/structmember.h:#define T_ULONG 12
acme@raspberrypi:~/git/perf-tools-next $

while on fedora 42 x86_64:

Quoted 7 lines

⬢ [acme@toolbx perf-tools-next]$ grep -rw Py_T_ULONG /usr/include/
/usr/include/python3.13/descrobject.h:#define Py_T_ULONG 12
/usr/include/python3.13/structmember.h:#define T_ULONG Py_T_ULONG
⬢ [acme@toolbx perf-tools-next]$

So I'm making it use the T_ULONG and others as all the other PyMemberDef arrays in tools/perf/util/python.c, ok?

The fix makes sense to me. Checking the documentation it seems Py_T_ULONG is preferred:
https://docs.python.org/3/c-api/structures.html#member-types
perhaps we can add:

Quoted 7 lines
```
#ifndef Py_T_ULONG
#define Py_T_ULONG T_ULONG
#endif
```

so that we use the approach matching the documentation while fixing the RaspberryPi issue.

That can be done as a followup, as there are lots of preexisting usage for struct method definitions.

- Arnaldo

Quoted 89 lines

Thanks,
Ian

Quoted 86 lines

- Arnaldo

Quoted 84 lines
tools/perf/util/python.c +64 -2
+static PyObject *pyrf_counts_values_get_values(struct pyrf_counts_values *self, void *closure)
+{
+     PyObject *vals = PyList_New(5);
+
+     if (!vals)
+             return NULL;
+     for (int i = 0; i < 5; i++)
+             PyList_SetItem(vals, i, PyLong_FromLong(self->values.values[i]));
+
+     return vals;
+}
+
+static int pyrf_counts_values_set_values(struct pyrf_counts_values *self, PyObject *list,
+                                      void *closure)
+{
+     Py_ssize_t size;
+     PyObject *item = NULL;
+
+     if (!PyList_Check(list)) {
+             PyErr_SetString(PyExc_TypeError, "Value assigned must be a list");
+             return -1;
+     }
+
+     size = PyList_Size(list);
+     for (Py_ssize_t i = 0; i < size; i++) {
+             item = PyList_GetItem(list, i);
+             if (!PyLong_Check(item)) {
+                     PyErr_SetString(PyExc_TypeError, "List members should be numbers");
+                     return -1;
+             }
+             self->values.values[i] = PyLong_AsLong(item);
+     }
+
+     return 0;
+}
+
+static PyGetSetDef pyrf_counts_values_getset[] = {
+     {"values", (getter)pyrf_counts_values_get_values, (setter)pyrf_counts_values_set_values,
+             "Name field", NULL},
+     {NULL}
+};
+
+static PyTypeObject pyrf_counts_values__type = {
+     PyVarObject_HEAD_INIT(NULL, 0)
+     .tp_name        = "perf.counts_values",
+     .tp_basicsize   = sizeof(struct pyrf_counts_values),
+     .tp_dealloc     = (destructor)pyrf_counts_values__delete,
+     .tp_flags       = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE,
+     .tp_doc         = pyrf_counts_values__doc,
+     .tp_members     = pyrf_counts_values_members,
+     .tp_getset      = pyrf_counts_values_getset,
+};
+
+static int pyrf_counts_values__setup_types(void)
+{
+     pyrf_counts_values__type.tp_new = PyType_GenericNew;
+     return PyType_Ready(&pyrf_counts_values__type);
+}
+
 struct pyrf_evsel {
      PyObject_HEAD
 
@@ -1475,7 +1561,8 @@ PyMODINIT_FUNC PyInit_perf(void)
          pyrf_evlist__setup_types() < 0 ||
          pyrf_evsel__setup_types() < 0 ||
          pyrf_thread_map__setup_types() < 0 ||
-         pyrf_cpu_map__setup_types() < 0)
+         pyrf_cpu_map__setup_types() < 0 ||
+         pyrf_counts_values__setup_types() < 0)
              return module;
 
      /* The page_size is placed in util object. */
@@ -1520,6 +1607,9 @@ PyMODINIT_FUNC PyInit_perf(void)
      Py_INCREF(&pyrf_cpu_map__type);
      PyModule_AddObject(module, "cpu_map", (PyObject*)&pyrf_cpu_map__type);
 
+     Py_INCREF(&pyrf_counts_values__type);
+     PyModule_AddObject(module, "counts_values", (PyObject *)&pyrf_counts_values__type);
+
      dict = PyModule_GetDict(module);
      if (dict == NULL)
              goto error;
--

2.49.0.1101.gccaa498523-goog

#11 lore raw

Re: [PATCH v3 4/7] perf python: Add support for perf_counts_values to return counter data

Quoted 86 lines · Arnaldo Carvalho de Melo
Quoted 77 lines · Ian Rogers
Quoted 63 lines · Arnaldo Carvalho de Melo
Quoted 34 lines · Ian Rogers

From: Gautam Menghani <[email protected]>

Add support for perf_counts_values struct to enable the python bindings to read and return the counter data.

---
 tools/perf/util/python.c | 92 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 91 insertions(+), 1 deletion(-)
tools/perf/util/python.c +16 -0
index ead3afd2d996..1cbddfe77c7c 100644
@@ -626,6 +626,92 @@ static int pyrf_thread_map__setup_types(void)
      return PyType_Ready(&pyrf_thread_map__type);
 }
 
+struct pyrf_counts_values {
+     PyObject_HEAD
+
+     struct perf_counts_values values;
+};
+
+static const char pyrf_counts_values__doc[] = PyDoc_STR("perf counts values object.");
+
+static void pyrf_counts_values__delete(struct pyrf_counts_values *pcounts_values)
+{
+     Py_TYPE(pcounts_values)->tp_free((PyObject *)pcounts_values);
+}
+
+#define counts_values_member_def(member, ptype, help) \
+     { #member, ptype, \
+       offsetof(struct pyrf_counts_values, values.member), \

tools/perf/util/python.c static int pyrf_thread_map__setup_types(void) · line 654

Quoted 10 lines
tools/perf/util/python.c +10 -0
+       0, help }
+
+static PyMemberDef pyrf_counts_values_members[] = {
+     counts_values_member_def(val, Py_T_ULONG, "Value of event"),
+     counts_values_member_def(ena, Py_T_ULONG, "Time for which enabled"),
+     counts_values_member_def(run, Py_T_ULONG, "Time for which running"),
+     counts_values_member_def(id, Py_T_ULONG, "Unique ID for an event"),
+     counts_values_member_def(lost, Py_T_ULONG, "Num of lost samples"),
+     {NULL}
+};

So the above is failing on a aarch64 debian (rpi5):

acme@raspberrypi:~/git/perf-tools-next $ dpkg -S /usr/include/python3.11/structmember.h libpython3.11-dev:arm64: /usr/include/python3.11/structmember.h acme@raspberrypi:~/git/perf-tools-next $

Where it only has:

acme@raspberrypi:~/git/perf-tools-next $ grep -r Py_T_ULONG /usr/include/ acme@raspberrypi:~/git/perf-tools-next $ grep -rw Py_T_ULONG /usr/include/ acme@raspberrypi:~/git/perf-tools-next $ grep -rw T_ULONG /usr/include/
/usr/include/python3.11/structmember.h:#define T_ULONG 12
acme@raspberrypi:~/git/perf-tools-next $

while on fedora 42 x86_64:

Quoted 7 lines

⬢ [acme@toolbx perf-tools-next]$ grep -rw Py_T_ULONG /usr/include/
/usr/include/python3.13/descrobject.h:#define Py_T_ULONG 12
/usr/include/python3.13/structmember.h:#define T_ULONG Py_T_ULONG
⬢ [acme@toolbx perf-tools-next]$

So I'm making it use the T_ULONG and others as all the other PyMemberDef arrays in tools/perf/util/python.c, ok?

The fix makes sense to me. Checking the documentation it seems Py_T_ULONG is preferred:
https://docs.python.org/3/c-api/structures.html#member-types
perhaps we can add:

Quoted 7 lines
```
#ifndef Py_T_ULONG
#define Py_T_ULONG T_ULONG
#endif
```

so that we use the approach matching the documentation while fixing the RaspberryPi issue.

Quoted 2 lines

That can be done as a followup, as there are lots of preexisting usage for struct method definitions.

And there is one other issue:

LINK /tmp/build/perf/perf
GEN /tmp/build/perf/python/perf.cpython-36m-x86_64-linux-gnu.so
/git/perf-6.15.0-rc7/tools/perf/util/python.c:653:7: error: missing field 'type' initializer [-Werror,-Wmissing-field-initializers]
653 | {NULL}
| ^
/git/perf-6.15.0-rc7/tools/perf/util/python.c:695:7: error: missing field 'get' initializer [-Werror,-Wmissing-field-initializers]
695 | {NULL}
| ^
2 errors generated.
error: command 'clang' failed with exit status 1
cp: cannot stat '/tmp/build/perf/python_ext_build/lib/perf*.so': No such file or directory

I'll look at this after dinner.

- Arnaldo

#12 lore raw

Re: [PATCH v3 4/7] perf python: Add support for perf_counts_values to return counter data

Quoted 105 lines · Arnaldo Carvalho de Melo
Quoted 86 lines · Arnaldo Carvalho de Melo
Quoted 77 lines · Ian Rogers
Quoted 63 lines · Arnaldo Carvalho de Melo
Quoted 34 lines · Ian Rogers

From: Gautam Menghani <[email protected]>

Add support for perf_counts_values struct to enable the python bindings to read and return the counter data.

---
 tools/perf/util/python.c | 92 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 91 insertions(+), 1 deletion(-)
tools/perf/util/python.c +16 -0
index ead3afd2d996..1cbddfe77c7c 100644
@@ -626,6 +626,92 @@ static int pyrf_thread_map__setup_types(void)
      return PyType_Ready(&pyrf_thread_map__type);
 }
 
+struct pyrf_counts_values {
+     PyObject_HEAD
+
+     struct perf_counts_values values;
+};
+
+static const char pyrf_counts_values__doc[] = PyDoc_STR("perf counts values object.");
+
+static void pyrf_counts_values__delete(struct pyrf_counts_values *pcounts_values)
+{
+     Py_TYPE(pcounts_values)->tp_free((PyObject *)pcounts_values);
+}
+
+#define counts_values_member_def(member, ptype, help) \
+     { #member, ptype, \
+       offsetof(struct pyrf_counts_values, values.member), \

tools/perf/util/python.c static int pyrf_thread_map__setup_types(void) · line 654

Quoted 10 lines
tools/perf/util/python.c +10 -0
+       0, help }
+
+static PyMemberDef pyrf_counts_values_members[] = {
+     counts_values_member_def(val, Py_T_ULONG, "Value of event"),
+     counts_values_member_def(ena, Py_T_ULONG, "Time for which enabled"),
+     counts_values_member_def(run, Py_T_ULONG, "Time for which running"),
+     counts_values_member_def(id, Py_T_ULONG, "Unique ID for an event"),
+     counts_values_member_def(lost, Py_T_ULONG, "Num of lost samples"),
+     {NULL}
+};

So the above is failing on a aarch64 debian (rpi5):

acme@raspberrypi:~/git/perf-tools-next $ dpkg -S /usr/include/python3.11/structmember.h libpython3.11-dev:arm64: /usr/include/python3.11/structmember.h acme@raspberrypi:~/git/perf-tools-next $

Where it only has:

acme@raspberrypi:~/git/perf-tools-next $ grep -r Py_T_ULONG /usr/include/ acme@raspberrypi:~/git/perf-tools-next $ grep -rw Py_T_ULONG /usr/include/ acme@raspberrypi:~/git/perf-tools-next $ grep -rw T_ULONG /usr/include/
/usr/include/python3.11/structmember.h:#define T_ULONG 12
acme@raspberrypi:~/git/perf-tools-next $

while on fedora 42 x86_64:

Quoted 7 lines

⬢ [acme@toolbx perf-tools-next]$ grep -rw Py_T_ULONG /usr/include/
/usr/include/python3.13/descrobject.h:#define Py_T_ULONG 12
/usr/include/python3.13/structmember.h:#define T_ULONG Py_T_ULONG
⬢ [acme@toolbx perf-tools-next]$

So I'm making it use the T_ULONG and others as all the other PyMemberDef arrays in tools/perf/util/python.c, ok?

The fix makes sense to me. Checking the documentation it seems Py_T_ULONG is preferred:
https://docs.python.org/3/c-api/structures.html#member-types
perhaps we can add:

Quoted 7 lines
```
#ifndef Py_T_ULONG
#define Py_T_ULONG T_ULONG
#endif
```

so that we use the approach matching the documentation while fixing the RaspberryPi issue.

Quoted 2 lines

That can be done as a followup, as there are lots of preexisting usage for struct method definitions.

And there is one other issue:

LINK /tmp/build/perf/perf
GEN /tmp/build/perf/python/perf.cpython-36m-x86_64-linux-gnu.so
/git/perf-6.15.0-rc7/tools/perf/util/python.c:653:7: error: missing field 'type' initializer [-Werror,-Wmissing-field-initializers]
653 | {NULL}
| ^
/git/perf-6.15.0-rc7/tools/perf/util/python.c:695:7: error: missing field 'get' initializer [-Werror,-Wmissing-field-initializers]
695 | {NULL}
| ^
2 errors generated.
error: command 'clang' failed with exit status 1
cp: cannot stat '/tmp/build/perf/python_ext_build/lib/perf*.so': No such file or directory

Quoted 1 line

I'll look at this after dinner.

Doing the same thing as for all the other PyMemberDef arrays makes it pass that hurdle:

⬢ [acme@toolbx perf-tools-next]$ git diff

tools/perf/util/python.c +1 -1
index cb5a76674a5f1078..60ff12e90b91f97c 100644
@@ -650,7 +650,7 @@ static PyMemberDef pyrf_counts_values_members[] = {
        counts_values_member_def(run, T_ULONG, "Time for which running"),
        counts_values_member_def(id, T_ULONG, "Unique ID for an event"),
        counts_values_member_def(lost, T_ULONG, "Num of lost samples"),
-       {NULL}
+       { .name = NULL, },
 };
 
 static PyObject *pyrf_counts_values_get_values(struct pyrf_counts_values *self, void *closure)

Then there is:

GEN /tmp/build/perf/python/perf.cpython-36m-x86_64-linux-gnu.so
/git/perf-6.15.0-rc7/tools/perf/util/python.c:695:8: error: missing field 'get' initializer [-Werror,-Wmissing-field-initializers]
695 | {NULL,}
| ^
1 error generated.
error: command 'clang' failed with exit status 1

I tried with the ending NULL, but it wasn't enough, so I'm going with:

⬢ [acme@toolbx perf-tools-next]$ git diff

tools/perf/util/python.c +1 -1
index 5fa113daf4488120..21e2da1ec0c6342c 100644
@@ -692,7 +692,7 @@ static int pyrf_counts_values_set_values(struct pyrf_counts_values *self, PyObje
 static PyGetSetDef pyrf_counts_values_getset[] = {
        {"values", (getter)pyrf_counts_values_get_values, (setter)pyrf_counts_values_set_values,
                "Name field", NULL},
-       {NULL,}
+       { .name = NULL, },
 };
 
 static PyTypeObject pyrf_counts_values__type = {

⬢ [acme@toolbx perf-tools-next]$

- Arnaldo

#13 lore raw

Re: [PATCH v3 4/7] perf python: Add support for perf_counts_values to return counter data

Hi Arnaldo,

Thanks for fixing this, I should have looked at the other initializations in the file.

Thanks,
Gautam

author reviewer
#14 lore raw

PATCH v3 5/7 perf python: Add evsel read method

In reply to Ian Rogers

From: Gautam Menghani <[email protected]>

Add the evsel read method to enable python to read counter data for the given evsel.

---
Ian modified from v2 to make the API take a CPU and thread then compute from these the appropriate indices. This was discussed as the preferred API with Arnaldo:
https://lore.kernel.org/linux-perf-users/[email protected]/

---
 tools/perf/util/python.c | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
tools/perf/util/python.c +37 -0
index 1cbddfe77c7c..281e706e102d 100644
@@ -888,6 +888,37 @@ static PyObject *pyrf_evsel__threads(struct pyrf_evsel *pevsel)
 	return (PyObject *)pthread_map;
 }
 
+static PyObject *pyrf_evsel__read(struct pyrf_evsel *pevsel,
+				  PyObject *args, PyObject *kwargs)
+{
+	struct evsel *evsel = &pevsel->evsel;
+	int cpu = 0, cpu_idx, thread = 0, thread_idx;
+	struct perf_counts_values counts;
+	struct pyrf_counts_values *count_values = PyObject_New(struct pyrf_counts_values,
+							       &pyrf_counts_values__type);
+
+	if (!count_values)
+		return NULL;
+
+	if (!PyArg_ParseTuple(args, "ii", &cpu, &thread))
+		return NULL;
+
+	cpu_idx = perf_cpu_map__idx(evsel->core.cpus, (struct perf_cpu){.cpu = cpu});
+	if (cpu_idx < 0) {
+		PyErr_Format(PyExc_TypeError, "CPU %d is not part of evsel's CPUs", cpu);
+		return NULL;
+	}
+	thread_idx = perf_thread_map__idx(evsel->core.threads, thread);
+	if (cpu_idx < 0) {
+		PyErr_Format(PyExc_TypeError, "Thread %d is not part of evsel's threads",
+			     thread);
+		return NULL;
+	}
+	perf_evsel__read(&(evsel->core), cpu_idx, thread_idx, &counts);
+	count_values->values = counts;
+	return (PyObject *)count_values;
+}
+
 static PyObject *pyrf_evsel__str(PyObject *self)
 {
 	struct pyrf_evsel *pevsel = (void *)self;
@@ -918,6 +949,12 @@ static PyMethodDef pyrf_evsel__methods[] = {
 		.ml_flags = METH_NOARGS,
 		.ml_doc	  = PyDoc_STR("threads the event is to be used with.")
 	},
+	{
+		.ml_name  = "read",
+		.ml_meth  = (PyCFunction)pyrf_evsel__read,
+		.ml_flags = METH_VARARGS | METH_KEYWORDS,
+		.ml_doc	  = PyDoc_STR("read counters")
+	},
 	{ .ml_name = NULL, }
 };
 
Signature · 1 line
2.49.0.1101.gccaa498523-goog
#15 lore raw

Re: [PATCH v3 5/7] perf python: Add evsel read method

Quoted 9 lines · Ian Rogers

From: Gautam Menghani <[email protected]>

Add the evsel read method to enable python to read counter data for the given evsel.

---
Ian modified from v2 to make the API take a CPU and thread then

I'll add this note right before your Signed-off-by, with a link to the discussion, i.e. as:

[ make the API take a CPU and thread then compute from these the appropriate indices.

Quoted 65 lines

compute from these the appropriate indices. This was discussed as the preferred API with Arnaldo:
https://lore.kernel.org/linux-perf-users/[email protected]/

---
 tools/perf/util/python.c | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
tools/perf/util/python.c +37 -0
index 1cbddfe77c7c..281e706e102d 100644
@@ -888,6 +888,37 @@ static PyObject *pyrf_evsel__threads(struct pyrf_evsel *pevsel)
 	return (PyObject *)pthread_map;
 }
 
+static PyObject *pyrf_evsel__read(struct pyrf_evsel *pevsel,
+				  PyObject *args, PyObject *kwargs)
+{
+	struct evsel *evsel = &pevsel->evsel;
+	int cpu = 0, cpu_idx, thread = 0, thread_idx;
+	struct perf_counts_values counts;
+	struct pyrf_counts_values *count_values = PyObject_New(struct pyrf_counts_values,
+							       &pyrf_counts_values__type);
+
+	if (!count_values)
+		return NULL;
+
+	if (!PyArg_ParseTuple(args, "ii", &cpu, &thread))
+		return NULL;
+
+	cpu_idx = perf_cpu_map__idx(evsel->core.cpus, (struct perf_cpu){.cpu = cpu});
+	if (cpu_idx < 0) {
+		PyErr_Format(PyExc_TypeError, "CPU %d is not part of evsel's CPUs", cpu);
+		return NULL;
+	}
+	thread_idx = perf_thread_map__idx(evsel->core.threads, thread);
+	if (cpu_idx < 0) {
+		PyErr_Format(PyExc_TypeError, "Thread %d is not part of evsel's threads",
+			     thread);
+		return NULL;
+	}
+	perf_evsel__read(&(evsel->core), cpu_idx, thread_idx, &counts);
+	count_values->values = counts;
+	return (PyObject *)count_values;
+}
+
 static PyObject *pyrf_evsel__str(PyObject *self)
 {
 	struct pyrf_evsel *pevsel = (void *)self;
@@ -918,6 +949,12 @@ static PyMethodDef pyrf_evsel__methods[] = {
 		.ml_flags = METH_NOARGS,
 		.ml_doc	  = PyDoc_STR("threads the event is to be used with.")
 	},
+	{
+		.ml_name  = "read",
+		.ml_meth  = (PyCFunction)pyrf_evsel__read,
+		.ml_flags = METH_VARARGS | METH_KEYWORDS,
+		.ml_doc	  = PyDoc_STR("read counters")
+	},
 	{ .ml_name = NULL, }
 };
 
Signature · 1 line
2.49.0.1101.gccaa498523-goog
author reviewer
#16 lore raw

PATCH v3 6/7 perf python: Add evlist close support

In reply to Ian Rogers

From: Gautam Menghani <[email protected]>

Add support for the evlist close function.

---
 tools/perf/util/python.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
tools/perf/util/python.c +16 -0
index 281e706e102d..cca744d51349 100644
@@ -1210,6 +1210,16 @@ static PyObject *pyrf_evlist__open(struct pyrf_evlist *pevlist,
 	return Py_None;
 }
 
+static PyObject *pyrf_evlist__close(struct pyrf_evlist *pevlist)
+{
+	struct evlist *evlist = &pevlist->evlist;
+
+	evlist__close(evlist);
+
+	Py_INCREF(Py_None);
+	return Py_None;
+}
+
 static PyObject *pyrf_evlist__config(struct pyrf_evlist *pevlist)
 {
 	struct record_opts opts = {
@@ -1268,6 +1278,12 @@ static PyMethodDef pyrf_evlist__methods[] = {
 		.ml_flags = METH_VARARGS | METH_KEYWORDS,
 		.ml_doc	  = PyDoc_STR("open the file descriptors.")
 	},
+	{
+		.ml_name  = "close",
+		.ml_meth  = (PyCFunction)pyrf_evlist__close,
+		.ml_flags = METH_NOARGS,
+		.ml_doc	  = PyDoc_STR("close the file descriptors.")
+	},
 	{
 		.ml_name  = "poll",
 		.ml_meth  = (PyCFunction)pyrf_evlist__poll,
Signature · 1 line
2.49.0.1101.gccaa498523-goog
author reviewer
#17 lore raw

PATCH v3 7/7 perf python: Add counting.py as example for counting perf events

In reply to Ian Rogers

From: Gautam Menghani <[email protected]>

Add counting.py - a python version of counting.c to demonstrate measuring and reading of counts for given perf events.

---
Ian modified from v2 to make the API take a CPU and thread then compute from these the appropriate indices. This was discussed as the preferred API with Arnaldo:
https://lore.kernel.org/linux-perf-users/[email protected]/
The use of a thread_map and cpu_map was also removed to make the code cleaner, instead the cpus and threads of the parsed evsel are used. Support for command line events is also added. The indent is reduced from 8 to 4 to match the preferred python PEP8 indent.

---
 tools/perf/python/counting.py | 36 +++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
 create mode 100755 tools/perf/python/counting.py
tools/perf/python/counting.py new file +36 -0
new file mode 100755
index 000000000000..02121d2bb11d
@@ -0,0 +1,36 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+# -*- python -*-
+# -*- coding: utf-8 -*-
+
+import argparse
+import perf
+
+def main(event: str):
+    evlist = perf.parse_events(event)
+
+    for evsel in evlist:
+        evsel.read_format = perf.FORMAT_TOTAL_TIME_ENABLED | perf.FORMAT_TOTAL_TIME_RUNNING
+
+    evlist.open()
+    evlist.enable()
+
+    count = 100000
+    while count > 0:
+        count -= 1
+
+    evlist.disable()
+
+    for evsel in evlist:
+        for cpu in evsel.cpus():
+            for thread in evsel.threads():
+                counts = evsel.read(cpu, thread)
+                print(f"For {evsel} val: {counts.val} enable: {counts.ena} run: {counts.run}")
+
+    evlist.close()
+
+if __name__ == '__main__':
+    ap = argparse.ArgumentParser()
+    ap.add_argument('-e', '--event', help="Events to open", default="cpu-clock,task-clock")
+    args = ap.parse_args()
+    main(args.event)
Signature · 1 line
2.49.0.1101.gccaa498523-goog
maintainer Applied
#18 lore raw

Re: [PATCH v3 7/7] perf python: Add counting.py as example for counting perf events

Quoted 7 lines · Ian Rogers

From: Gautam Menghani <[email protected]>

Add counting.py - a python version of counting.c to demonstrate measuring and reading of counts for given perf events.

Tested and applied:

Committer testing:

Build perf and make the generated python binding somewhere you can point to to avoid using the one in the distro python3-perf (fedora, may be different in other distros):

  $ make -k O=/tmp/build/$(basename $PWD)/ -C tools/perf install-bin

Copy /tmp/build/perf-tools-next/python/perf.cpython-313-x86_64-linux-gnu.so to somewhere outside this toolbox container and then use it with root:

  # export PYTHONPATH=/root/python/
  # ls -la /root/python/
  total 10640
  drwxr-xr-x. 1 root root       72 May 21 11:40 .
  dr-xr-x---. 1 root root      574 May 21 11:40 ..
  -rwxr-xr-x. 1 acme acme 10894360 May 21 11:40 perf.cpython-313-x86_64-linux-gnu.so
  # tools/perf/python/counting.py | head -5
  For evsel(software/cpu-clock/) val: 2930946 enable: 2932479 run: 2932479
  For evsel(software/cpu-clock/) val: 2924975 enable: 2926267 run: 2926267
  For evsel(software/cpu-clock/) val: 2921017 enable: 2922430 run: 2922430
  For evsel(software/cpu-clock/) val: 2914966 enable: 2916549 run: 2916549
  For evsel(software/cpu-clock/) val: 2910027 enable: 2911589 run: 2911589
  #

It would be nice to have something that compares the output for some envent obtained from both 'perf stat' and using these new python counting classes, but that can be done later.

Applied to perf-tools-next,

- Arnaldo

Quoted 59 lines

---
Ian modified from v2 to make the API take a CPU and thread then compute from these the appropriate indices. This was discussed as the preferred API with Arnaldo:
https://lore.kernel.org/linux-perf-users/[email protected]/
The use of a thread_map and cpu_map was also removed to make the code cleaner, instead the cpus and threads of the parsed evsel are used. Support for command line events is also added. The indent is reduced from 8 to 4 to match the preferred python PEP8 indent.

---
 tools/perf/python/counting.py | 36 +++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
 create mode 100755 tools/perf/python/counting.py
tools/perf/python/counting.py new file +36 -0
new file mode 100755
index 000000000000..02121d2bb11d
@@ -0,0 +1,36 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+# -*- python -*-
+# -*- coding: utf-8 -*-
+
+import argparse
+import perf
+
+def main(event: str):
+    evlist = perf.parse_events(event)
+
+    for evsel in evlist:
+        evsel.read_format = perf.FORMAT_TOTAL_TIME_ENABLED | perf.FORMAT_TOTAL_TIME_RUNNING
+
+    evlist.open()
+    evlist.enable()
+
+    count = 100000
+    while count > 0:
+        count -= 1
+
+    evlist.disable()
+
+    for evsel in evlist:
+        for cpu in evsel.cpus():
+            for thread in evsel.threads():
+                counts = evsel.read(cpu, thread)
+                print(f"For {evsel} val: {counts.val} enable: {counts.ena} run: {counts.run}")
+
+    evlist.close()
+
+if __name__ == '__main__':
+    ap = argparse.ArgumentParser()
+    ap.add_argument('-e', '--event', help="Events to open", default="cpu-clock,task-clock")
+    args = ap.parse_args()
+    main(args.event)
Signature · 1 line
2.49.0.1101.gccaa498523-goog