Build trace-cruncher as Python pakage
cover Build trace-cruncher as Python pakage
This patch-set restructures the project and makes it build as a native Python package. Although it looks like a complete rewrite, this is essentially just a switching from using Cython to using directly the C API of Python. Cython is still being used but only for the implementation of the NumPy data wrapper. The functionalities that are wrapping Ftrace are extended substantially. This is possible due to switching to use of the recently released libraries: libtraceevent and libtracefs.
Yordan Karadzhov (VMware) (9):
trace-cruncher: Refactor the part that wraps ftrace
trace-cruncher: Refactor the part that wraps libkshark
trace-cruncher: Add "utils"
trace-cruncher: Refactor the examples
trace-cruncher: Add Makefile
trace-cruncher: Update README.md
trace-cruncher: Remove all leftover files.
trace-cruncher: Add testing
trace-cruncher: Add github workflow for CI testing
.github/workflows/main.yml | 58 + 0001-kernel-shark-Add-_DEVEL-build-flag.patch | 90 -- 0002-kernel-shark-Add-reg_pid-plugin.patch | 231 --- Makefile | 33 + README.md | 84 +- clean.sh | 6 - examples/gpareto_fit.py | 328 ---- examples/ksharksetup.py | 24 - examples/page_faults.py | 120 -- examples/sched_wakeup.py | 70 +- libkshark-py.c | 224 --- libkshark_wrapper.pyx | 361 ----- np_setup.py | 90 -- setup.py | 81 + src/common.h | 100 ++ src/ftracepy-utils.c | 1367 +++++++++++++++++ src/ftracepy-utils.h | 127 ++ src/ftracepy.c | 262 ++++ src/ksharkpy-utils.c | 411 +++++ src/ksharkpy-utils.h | 41 + src/ksharkpy.c | 94 ++ src/npdatawrapper.pyx | 203 +++ src/trace2matrix.c | 40 + tests/0_get_data/__init__.py | 0 tests/0_get_data/test_get_data.py | 26 + tests/1_unit/__init__.py | 0 tests/1_unit/test_01_ftracepy_unit.py | 454 ++++++ tests/1_unit/test_02_datawrapper_unit.py | 41 + tests/1_unit/test_03_ksharkpy_unit.py | 72 + tests/2_integration/__init__.py | 0 .../test_01_ftracepy_integration.py | 113 ++ .../test_03_ksharkpy_integration.py | 25 + tests/__init__.py | 0 tracecruncher/__init__.py | 0 tracecruncher/ft_utils.py | 28 + tracecruncher/ks_utils.py | 227 +++ 36 files changed, 3887 insertions(+), 1544 deletions(-) create mode 100644 .github/workflows/main.yml delete mode 100644 0001-kernel-shark-Add-_DEVEL-build-flag.patch delete mode 100644 0002-kernel-shark-Add-reg_pid-plugin.patch create mode 100644 Makefile delete mode 100755 clean.sh delete mode 100755 examples/gpareto_fit.py delete mode 100644 examples/ksharksetup.py delete mode 100755 examples/page_faults.py delete mode 100644 libkshark-py.c delete mode 100644 libkshark_wrapper.pyx delete mode 100755 np_setup.py create mode 100644 setup.py create mode 100644 src/common.h create mode 100644 src/ftracepy-utils.c create mode 100644 src/ftracepy-utils.h create mode 100644 src/ftracepy.c create mode 100644 src/ksharkpy-utils.c create mode 100644 src/ksharkpy-utils.h create mode 100644 src/ksharkpy.c create mode 100644 src/npdatawrapper.pyx create mode 100644 src/trace2matrix.c create mode 100644 tests/0_get_data/__init__.py create mode 100755 tests/0_get_data/test_get_data.py create mode 100644 tests/1_unit/__init__.py create mode 100644 tests/1_unit/test_01_ftracepy_unit.py create mode 100755 tests/1_unit/test_02_datawrapper_unit.py create mode 100755 tests/1_unit/test_03_ksharkpy_unit.py create mode 100644 tests/2_integration/__init__.py create mode 100755 tests/2_integration/test_01_ftracepy_integration.py create mode 100755 tests/2_integration/test_03_ksharkpy_integration.py create mode 100644 tests/__init__.py create mode 100644 tracecruncher/__init__.py create mode 100644 tracecruncher/ft_utils.py create mode 100644 tracecruncher/ks_utils.py
Signature · 1 line
2.27.0
PATCH 1/9 trace-cruncher: Refactor the part that wraps ftrace
This is the first patch from a patch-set that aims to refactor trace-cruncher completely. The goal it to be able to build the project as a native Python package, which contains several sub-packages implemented as C extensions via the Python's C API. In this patch the part of the interface that relies on libtracefs, libtraceevent (and libtracecmd in the future) gets re-implemented as an extension called "tracecruncher.ftracepy". Note that this new extension has a stand-alone build that is completely decoupled from the existing build system used by trace-cruncher.
--- setup.py | 68 +++ src/common.h | 100 +++ src/ftracepy-utils.c | 1367 ++++++++++++++++++++++++++++++++++++++++++ src/ftracepy-utils.h | 127 ++++ src/ftracepy.c | 262 ++++++++ 5 files changed, 1924 insertions(+) create mode 100644 setup.py create mode 100644 src/common.h create mode 100644 src/ftracepy-utils.c create mode 100644 src/ftracepy-utils.h create mode 100644 src/ftracepy.c
setup.py new file +68 -0
@@ -0,0 +1,68 @@ +#!/usr/bin/env python3 + +""" +SPDX-License-Identifier: LGPL-2.1 + +Copyright 2019 VMware Inc, Yordan Karadzhov (VMware) <[email protected]> +""" + +from setuptools import setup, find_packages +from distutils.core import Extension +from Cython.Build import cythonize + +import pkgconfig as pkg + + +def third_party_paths(): + pkg_traceevent = pkg.parse('libtraceevent') + pkg_ftracepy = pkg.parse('libtracefs') + pkg_tracecmd = pkg.parse('libtracecmd') + + include_dirs = [] + include_dirs.extend(pkg_traceevent['include_dirs']) + include_dirs.extend(pkg_ftracepy['include_dirs']) + include_dirs.extend(pkg_tracecmd['include_dirs']) + + library_dirs = [] + library_dirs.extend(pkg_traceevent['library_dirs']) + library_dirs.extend(pkg_ftracepy['library_dirs']) + library_dirs.extend(pkg_tracecmd['library_dirs']) + library_dirs = list(set(library_dirs)) + + return include_dirs, library_dirs + +include_dirs, library_dirs = third_party_paths() + +def extension(name, sources, libraries): + runtime_library_dirs = library_dirs + runtime_library_dirs.extend('$ORIGIN') + return Extension(name, sources=sources, + include_dirs=include_dirs, + library_dirs=library_dirs, + runtime_library_dirs=runtime_library_dirs, + libraries=libraries, + ) + +def main(): + module_ft = extension(name='tracecruncher.ftracepy', + sources=['src/ftracepy.c', 'src/ftracepy-utils.c'], + libraries=['traceevent', 'tracefs']) + + setup(name='tracecruncher', + version='0.1.0', + description='NumPy based interface for accessing tracing data in Python.', + author='Yordan Karadzhov (VMware)', + author_email='[email protected]', + url='https://github.com/vmware/trace-cruncher', + license='LGPL-2.1', + packages=find_packages(), + ext_modules=[module_ft], + classifiers=[ + 'Development Status :: 3 - Alpha', + 'Programming Language :: Python :: 3', + ] + ) + + +if __name__ == '__main__': + main()
src/common.h new file +100 -0
100 lines not on this page · raw message
src/ftracepy-utils.c new file +1367 -0
1,367 lines not on this page · raw message
src/ftracepy-utils.h new file +127 -0
127 lines not on this page · raw message
src/ftracepy.c new file +262 -0
262 lines not on this page · raw message
Signature · 1 line
2.27.0
Re: [PATCH 1/9] trace-cruncher: Refactor the part that wraps ftrace
Quoted 2 lines · On Mon, 19 Apr 2021 16:01:32 +0300 "Yordan Karadzhov (VMware)"…
This is the first patch from a patch-set that aims to refactor trace-cruncher completely. The goal it to be able to build the
General comment. Try to avoid mentioning "first patch from a patch set", as that becomes meaningless in git history. Three years from now, a git blame comes to this patch, that "first patch in a patch set" becomes meaningless. Basically, when writing a change log, you want to think, "What do I want this commit log to say when I read it by itself three years from now?" ;-)
Quoted 7 lines
project as a native Python package, which contains several sub-packages implemented as C extensions via the Python's C API. In this patch the part of the interface that relies on libtracefs, libtraceevent (and libtracecmd in the future) gets re-implemented as an extension called "tracecruncher.ftracepy". Note that this new extension has a stand-alone build that is completely decoupled from the existing build system used by trace-cruncher.
The way I would say it is this:
"In order to be able to bulid the project as a native Python package, which contains several sub-packages implement as C extensions via the Pythons' C API, the part of the interface that relies on libtracefs, libtraceevent (and libtracecmd in the future) needs to be re-implemented as an extension called "tracecruncher.ftracepy". Note that..."
That is, try to word it where as disjoint from the rest of the series.
Quoted 110 lines
Signed-off-by Yordan Karadzhov (VMware) <[email protected]>--- setup.py | 68 +++ src/common.h | 100 +++ src/ftracepy-utils.c | 1367 ++++++++++++++++++++++++++++++++++++++++++ src/ftracepy-utils.h | 127 ++++ src/ftracepy.c | 262 ++++++++ 5 files changed, 1924 insertions(+) create mode 100644 setup.py create mode 100644 src/common.h create mode 100644 src/ftracepy-utils.c create mode 100644 src/ftracepy-utils.h create mode 100644 src/ftracepy.csetup.py new file +68
-0@@ -0,0 +1,68 @@ +#!/usr/bin/env python3 + +""" +SPDX-License-Identifier: LGPL-2.1 + +Copyright 2019 VMware Inc, Yordan Karadzhov (VMware) <[email protected]> +""" + +from setuptools import setup, find_packages +from distutils.core import Extension +from Cython.Build import cythonize + +import pkgconfig as pkg + + +def third_party_paths(): + pkg_traceevent = pkg.parse('libtraceevent') + pkg_ftracepy = pkg.parse('libtracefs') + pkg_tracecmd = pkg.parse('libtracecmd') + + include_dirs = [] + include_dirs.extend(pkg_traceevent['include_dirs']) + include_dirs.extend(pkg_ftracepy['include_dirs']) + include_dirs.extend(pkg_tracecmd['include_dirs']) + + library_dirs = [] + library_dirs.extend(pkg_traceevent['library_dirs']) + library_dirs.extend(pkg_ftracepy['library_dirs']) + library_dirs.extend(pkg_tracecmd['library_dirs']) + library_dirs = list(set(library_dirs)) + + return include_dirs, library_dirs + +include_dirs, library_dirs = third_party_paths() + +def extension(name, sources, libraries): + runtime_library_dirs = library_dirs + runtime_library_dirs.extend('$ORIGIN') + return Extension(name, sources=sources, + include_dirs=include_dirs, + library_dirs=library_dirs, + runtime_library_dirs=runtime_library_dirs, + libraries=libraries, + ) + +def main(): + module_ft = extension(name='tracecruncher.ftracepy', + sources=['src/ftracepy.c', 'src/ftracepy-utils.c'], + libraries=['traceevent', 'tracefs']) + + setup(name='tracecruncher', + version='0.1.0', + description='NumPy based interface for accessing tracing data in Python.', + author='Yordan Karadzhov (VMware)', + author_email='[email protected]', + url='https://github.com/vmware/trace-cruncher', + license='LGPL-2.1', + packages=find_packages(), + ext_modules=[module_ft], + classifiers=[ + 'Development Status :: 3 - Alpha', + 'Programming Language :: Python :: 3', + ] + ) + + +if __name__ == '__main__': + main()src/common.h new file +15
-0@@ -0,0 +1,100 @@ +/* SPDX-License-Identifier: LGPL-2.1 */ + +/* +* Copyright (C) 2017 VMware Inc, Yordan Karadzhov <[email protected]> +*/ + +#ifndef _TC_COMMON_H +#define _TC_COMMON_H + +// C +#include <stdbool.h> +#include <string.h> + +#define TRACECRUNCHER_ERROR tracecruncher_error +#define KSHARK_ERROR kshark_error
src/common.h line 25
Quoted 10 lines
src/common.h new file +10
-0+#define TEP_ERROR tep_error +#define TFS_ERROR tfs_error + +#define KS_INIT_ERROR \ + PyErr_SetString(KSHARK_ERROR, "libshark failed to initialize"); + +#define MEM_ERROR \ + PyErr_SetString(TRACECRUNCHER_ERROR, "failed to allocate memory"); + +#define NO_ARG "none"
If you defined NO_ARG as:
static const char *NO_ARG = "/NONE/";
then you could test for equal below, as well as not worry that someone might name their instance "none" and screw this up.
src/common.h line 31
Quoted 6 lines
src/common.h new file +6
-0+ +static inline bool is_all (const char *arg) +{ +return strcmp (arg,"all" ) ==0 || +strcmp (arg,"All" ) ==0 || +strcmp (arg,"ALL" ) ==0 ;
If you want to make this truly case insensitive:
{
const char all[] = "all";
const char *p = &all[0];
for (; *arg; arg++, p++) {
if (tolower(*arg) != *p)
return false;
}
return !(*p);
}src/common.h line 36
Quoted 5 lines
src/common.h new file +5
-0+} + +static inline bool is_no_arg (const char *arg) +{ +return arg[0 ] =='\0' ||strcmp (arg,NO_ARG ) ==0 ;
If NO_ARG is a const char *, then you could have this be:
return arg[0] == '\0' || arg == NO_ARG;
Quoted 88 lines
src/common.h new file +64
-0+} + +static inline bool is_set (const char *arg) +{ +return !(is_all (arg) ||is_no_arg (arg)); +} + +static inline void no_free () +{ +} + +#define NO_FREE no_free + +#define STR (x) #x + +#define MAKE_TYPE_STR (x) STR(traceevent.x) + +#define MAKE_DIC_STR (x) STR(libtraceevent x object) + +#define C_OBJECT_WRAPPER_DECLARE (c_type, py_type) \ + typedef struct { \ + PyObject_HEAD \ + struct c_type *ptrObj; \ +} py_type; \ +PyObject *py_type##_New(struct c_type *evt_ptr); \ +bool py_type##TypeInit(); \ + +#define C_OBJECT_WRAPPER (c_type, py_type, ptr_free) \ +static PyTypeObject py_type##Type = { \ + PyVarObject_HEAD_INIT(NULL, 0) MAKE_TYPE_STR(c_type) \ +}; \ +PyObject *py_type##_New(struct c_type *evt_ptr) \ +{ \ + py_type *newObject; \ + newObject = PyObject_New(py_type, &py_type##Type); \ + newObject->ptrObj = evt_ptr; \ + return (PyObject *) newObject; \ +} \ +static int py_type##_init(py_type *self, PyObject *args, PyObject *kwargs) \ +{ \ + self->ptrObj = NULL; \ + return 0; \ +} \ +static void py_type##_dealloc(py_type *self) \ +{ \ + ptr_free(self->ptrObj); \ + Py_TYPE(self)->tp_free(self); \ +} \ +bool py_type##TypeInit() \ +{ \ + py_type##Type.tp_new = PyType_GenericNew; \ + py_type##Type.tp_basicsize = sizeof(py_type); \ + py_type##Type.tp_init = (initproc) py_type##_init; \ + py_type##Type.tp_dealloc = (destructor) py_type##_dealloc; \ + py_type##Type.tp_flags = Py_TPFLAGS_DEFAULT; \ + py_type##Type.tp_doc = MAKE_DIC_STR(c_type); \ + py_type##Type.tp_methods = py_type##_methods; \ + if (PyType_Ready(&py_type##Type) < 0) \ + return false; \ + Py_INCREF(&py_type##Type); \ + return true; \ +} \ + +#endif src/ftracepy-utils.c new file +18
-0@@ -0,0 +1,1367 @@ +// SPDX-License-Identifier: LGPL-2.1 + +/* +* Copyright (C) 2021 VMware Inc, Yordan Karadzhov (VMware) <[email protected]> +*/ + +#ifndef _GNU_SOURCE +/** Use GNU C Library. */ +#define _GNU_SOURCE +#endif // _GNU_SOURCE + +// C +#include <search.h> +#include <string.h> +#include <sys/wait.h> + +// trace-cruncher +#include "ftracepy-utils.h"
src/ftracepy-utils.c line 28
Quoted 10 lines
src/ftracepy-utils.c new file +10
-0+ +int tep_vwarning (const char *name,const char *fmt,va_list ap) +{ +return 0 ; +} + +static void *instance_root =NULL ; +PyObject *TFS_ERROR =NULL ; +PyObject *TEP_ERROR =NULL ; +PyObject *TRACECRUNCHER_ERROR =NULL ;
Static and global variables are defined as zero or NULL, hence you do not need to initialized them to NULL or zero.
Quoted 54 lines
src/ftracepy-utils.c new file +54
-0+ +PyObject *PyTepRecord_time (PyTepRecord * self) +{ +return PyLong_FromLongLong (self->ptrObj->ts); +} + +PyObject *PyTepRecord_cpu (PyTepRecord * self) +{ +return PyLong_FromLong (self->ptrObj->cpu); +} + +PyObject *PyTepEvent_name (PyTepEvent * self) +{ +return PyUnicode_FromString (self->ptrObj->name); +} + +PyObject *PyTepEvent_id (PyTepEvent * self) +{ +return PyLong_FromLong (self->ptrObj->id); +} + +PyObject *PyTepEvent_field_names (PyTepEvent * self) +{ +struct tep_format_field *field, **fields; +struct tep_event *event = self->ptrObj; +int i =0 , nr_fields; +PyObject *list; + + nr_fields= event->format.nr_fields + event->format.nr_common; + list =PyList_New (nr_fields); + +/* Get all common fields. */ + fields =tep_event_common_fields (event); +if (!fields) { +PyErr_Format (TEP_ERROR , +"Failed to get common fields for event \'%s\'" , + self->ptrObj->name); +return NULL ; + } + +for (field = *fields; field; field = field->next) +PyList_SET_ITEM (list, i++,PyUnicode_FromString (field->name)); +free (fields); + +/* Add all unique fields. */ + fields =tep_event_fields (event); +if (!fields) { +PyErr_Format (TEP_ERROR , +"Failed to get fields for event \'%s\'" , + self->ptrObj->name); +return NULL ; + } + +for (field = *fields; field; field = field->next)
src/ftracepy-utils.c line 92
Quoted 10 lines
src/ftracepy-utils.c new file +10
-0+PyList_SET_ITEM (list, i++,PyUnicode_FromString (field->name)); +free (fields); + +return list; +} + +PyObject *PyTepEvent_parse_record_field (PyTepEvent * self,PyObject *args, +PyObject *kwargs) +{ + int mumber_field_mask = TEP_FIELD_IS_SIGNED |
What's a "mumber"?
Quoted 124 lines
src/ftracepy-utils.c new file +124
-0+TEP_FIELD_IS_LONG | +TEP_FIELD_IS_FLAG ; +struct tep_format_field *field; +const char *field_name; +PyTepRecord *record; + +static char *kwlist[] = {"record" ,"field" ,NULL }; +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"Os" , + kwlist, + &record, + &field_name)) { +return NULL ; + } + + field =tep_find_field (self->ptrObj, field_name); +if (!field) + field =tep_find_common_field (self->ptrObj, field_name); + +if (!field) { +PyErr_Format (TEP_ERROR , +"Failed to find field \'%s\' in event \'%s\'" , + field_name, self->ptrObj->name); +return NULL ; + } + +if (field->flags &TEP_FIELD_IS_STRING ) { +char *val = record->ptrObj->data + field->offset; + +return PyUnicode_FromString (val); + }else if (field->flags & mumber_field_mask) { +unsigned long long val; + +tep_read_number_field (field, record->ptrObj->data, &val); +return PyLong_FromLong (val); + } + +PyErr_Format (TEP_ERROR , +"Unsupported field format \"%li\" (TODO: implement this)" , + field->flags); +return NULL ; +} + +PyObject *PyTepEvent_get_pid (PyTepEvent * self,PyObject *args, +PyObject *kwargs) +{ +static char *kwlist[] = {"record" ,NULL }; +const char *field_name ="common_pid" ; +struct tep_format_field *field; +unsigned long long val =0 ; +PyTepRecord *record; + +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"O" , + kwlist, + &record)) { +return NULL ; + } + + field =tep_find_common_field (self->ptrObj, field_name); +if (!field) { +PyErr_Format (TEP_ERROR , +"Failed to find field \'s\' in event \'%s\'" , + field_name, self->ptrObj->name); +return NULL ; + } + +tep_read_number_field (field, record->ptrObj->data, &val); + +return PyLong_FromLong (val); +} + +static const char **get_arg_list (PyObject *py_list) +{ +const char **argv =NULL ; +PyObject *arg_py; +int i, n; + +if (!PyList_CheckExact (py_list)) + goto fail; + + n =PyList_Size (py_list); + argv =calloc (n +1 ,sizeof (*argv)); +for (i =0 ; i < n; ++i) { + arg_py =PyList_GetItem (py_list, i); +if (!PyUnicode_Check (arg_py)) + goto fail; + + argv[i] =PyUnicode_DATA (arg_py); + } + +return argv; + + fail: +PyErr_SetString (TRACECRUNCHER_ERROR , +"Failed to parse argument list." ); +free (argv); +return NULL ; +} + +PyObject *PyTep_init_local (PyTep *self,PyObject *args, +PyObject *kwargs) +{ +static char *kwlist[] = {"dir" ,"systems" ,NULL }; +struct tep_handle *tep =NULL ; +PyObject *system_list =NULL ; +const char *dir_str; + +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"s|O" , + kwlist, + &dir_str, + &system_list)) { +return NULL ; + } + +if (system_list) { +const char **sys_names =get_arg_list (system_list); + +if (sys_names) { + tep =tracefs_local_events_system (dir_str, sys_names);
src/ftracepy-utils.c line 226
Quoted 10 lines
src/ftracepy-utils.c new file +10
-0+free (sys_names); + } + }else { + tep =tracefs_local_events (dir_str); + } + +if (!tep) { +PyErr_Format (TFS_ERROR , +"Failed to get local events from \'%s\'." , + dir_str);
I wonder if that's the right message if the get_arg_list() failed.
Quoted 78 lines
src/ftracepy-utils.c new file +78
-0+return NULL ; + } + +tep_free (self->ptrObj); + self->ptrObj = tep; + + Py_RETURN_NONE; +} + +PyObject *PyTep_get_event (PyTep *self,PyObject *args, +PyObject *kwargs) +{ +static char *kwlist[] = {"system" ,"name" ,NULL }; +const char *system, *event_name; +struct tep_event *event; + +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"ss" , + kwlist, + &system, + &event_name)) { +return NULL ; + } + + event =tep_find_event_by_name (self->ptrObj, system, event_name); + +return PyTepEvent_New (event); +} + +static bool check_file (struct tracefs_instance *instance,const char *file) +{ +if (!tracefs_file_exists (instance, file)) { +PyErr_Format (TFS_ERROR ,"File %s does not exist." , file); +return false; + } + +return true; +} + +static bool check_dir (struct tracefs_instance *instance,const char *dir) +{ +if (!tracefs_dir_exists (instance, dir)) { +PyErr_Format (TFS_ERROR ,"Directory %s does not exist." , dir); +return false; + } + +return true; +} + +static int write_to_file (struct tracefs_instance *instance, +const char *file, +const char *val) +{ +int size; + +if (!check_file (instance, file)) +return -1 ; + + size =tracefs_instance_file_write (instance, file, val); +if (size <=0 ) +PyErr_Format (TFS_ERROR ,"Can not write to file %s" , file); + +return size; +} + +static int read_from_file (struct tracefs_instance *instance, +const char *file, +char **val) +{ +int size; + +if (!check_file (instance, file)) +return -1 ; + + *val =tracefs_instance_file_read (instance, file, &size); +if (size <=0 ) +PyErr_Format (TFS_ERROR ,"Can not read from file %s" , file);
src/ftracepy-utils.c line 314
Quoted 10 lines
src/ftracepy-utils.c new file +10
-0+ +return size; +} + +static inline void trim_new_line (char *val) +{ + val[strlen (val) -1 ] ='\0' ; +} + +static bool write_to_file_and_chack(struct tracefs_instance *instance,
"chack"?
Quoted 7 lines
src/ftracepy-utils.c new file +7
-0+const char *file, + const char *val) +{ + char *read_val; +int ret; + +if (write_to_file (instance, file, val) <=0 )
src/ftracepy-utils.c line 331
Quoted 10 lines
src/ftracepy-utils.c new file +10
-0+return false; + +if (read_from_file (instance, file, &read_val) <=0 ) +return false; + +trim_new_line (read_val); + ret =strcmp (read_val, val); +free (read_val); + +return ret ==0 ? true : false;
return !ret;
?
Quoted 97 lines
src/ftracepy-utils.c new file +97
-0+} + +static PyObject *tfs_list2py_list (char **list) +{ +PyObject *py_list =PyList_New (0 ); +int i; + +for (i =0 ; list && list[i]; i++) +PyList_Append (py_list,PyUnicode_FromString (list[i])); + +tracefs_list_free (list); + +return py_list; +} + +struct instance_wrapper { +struct tracefs_instance *ptr; +const char *name; +}; + +const char *instance_wrapper_get_name (const struct instance_wrapper *iw) +{ +if (!iw->ptr) +return iw->name; + +return tracefs_instance_get_name (iw->ptr); +} + +static int instance_compare (const void *a,const void *b) +{ +const struct instance_wrapper *iwa, *iwb; + + iwa = (const struct instance_wrapper *) a; + iwb = (const struct instance_wrapper *) b; + +return strcmp (instance_wrapper_get_name (iwa), +instance_wrapper_get_name (iwb)); +} + +void instance_wrapper_free (void *ptr) +{ +struct instance_wrapper *iw; +if (!ptr) +return ; + + iw = ptr; +if (iw->ptr) +tracefs_instance_destroy (iw->ptr); + +free (ptr); +} + +static void destroy_all_instances (void ) +{ +tdestroy (instance_root, instance_wrapper_free); + instance_root =NULL ; +} + +static struct tracefs_instance *find_instance (const char *name) +{ +struct instance_wrapper iw, **iw_ptr; +if (!is_set (name)) +return NULL ; + +if (!tracefs_instance_exists (name)) { +PyErr_Format (TFS_ERROR ,"Trace instance \'%s\' does not exist." , + name); +return NULL ; + } + + iw.ptr =NULL ; + iw.name = name; + iw_ptr =tfind (&iw, &instance_root, instance_compare); +if (!iw_ptr || !(*iw_ptr) || !(*iw_ptr)->ptr || +strcmp (tracefs_instance_get_name ((*iw_ptr)->ptr), name) !=0 ) { +PyErr_Format (TFS_ERROR ,"Unable to find trace instances \'%s\'." , + name); +return NULL ; + } + +return (*iw_ptr)->ptr; +} + +bool get_optional_instance (const char *instance_name, +struct tracefs_instance **instance) +{ + *instance =NULL ; +if (is_set (instance_name)) { + *instance =find_instance (instance_name); +if (!instance) { +PyErr_Format (TFS_ERROR , +"Failed to find instance \'%s\'." , + instance_name); +return false; + } + } +
src/ftracepy-utils.c line 438
Quoted 10 lines
src/ftracepy-utils.c new file +10
-0+return true; +} + +bool get_instance_from_arg (PyObject *args,PyObject *kwargs, +struct tracefs_instance **instance) +{ +const char *instance_name; + +static char *kwlist[] = {"instance" ,NULL }; + instance_name =NO_ARG ;
As stated above, if NO_ARG was a const char *, then you don't need to worry about instances called "none".
Quoted 43 lines
src/ftracepy-utils.c new file +43
-0+if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"|s" , + kwlist, + &instance_name)) { +return false; + } + +if (!get_optional_instance (instance_name, instance)) +return false; + +return true; +} + +PyObject *PyFtrace_dir (PyObject *self) +{ +return PyUnicode_FromString (tracefs_tracing_dir ()); +} + +PyObject *PyFtrace_create_instance (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +struct instance_wrapper *iw, **iw_ptr; +struct tracefs_instance *instance; +char *name =NO_ARG ; + +static char *kwlist[] = {"name" ,NULL }; +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"s" , + kwlist, + &name)) { +return NULL ; + } + +if (!is_set (name)) { +PyErr_Format (TFS_ERROR , +"\'%s\' is not a valid name for trace instance." , + name); +return NULL ; + } + + instance =tracefs_instance_create (name);
src/ftracepy-utils.c line 491
Quoted 10 lines
src/ftracepy-utils.c new file +10
-0+if (!instance || + !tracefs_instance_exists (name) || + !tracefs_instance_is_new (instance)) { +PyErr_Format (TFS_ERROR , +"Failed to create new trace instance \'%s\'." , + name); +return NULL ; + } + + iw =malloc (sizeof (*iw));
Need to check if malloc fails.
Quoted 227 lines
src/ftracepy-utils.c new file +227
-0+ iw->ptr = instance; + iw_ptr =tsearch (iw, &instance_root, instance_compare); +if (!iw_ptr || !(*iw_ptr) || !(*iw_ptr)->ptr || +strcmp (tracefs_instance_get_name ((*iw_ptr)->ptr), name) !=0 ) { +PyErr_Format (TFS_ERROR , +"Failed to store new trace instance \'%s\'." , + name); +tracefs_instance_destroy (instance); +tracefs_instance_free (instance); +return NULL ; + } + + Py_RETURN_NONE; +} + +PyObject *PyFtrace_destroy_instance (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +struct tracefs_instance *instance; +struct instance_wrapper iw; +char *name; + +static char *kwlist[] = {"name" ,NULL }; +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"s" , + kwlist, + &name)) { +return NULL ; + } + +if (is_all (name)) { +destroy_all_instances (); + Py_RETURN_NONE; + } + + instance =find_instance (name); +if (!instance) { +PyErr_Format (TFS_ERROR , +"Unable to destroy trace instances \'%s\'." , + name); +return NULL ; + } + + iw.ptr =NULL ; + iw.name = name; +tdelete (&iw, &instance_root, instance_compare); + +tracefs_instance_destroy (instance); +tracefs_instance_free (instance); + + Py_RETURN_NONE; +} + +PyObject *instance_list; + +static void instance_action (const void *nodep,VISIT which,int depth) +{ +struct instance_wrapper *iw = *(struct instance_wrapper **) nodep; +const char *name; + +switch (which) { +case preorder: +case endorder: +break ; + +case postorder: +case leaf: + name =tracefs_instance_get_name (iw->ptr); +PyList_Append (instance_list,PyUnicode_FromString (name)); +break ; + } +} + +PyObject *PyFtrace_get_all_instances (PyObject *self) +{ + instance_list =PyList_New (0 ); +twalk (instance_root, instance_action); + +return instance_list; +} + +PyObject *PyFtrace_destroy_all_instances (PyObject *self) +{ +destroy_all_instances (); + + Py_RETURN_NONE; +} + +PyObject *PyFtrace_instance_dir (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +struct tracefs_instance *instance; + +if (!get_instance_from_arg (args, kwargs, &instance)) +return NULL ; + +return PyUnicode_FromString (tracefs_instance_get_dir (instance)); +} + +PyObject *PyFtrace_available_tracers (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +struct tracefs_instance *instance; +char **list; + +if (!get_instance_from_arg (args, kwargs, &instance)) +return NULL ; + + list =tracefs_tracers (tracefs_instance_get_dir (instance)); +if (!list) +return NULL ; + +return tfs_list2py_list (list); +} + +PyObject *PyFtrace_set_current_tracer (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +const char *file ="current_tracer" , *tracer, *instance_name; +struct tracefs_instance *instance; + +static char *kwlist[] = {"tracer" ,"instance" ,NULL }; + tracer = instance_name =NO_ARG ; +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"|ss" , + kwlist, + &tracer, + &instance_name)) { +return NULL ; + } + +if (!get_optional_instance (instance_name, &instance)) +return NULL ; + +if (is_set (tracer) && +strcmp (tracer,"nop" ) !=0 ) { +char **all_tracers = +tracefs_tracers (tracefs_instance_get_dir (instance)); +int i; + +for (i =0 ; all_tracers && all_tracers[i]; i++) { +if (!strcmp (all_tracers[i], tracer)) +break ; + } + +if (!all_tracers || !all_tracers[i]) { +PyErr_Format (TFS_ERROR , +"Tracer \'%s\' is not available." , + tracer); +return NULL ; + } + }else if (!is_set (tracer)) { + tracer ="nop" ; + } + +if (!write_to_file_and_chack (instance, file, tracer)) { +PyErr_Format (TFS_ERROR ,"Failed to enable tracer \'%s\'" , + tracer); +return NULL ; + } + + Py_RETURN_NONE; +} + +PyObject *PyFtrace_get_current_tracer (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +const char *file ="current_tracer" ; +struct tracefs_instance *instance; +PyObject *ret; +char *tracer; + +if (!get_instance_from_arg (args, kwargs, &instance)) +return NULL ; + +if (read_from_file (instance, file, &tracer) <=0 ) +return NULL ; + +trim_new_line (tracer); + ret =PyUnicode_FromString (tracer); +free (tracer); + +return ret; +} + +PyObject *PyFtrace_available_event_systems (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +struct tracefs_instance *instance; +char **list; + +if (!get_instance_from_arg (args, kwargs, &instance)) +return NULL ; + + list =tracefs_event_systems (tracefs_instance_get_dir (instance)); +if (!list) +return NULL ; + +return tfs_list2py_list (list); +} + +PyObject *PyFtrace_available_system_events (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +static char *kwlist[] = {"system" ,"instance" ,NULL }; +const char *instance_name =NO_ARG , *system; +struct tracefs_instance *instance; +char **list; + +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"s|s" , + kwlist, + &system, + &instance_name)) { +return NULL ; + } + +if (!get_optional_instance (instance_name, &instance)) +return NULL ; + + list =tracefs_system_events (tracefs_instance_get_dir (instance), + system); +if (!list) +return NULL ;
src/ftracepy-utils.c line 728
Quoted 10 lines
src/ftracepy-utils.c new file +10
-0+ +return tfs_list2py_list (list); +} + +bool get_event_enable_file (struct tracefs_instance *instance, +const char *system,const char *event, +char **path) +{ +char *buff =calloc (PATH_MAX ,1 ); +const char *instance_name;
Need to check buff.
Quoted 18 lines
src/ftracepy-utils.c new file +18
-0+ +if ((is_all (system) &&is_all (event)) || + (is_all (system) &&is_no_arg (event)) || + (is_no_arg (system) &&is_all (event))) { +strcpy (buff,"events/enable" ); + + *path = buff; + }else if (is_set (system)) { +strcpy (buff,"events/" ); +strcat (buff, system); +if (!check_dir (instance, buff)) + goto fail; + +if (is_set (event)) { +strcat (buff,"/" ); +strcat (buff, event); +if (!check_dir (instance, buff)) + goto fail;
src/ftracepy-utils.c line 756
Quoted 10 lines
src/ftracepy-utils.c new file +10
-0+ +strcat (buff,"/enable" ); + }else { +strcat (buff,"/enable" ); + } + + *path = buff; + }else { + goto fail; + }
We really need to implement a
tracefs_enable_event(struct tracefs_instance *instance, const char *system, const char *event);
That way we don't need to search for the file to write to here.
Quoted 519 lines
src/ftracepy-utils.c new file +519
-0+ +return true; + + fail: + instance_name = + instance ?tracefs_instance_get_name (instance) :"top" ; +PyErr_Format (TFS_ERROR , +"Failed to locate event:\n Instance: %s System: %s Event: %s" , + instance_name, system, event); +free (buff); + *path =NULL ; +return false; +} + +static bool enable_event (PyObject *self,PyObject *args,PyObject *kwargs, +const char *val) +{ +struct tracefs_instance *instance; +char *file; +int ret; +static char *kwlist[] = {"instance" ,"system" ,"event" ,NULL }; +const char *instance_name, *system, *event; + + instance_name = system = event =NO_ARG ; +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"|sss" , + kwlist, + &instance_name, + &system, + &event)) { +return false; + } + +if (!get_optional_instance (instance_name, &instance)) +return false; + +if (!get_event_enable_file (instance, system, event, &file)) +return false; + + ret =write_to_file_and_chack (instance, file, val); +free (file); + +return ret <0 ? false : true; +} + +#define ON "1" +#define OFF "0" + +PyObject *PyFtrace_enable_event (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +if (!enable_event (self, args, kwargs,ON )) +return NULL ; + + Py_RETURN_NONE; +} + +PyObject *PyFtrace_disable_event (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +if (!enable_event (self, args, kwargs,OFF )) +return NULL ; + + Py_RETURN_NONE; +} + +static bool enable_events (PyObject *self,PyObject *args,PyObject *kwargs, +const char *val) +{ +static char *kwlist[] = {"instance" ,"systems" ,"events" ,NULL }; +PyObject *system_list =NULL , *event_list =NULL , *system_event_list; +const char **systems =NULL , **events =NULL ; +struct tracefs_instance *instance; +const char *instance_name; +char *file =NULL ; +int ret; + + instance_name =NO_ARG ; +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"|sOO" , + kwlist, + &instance_name, + &system_list, + &event_list)) { +return false; + } + +if (!get_optional_instance (instance_name, &instance)) +return false; + +if (!system_list && !event_list) { + ret =write_to_file_and_chack (instance,"events/enable" , val); +return ret <0 ? false : true; + } + +if (!system_list && event_list) { +if (PyUnicode_Check (event_list) && +is_all (PyUnicode_DATA (event_list))) { + ret =write_to_file_and_chack (instance, +"events/enable" , + val); +return ret <0 ? false : true; + }else { +PyErr_SetString (TFS_ERROR , +"Failed to enable events for unspecified system" ); +return false; + } + } + + systems =get_arg_list (system_list); + +if (!event_list) { +for (int s =0 ; systems[s]; ++s) { +asprintf (&file,"events/%s/enable" , systems[s]); + ret =write_to_file_and_chack (instance, file, val); +free (file); +if (ret <0 ) +return false; + } + +return true; + } + +if (!PyList_CheckExact (event_list)) + goto fail_with_err; + +for (int s =0 ; systems[s]; ++s) { + system_event_list =PyList_GetItem (event_list, s); +if (!system_event_list || !PyList_CheckExact (system_event_list)) + goto fail_with_err; + + events =get_arg_list (system_event_list); +for (int e =0 ; events[e]; ++e) { +if (!get_event_enable_file (instance, + systems[s], events[e], + &file)) + goto fail; + +if (!write_to_file (instance, file, val)) + goto fail; + +free (file); + file =NULL ; + } + +free (events); + events =NULL ; + } + +free (systems); + +return true; + + fail_with_err: +PyErr_SetString (TFS_ERROR ,"Inconsistent \"events\" argument. " ); + + fail: +free (systems); +free (events); +free (file); + +return false; +} + +PyObject *PyFtrace_enable_events (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +if (!enable_events (self, args, kwargs,ON )) +return NULL ; + + Py_RETURN_NONE; +} + +PyObject *PyFtrace_disable_events (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +if (!enable_events (self, args, kwargs,OFF )) +return NULL ; + + Py_RETURN_NONE; +} + +PyObject *PyFtrace_event_is_enabled (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +static char *kwlist[] = {"instance" ,"system" ,"event" ,NULL }; +const char *instance_name, *system, *event; +struct tracefs_instance *instance; +char *file, *val; +PyObject *ret; + + instance_name = system = event =NO_ARG ; +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"|sss" , + kwlist, + &instance_name, + &system, + &event)) { +return false; + } + +if (!get_optional_instance (instance_name, &instance)) +return false; + +if (!get_event_enable_file (instance, system, event, &file)) +return NULL ; + +if (read_from_file (instance, file, &val) <=0 ) +return NULL ; + +trim_new_line (val); + ret =PyUnicode_FromString (val); + +free (file); +free (val); + +return ret; +} + +static bool tracing_ON (struct tracefs_instance *instance) +{ +int ret =tracefs_trace_on (instance); + +if (ret <0 || +tracefs_trace_is_on (instance) !=1 ) { +const char *instance_name = + instance ?tracefs_instance_get_name (instance) :"top" ; + +PyErr_Format (TFS_ERROR , +"Failed to start tracing (Instance: %s)" , + instance_name); +return false; + } + +return true; +} + +PyObject *PyFtrace_tracing_ON (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +struct tracefs_instance *instance; + +if (!get_instance_from_arg (args, kwargs, &instance)) +return NULL ; + +if (!tracing_ON (instance)) +return NULL ; + + Py_RETURN_NONE; +} + +static bool tracing_OFF (struct tracefs_instance *instance) +{ +int ret =tracefs_trace_off (instance); + +if (ret <0 || +tracefs_trace_is_on (instance) !=0 ) { +const char *instance_name = + instance ?tracefs_instance_get_name (instance) :"top" ; + +PyErr_Format (TFS_ERROR , +"Failed to stop tracing (Instance: %s)" , + instance_name); +return false; + } + +return true; +} + +PyObject *PyFtrace_tracing_OFF (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +struct tracefs_instance *instance; + +if (!get_instance_from_arg (args, kwargs, &instance)) +return NULL ; + +if (!tracing_OFF (instance)) +return NULL ; + + Py_RETURN_NONE; +} + +PyObject *PyFtrace_is_tracing_ON (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +struct tracefs_instance *instance; +int ret; + +if (!get_instance_from_arg (args, kwargs, &instance)) +return NULL ; + + ret =tracefs_trace_is_on (instance); +if (ret <0 ) { +const char *instance_name = + instance ?tracefs_instance_get_name (instance) :"top" ; + +PyErr_Format (TFS_ERROR , +"Failed to check if tracing is ON (Instance: %s)" , + instance_name); +return NULL ; + } + +if (ret ==0 ) + Py_RETURN_FALSE; + + Py_RETURN_TRUE; +} + +static bool set_pid (struct tracefs_instance *instance, +const char *file,pid_t pid) +{ +char pid_str[100 ]; + +if (sprintf (pid_str,"%d" , pid) <=0 || + !write_to_file_and_chack (instance, file, pid_str)) { +PyErr_Format (TFS_ERROR ,"Failed to set PID %i for \"%s\"" , + pid, file); +return false; + } + +return true; +} + +PyObject *PyFtrace_set_event_pid (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +const char *instance_name =NO_ARG ; +struct tracefs_instance *instance; +int pid; + +static char *kwlist[] = {"pid" ,"instance" ,NULL }; +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"i|s" , + kwlist, + &pid, + &instance_name)) { +return NULL ; + } + +if (!get_optional_instance (instance_name, &instance)) +return NULL ; + +if (!set_pid (instance,"set_event_pid" , pid)) +return NULL ; + + Py_RETURN_NONE; +} + +PyObject *PyFtrace_set_ftrace_pid (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +const char *instance_name =NO_ARG ; +struct tracefs_instance *instance; +int pid; + +static char *kwlist[] = {"pid" ,"instance" ,NULL }; +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"i|s" , + kwlist, + &pid, + &instance_name)) { +return NULL ; + } + +if (!get_optional_instance (instance_name, &instance)) +return NULL ; + +if (!set_pid (instance,"set_ftrace_pid" , pid)) +return NULL ; + + Py_RETURN_NONE; +} + +static bool set_opt (struct tracefs_instance *instance, +const char *opt,const char *val) +{ +char file[PATH_MAX ]; + +if (sprintf (file,"options/%s" , opt) <=0 || + !write_to_file_and_chack (instance, file, val)) { +PyErr_Format (TFS_ERROR ,"Failed to set option \"%s\"" , opt); +return false; + } + +return true; +} + +static PyObject *set_option_py_args (PyObject *args,PyObject *kwargs, +const char *val) +{ +const char *instance_name =NO_ARG , *opt; +struct tracefs_instance *instance; + +static char *kwlist[] = {"option" ,"instance" ,NULL }; +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"s|s" , + kwlist, + &opt, + &instance_name)) { +return NULL ; + } + +if (!get_optional_instance (instance_name, &instance)) +return NULL ; + +if (!set_opt (instance, opt, val)) +return NULL ; + + Py_RETURN_NONE; +} + +PyObject *PyFtrace_enable_option (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +return set_option_py_args (args, kwargs,ON ); +} + +PyObject *PyFtrace_disable_option (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +return set_option_py_args (args, kwargs,OFF ); +} + +PyObject *PyFtrace_option_is_set (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +const char *instance_name =NO_ARG , *opt; +struct tracefs_instance *instance; +enum tracefs_option_id opt_id; + +static char *kwlist[] = {"option" ,"instance" ,NULL }; +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"s|s" , + kwlist, + &opt, + &instance_name)) { +return NULL ; + } + +if (!get_optional_instance (instance_name, &instance)) +return NULL ; + + opt_id =tracefs_option_id (opt); +if (tracefs_option_is_enabled (instance, opt_id)) + Py_RETURN_TRUE; + + Py_RETURN_FALSE; +} + +static PyObject *get_option_list (struct tracefs_instance *instance, +bool enabled) +{ +const struct tracefs_options_mask *mask; +PyObject *list =PyList_New (0 ); +int i; + + mask = enabled ?tracefs_options_get_enabled (instance) : +tracefs_options_get_supported (instance); + +for (i =0 ; i <TRACEFS_OPTION_MAX ; ++i) +if (tracefs_option_mask_is_set (mask, i)) { +const char *opt =tracefs_option_name (i); +PyList_Append (list,PyUnicode_FromString (opt)); + } + +return list; +} + +PyObject *PyFtrace_enabled_options (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +struct tracefs_instance *instance; + +if (!get_instance_from_arg (args, kwargs, &instance)) +return NULL ; + +return get_option_list (instance, true); +} + +PyObject *PyFtrace_supported_options (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +struct tracefs_instance *instance; + +if (!get_instance_from_arg (args, kwargs, &instance)) +return NULL ; + +return get_option_list (instance, false); +} + +static PyObject *get_callback_func (const char *plugin_name,const char * py_callback) +{ +PyObject *py_name, *py_module, *py_func; + + py_name =PyUnicode_FromString (plugin_name); + py_module =PyImport_Import (py_name); +if (!py_module) { +PyErr_Format (TFS_ERROR ,"Failed to import plugin \'%s\'" , + plugin_name); +return NULL ; + } + + py_func =PyObject_GetAttrString (py_module, py_callback); +if (!py_func || !PyCallable_Check (py_func)) { +PyErr_Format (TFS_ERROR , +"Failed to import callback from plugin \'%s\'" , + plugin_name); +return NULL ; + } + +return py_func;
src/ftracepy-utils.c line 1285
Quoted 10 lines
src/ftracepy-utils.c new file +10
-0+} + +static void start_tracing (struct tracefs_instance *instance, +char **argv,char **env) +{ +if (tracefs_option_enable (instance,TRACEFS_OPTION_EVENT_FORK ) <0 || +tracefs_option_enable (instance,TRACEFS_OPTION_FUNCTION_FORK ) <0 || + !set_pid (instance,"set_ftrace_pid" ,getpid ()) || + !set_pid (instance,"set_event_pid" ,getpid ())) +exit (1 );
So trace cruncher will only trace the given process and its children. There's no other alternative?
-- Steve
Quoted 483 lines
src/ftracepy-utils.c new file +82
-0+ +tracing_ON (instance); +if (execve (argv[0 ], argv, env) <0 ) { +PyErr_Format (TFS_ERROR ,"Failed to exec \'%s\'" , + argv[0 ]); + } + +exit (1 ); +} + +static int callback (struct tep_event *event,struct tep_record *record, +int cpu,void *py_func) +{ + record->cpu = cpu;// Remove when the bug in libtracefs is fixed. + +PyObject *py_tep_event =PyTepEvent_New (event); +PyObject *py_tep_record =PyTepRecord_New (record); + +PyObject *arglist =PyTuple_New (2 ); +PyTuple_SetItem (arglist,0 , py_tep_event); +PyTuple_SetItem (arglist,1 , py_tep_record); + +PyObject_CallObject ((PyObject *) py_func, arglist); + +return 0 ; +} + +PyObject *PyFtrace_trace_shell_process (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +const char *plugin ="__main__" , *py_callback ="callback" ; +char *process, *instance_name; +struct tracefs_instance *instance; +static char *kwlist[] = {"process" ,"plugin" ,"callback" ,"instance" ,NULL }; +struct tep_handle *tep; +PyObject *py_func; +pid_t pid; + + instance_name =NO_ARG ; +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"s|sss" , + kwlist, + &process, + &plugin, + &py_callback, + &instance_name)) { +return NULL ; + } + + py_func =get_callback_func (plugin, py_callback); +if (!py_func) +return NULL ; + +if (!get_optional_instance (instance_name, &instance)) +return NULL ; + + tep =tracefs_local_events (tracefs_instance_get_dir (instance)); + +char *argv[] = {getenv ("SHELL" ),"-c" , process,NULL }; +char *env[] = {NULL }; + + pid =fork (); +if (pid <0 ) { +PyErr_SetString (TFS_ERROR ,"Failed to fork" ); +return NULL ; + } + +if (pid ==0 ) +start_tracing (instance, argv, env); + +do { +tracefs_iterate_raw_events (tep, instance,NULL ,0 , callback, py_func); + }while (waitpid (pid,NULL ,WNOHANG ) != pid); + + Py_RETURN_NONE; +} + +void PyFtrace_at_exit (void ) +{ +destroy_all_instances (); +}src/ftracepy-utils.h new file +127
-0@@ -0,0 +1,127 @@ +/* SPDX-License-Identifier: LGPL-2.1 */ + +/* +* Copyright (C) 2021 VMware Inc, Yordan Karadzhov <[email protected]> +*/ + +#ifndef _TC_FTRACE_PY_UTILS +#define _TC_FTRACE_PY_UTILS + +// Python +#include <Python.h> + +// libtracefs +#include "tracefs.h" + +// trace-cruncher +#include "common.h" + +C_OBJECT_WRAPPER_DECLARE(tep_record, PyTepRecord) + +C_OBJECT_WRAPPER_DECLARE(tep_event, PyTepEvent) + +C_OBJECT_WRAPPER_DECLARE (tep_handle, PyTep) + +PyObject *PyTepRecord_time(PyTepRecord* self); + +PyObject *PyTepRecord_cpu (PyTepRecord * self); + +PyObject *PyTepEvent_name (PyTepEvent * self); + +PyObject *PyTepEvent_id (PyTepEvent * self); + +PyObject *PyTepEvent_field_names (PyTepEvent * self); + +PyObject *PyTepEvent_parse_record_field (PyTepEvent * self,PyObject *args, +PyObject *kwargs); + +PyObject *PyTepEvent_get_pid (PyTepEvent * self,PyObject *args, +PyObject *kwargs); + +PyObject *PyTep_init_local (PyTep *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyTep_get_event (PyTep *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_dir (PyObject *self); + +PyObject *PyFtrace_create_instance (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_destroy_instance (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_get_all_instances (PyObject *self); + +PyObject *PyFtrace_destroy_all_instances (PyObject *self); + +PyObject *PyFtrace_instance_dir (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_available_tracers (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_set_current_tracer (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_get_current_tracer (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_available_event_systems (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_available_system_events (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_enable_event (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_disable_event (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_enable_events (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_disable_events (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_event_is_enabled (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_tracing_ON (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_tracing_OFF (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_is_tracing_ON (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_set_event_pid (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_set_ftrace_pid (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_enable_option (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_disable_option (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_option_is_set (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_supported_options (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_enabled_options (PyObject *self,PyObject *args, +PyObject *kwargs); + +PyObject *PyFtrace_trace_shell_process (PyObject *self,PyObject *args, +PyObject *kwargs); + +void PyFtrace_at_exit (void ); + +#endif src/ftracepy.c new file +262
-0@@ -0,0 +1,262 @@ +// SPDX-License-Identifier: LGPL-2.1 + +/* +* Copyright (C) 2021 VMware Inc, Yordan Karadzhov (VMware) <[email protected]> +*/ + +// trace-cruncher +#include "ftracepy-utils.h" + +extern PyObject *TFS_ERROR ; +extern PyObject *TEP_ERROR ; +extern PyObject *TRACECRUNCHER_ERROR ; + +static PyMethodDef PyTepRecord_methods[] = { + {"time" , + (PyCFunction ) PyTepRecord_time, +METH_NOARGS , +"Get the time of the record." + }, + {"CPU" , + (PyCFunction ) PyTepRecord_cpu, +METH_NOARGS , +"Get the CPU Id of the record." + }, + {NULL } +}; + +C_OBJECT_WRAPPER(tep_record, PyTepRecord, NO_FREE) + +static PyMethodDef PyTepEvent_methods[] = { + {"name" , + (PyCFunction ) PyTepEvent_name, +METH_NOARGS , +"Get the name of the event." + }, + {"id" , + (PyCFunction ) PyTepEvent_id, +METH_NOARGS , +"Get the unique identifier of the event." + }, + {"field_names" , + (PyCFunction ) PyTepEvent_field_names, +METH_NOARGS , +"Get the names of all fields." + }, + {"parse_record_field" , + (PyCFunction ) PyTepEvent_parse_record_field, +METH_VARARGS |METH_KEYWORDS , +"Get the content of a record field." + }, + {"get_pid" , + (PyCFunction ) PyTepEvent_get_pid, +METH_VARARGS |METH_KEYWORDS , + }, + {NULL } +}; + +C_OBJECT_WRAPPER(tep_event, PyTepEvent, NO_FREE) + +static PyMethodDef PyTep_methods[] = { + {"init_local" , + (PyCFunction ) PyTep_init_local, +METH_VARARGS |METH_KEYWORDS , +"Initialize from local instance." + }, + {"get_event" , + (PyCFunction ) PyTep_get_event, +METH_VARARGS |METH_KEYWORDS , +"Get a PyTepEvent object." + }, + {NULL } +}; + +C_OBJECT_WRAPPER(tep_handle, PyTep, tep_free) + +static PyMethodDef ftracepy_methods[] = { + {"dir" , + (PyCFunction ) PyFtrace_dir, +METH_NOARGS , +"Get the absolute path to the tracefs directory." + }, + {"create_instance" , + (PyCFunction ) PyFtrace_create_instance, +METH_VARARGS |METH_KEYWORDS , +"Create new tracefs instance." + }, + {"get_all_instances" , + (PyCFunction ) PyFtrace_get_all_instances, +METH_NOARGS , +"Get all existing tracefs instances." + }, + {"destroy_instance" , + (PyCFunction ) PyFtrace_destroy_instance, +METH_VARARGS |METH_KEYWORDS , +"Destroy existing tracefs instance." + }, + {"destroy_all_instances" , + (PyCFunction ) PyFtrace_destroy_all_instances, +METH_NOARGS , +"Destroy all existing tracefs instances." + }, + {"instance_dir" , + (PyCFunction ) PyFtrace_instance_dir, +METH_VARARGS |METH_KEYWORDS , +"Get the absolute path to the instance directory." + }, + {"available_tracers" , + (PyCFunction ) PyFtrace_available_tracers, +METH_VARARGS |METH_KEYWORDS , +"Get a list of available tracers." + }, + {"set_current_tracer" , + (PyCFunction ) PyFtrace_set_current_tracer, +METH_VARARGS |METH_KEYWORDS , +"Enable a tracer." + }, + {"get_current_tracer" , + (PyCFunction ) PyFtrace_get_current_tracer, +METH_VARARGS |METH_KEYWORDS , +"Check the enabled tracer." + }, + {"available_event_systems" , + (PyCFunction ) PyFtrace_available_event_systems, +METH_VARARGS |METH_KEYWORDS , +"Get a list of available trace event systems." + }, + {"available_system_events" , + (PyCFunction ) PyFtrace_available_system_events, +METH_VARARGS |METH_KEYWORDS , +"Get a list of available trace event for a given system." + }, + {"enable_event" , + (PyCFunction ) PyFtrace_enable_event, +METH_VARARGS |METH_KEYWORDS , +"Enable trece event." + }, + {"disable_event" , + (PyCFunction ) PyFtrace_disable_event, +METH_VARARGS |METH_KEYWORDS , +"Disable trece event." + }, + {"enable_events" , + (PyCFunction ) PyFtrace_enable_events, +METH_VARARGS |METH_KEYWORDS , +"Enable multiple trece event." + }, + {"disable_events" , + (PyCFunction ) PyFtrace_disable_events, +METH_VARARGS |METH_KEYWORDS , +"Disable multiple trece event." + }, + {"event_is_enabled" , + (PyCFunction ) PyFtrace_event_is_enabled, +METH_VARARGS |METH_KEYWORDS , +"Check if event is enabled." + }, + {"tracing_ON" , + (PyCFunction ) PyFtrace_tracing_ON, +METH_VARARGS |METH_KEYWORDS , +"Start tracing." + }, + {"tracing_OFF" , + (PyCFunction ) PyFtrace_tracing_OFF, +METH_VARARGS |METH_KEYWORDS , +"Stop tracing." + }, + {"is_tracing_ON" , + (PyCFunction ) PyFtrace_is_tracing_ON, +METH_VARARGS |METH_KEYWORDS , +"Check if tracing is ON." + }, + {"set_event_pid" , + (PyCFunction ) PyFtrace_set_event_pid, +METH_VARARGS |METH_KEYWORDS , +"." + }, + {"set_ftrace_pid" , + (PyCFunction ) PyFtrace_set_ftrace_pid, +METH_VARARGS |METH_KEYWORDS , +"." + }, + {"enable_option" , + (PyCFunction ) PyFtrace_enable_option, +METH_VARARGS |METH_KEYWORDS , +"Enable trece option." + }, + {"disable_option" , + (PyCFunction ) PyFtrace_disable_option, +METH_VARARGS |METH_KEYWORDS , +"Disable trece option." + }, + {"option_is_set" , + (PyCFunction ) PyFtrace_option_is_set, +METH_VARARGS |METH_KEYWORDS , +"Check if trece option is enabled." + }, + {"supported_options" , + (PyCFunction ) PyFtrace_supported_options, +METH_VARARGS |METH_KEYWORDS , +"Gat a list of all supported options." + }, + {"enabled_options" , + (PyCFunction ) PyFtrace_enabled_options, +METH_VARARGS |METH_KEYWORDS , +"Gat a list of all supported options." + }, + {"trace_shell_process" , + (PyCFunction ) PyFtrace_trace_shell_process, +METH_VARARGS |METH_KEYWORDS , +"Trace a shell process." + }, + {NULL ,NULL ,0 ,NULL } +}; + +static struct PyModuleDef ftracepy_module = { + PyModuleDef_HEAD_INIT, +"ftracepy" , +"Python interface for Ftrace." , +-1 , + ftracepy_methods +}; + +PyMODINIT_FUNC PyInit_ftracepy (void ) +{ +if (!PyTepTypeInit ()) +return NULL ; + +if (!PyTepEventTypeInit ()) +return NULL ; + +if (!PyTepRecordTypeInit ()) +return NULL ; + +TFS_ERROR =PyErr_NewException ("tracecruncher.ftracepy.tfs_error" , +NULL ,NULL ); + +TEP_ERROR =PyErr_NewException ("tracecruncher.ftracepy.tep_error" , +NULL ,NULL ); + +TRACECRUNCHER_ERROR =PyErr_NewException ("tracecruncher.tc_error" , +NULL ,NULL ); + +PyObject *module =PyModule_Create (&ftracepy_module); + +PyModule_AddObject (module,"tep_handle" , (PyObject *) &PyTepType); +PyModule_AddObject (module,"tep_event" , (PyObject *) &PyTepEventType); +PyModule_AddObject (module,"tep_record" , (PyObject *) &PyTepRecordType); + +PyModule_AddObject (module,"tfs_error" ,TFS_ERROR ); +PyModule_AddObject (module,"tep_error" ,TEP_ERROR ); +PyModule_AddObject (module,"tc_error" ,TRACECRUNCHER_ERROR ); + +if (geteuid () !=0 ) { +PyErr_SetString (TFS_ERROR , +"Permission denied. Root privileges are required." ); +return NULL ; + } + +Py_AtExit (PyFtrace_at_exit); + +return module; +}
Re: [PATCH 1/9] trace-cruncher: Refactor the part that wraps ftrace
Hi Steven,
Thanks a lot for the review!
I will add all your fixes in v2.
Quoted 4 lines · Steven Rostedt
Quoted 4 lines
src/ftracepy-utils.c new file +4
-0+} + +static void start_tracing (struct tracefs_instance *instance, +char **argv,char **env)
Quoted 9 lines
src/ftracepy-utils.c line 1285
Quoted 6 lines
src/ftracepy-utils.c new file +6
-0+{ +if (tracefs_option_enable (instance,TRACEFS_OPTION_EVENT_FORK ) <0 || +tracefs_option_enable (instance,TRACEFS_OPTION_FUNCTION_FORK ) <0 || + !set_pid (instance,"set_ftrace_pid" ,getpid ()) || + !set_pid (instance,"set_event_pid" ,getpid ())) +exit (1 );So trace cruncher will only trace the given process and its children. There's no other alternative?
Of course not. This is just a static helper function that is used by the trace_shell_process(). It has no exposure to the user of the module.
trace_shell_process() on the other hand is part of the API of the module and indeed will trace only one process.
It is just the first method to be implemented. You will see more methods for tracing in the incoming patches.
But you are right that I have to rename start_tracing(), so that it doesn't sound like something that has a more general use.
Thanks!
Yordan
Quoted 82 lines
-- Steve
Quoted 78 lines
src/ftracepy-utils.c new file +78
-0+ +tracing_ON (instance); +if (execve (argv[0 ], argv, env) <0 ) { +PyErr_Format (TFS_ERROR ,"Failed to exec \'%s\'" , + argv[0 ]); + } + +exit (1 ); +} + +static int callback (struct tep_event *event,struct tep_record *record, +int cpu,void *py_func) +{ + record->cpu = cpu;// Remove when the bug in libtracefs is fixed. + +PyObject *py_tep_event =PyTepEvent_New (event); +PyObject *py_tep_record =PyTepRecord_New (record); + +PyObject *arglist =PyTuple_New (2 ); +PyTuple_SetItem (arglist,0 , py_tep_event); +PyTuple_SetItem (arglist,1 , py_tep_record); + +PyObject_CallObject ((PyObject *) py_func, arglist); + +return 0 ; +} + +PyObject *PyFtrace_trace_shell_process (PyObject *self,PyObject *args, +PyObject *kwargs) +{ +const char *plugin ="__main__" , *py_callback ="callback" ; +char *process, *instance_name; +struct tracefs_instance *instance; +static char *kwlist[] = {"process" ,"plugin" ,"callback" ,"instance" ,NULL }; +struct tep_handle *tep; +PyObject *py_func; +pid_t pid; + + instance_name =NO_ARG ; +if (!PyArg_ParseTupleAndKeywords (args, + kwargs, +"s|sss" , + kwlist, + &process, + &plugin, + &py_callback, + &instance_name)) { +return NULL ; + } + + py_func =get_callback_func (plugin, py_callback); +if (!py_func) +return NULL ; + +if (!get_optional_instance (instance_name, &instance)) +return NULL ; + + tep =tracefs_local_events (tracefs_instance_get_dir (instance)); + +char *argv[] = {getenv ("SHELL" ),"-c" , process,NULL }; +char *env[] = {NULL }; + + pid =fork (); +if (pid <0 ) { +PyErr_SetString (TFS_ERROR ,"Failed to fork" ); +return NULL ; + } + +if (pid ==0 ) +start_tracing (instance, argv, env); + +do { +tracefs_iterate_raw_events (tep, instance,NULL ,0 , callback, py_func); + }while (waitpid (pid,NULL ,WNOHANG ) != pid); + + Py_RETURN_NONE; +} +
PATCH 2/9 trace-cruncher: Refactor the part that wraps libkshark
In reply to Yordan Karadzhov (VMware)The part of the interface that relies on libkshark gets re-implemented as an extension called "tracecruncher.ksharkpy". The new extension gets build together with the previously implemented "tracecruncher.ftracefy" extension.
--- setup.py | 17 +- src/ksharkpy-utils.c | 411 ++++++++++++++++++++++++++++++++++++++++++ src/ksharkpy-utils.h | 41 +++++ src/ksharkpy.c | 94 ++++++++++ src/npdatawrapper.pyx | 203 +++++++++++++++++++++ src/trace2matrix.c | 40 ++++ 6 files changed, 804 insertions(+), 2 deletions(-) create mode 100644 src/ksharkpy-utils.c create mode 100644 src/ksharkpy-utils.h create mode 100644 src/ksharkpy.c create mode 100644 src/npdatawrapper.pyx create mode 100644 src/trace2matrix.c
setup.py +15 -2
@@ -11,22 +11,26 @@ from distutils.core import Extension from Cython.Build import cythonize import pkgconfig as pkg +import numpy as np def third_party_paths(): pkg_traceevent = pkg.parse('libtraceevent') pkg_ftracepy = pkg.parse('libtracefs') pkg_tracecmd = pkg.parse('libtracecmd') + pkg_kshark = pkg.parse('libkshark') - include_dirs = [] + include_dirs = [np.get_include()] include_dirs.extend(pkg_traceevent['include_dirs']) include_dirs.extend(pkg_ftracepy['include_dirs']) include_dirs.extend(pkg_tracecmd['include_dirs']) + include_dirs.extend(pkg_kshark['include_dirs']) library_dirs = [] library_dirs.extend(pkg_traceevent['library_dirs']) library_dirs.extend(pkg_ftracepy['library_dirs']) library_dirs.extend(pkg_tracecmd['library_dirs']) + library_dirs.extend(pkg_kshark['library_dirs']) library_dirs = list(set(library_dirs)) return include_dirs, library_dirs
@@ -48,6 +52,15 @@ def main(): sources=['src/ftracepy.c', 'src/ftracepy-utils.c'], libraries=['traceevent', 'tracefs']) + cythonize('src/npdatawrapper.pyx', language_level = "3") + module_data = extension(name='tracecruncher.npdatawrapper', + sources=['src/npdatawrapper.c'], + libraries=['kshark']) + + module_ks = extension(name='tracecruncher.ksharkpy', + sources=['src/ksharkpy.c', 'src/ksharkpy-utils.c'], + libraries=['kshark']) + setup(name='tracecruncher', version='0.1.0', description='NumPy based interface for accessing tracing data in Python.',
@@ -56,7 +69,7 @@ def main(): url='https://github.com/vmware/trace-cruncher', license='LGPL-2.1', packages=find_packages(), - ext_modules=[module_ft], + ext_modules=[module_ft, module_data, module_ks], classifiers=[ 'Development Status :: 3 - Alpha', 'Programming Language :: Python :: 3',
src/ksharkpy-utils.c new file +411 -0
411 lines not on this page · raw message
src/ksharkpy-utils.h new file +41 -0
41 lines not on this page · raw message
src/ksharkpy.c new file +94 -0
94 lines not on this page · raw message
src/npdatawrapper.pyx new file +203 -0
203 lines not on this page · raw message
src/trace2matrix.c new file +40 -0
@@ -0,0 +1,40 @@ +// SPDX-License-Identifier: LGPL-2.1 + +/* +* Copyright 2019 VMware Inc, Yordan Karadzhov <[email protected]> +*/ + +// KernelShark +#include "libkshark.h" + +ssize_t trace2matrix (int sd, +int16_t **event_array, +int16_t **cpu_array, +int32_t **pid_array, +int64_t **offset_array, +int64_t **ts_array) +{ +struct kshark_generic_stream_interface *interface; +struct kshark_context *kshark_ctx =NULL ; +struct kshark_data_stream *stream; +ssize_t total =0 ; + +if (!kshark_instance (&kshark_ctx)) +return -1 ; + + stream =kshark_get_data_stream (kshark_ctx, sd); +if (!stream) +return -1 ; + + interface = stream->interface; +if (interface->type ==KS_GENERIC_DATA_INTERFACE && + interface->load_matrix) { + total = interface->load_matrix (stream, kshark_ctx, event_array, + cpu_array, + pid_array, + offset_array, + ts_array); + } + +return total; +}
Signature · 1 line
2.27.0
PATCH 3/9 trace-cruncher: Add "utils"
In reply to Yordan Karadzhov (VMware)Place all the code, that is pure Python, in tracecrunche/ks_utils.py and tracecrunche/ft_utils.py
--- tracecruncher/__init__.py | 0 tracecruncher/ft_utils.py | 28 +++++ tracecruncher/ks_utils.py | 227 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 255 insertions(+) create mode 100644 tracecruncher/__init__.py create mode 100644 tracecruncher/ft_utils.py create mode 100644 tracecruncher/ks_utils.py
tracecruncher/__init__.py new file +0 -0
tracecruncher/ft_utils.py new file +28 -0
@@ -0,0 +1,28 @@ +""" +SPDX-License-Identifier: LGPL-2.1 + +Copyright 2019 VMware Inc, Yordan Karadzhov (VMware) <[email protected]> +""" + +import sys +import time +import hashlib + +from . import ftracepy as ft + + +def find_event_id(system, event): + """ Get the unique identifier of a trace event. + """ + tep = ft.tep_handle(); + tep.init_local(dir=ft.dir(), systems=[system]); + + return tep.get_event(system=system, name=event).id() + +def instance_autoname(): + """ Generates a quasi-random (unique) name for a trace instance. + """ + unique = ''.join(sys.argv) + str(time.time()) + hash = hashlib.sha1() + hash.update(unique.encode('utf-8')) + return hash.hexdigest()
tracecruncher/ks_utils.py new file +227 -0
227 lines not on this page · raw message
Signature · 1 line
2.27.0
PATCH 4/9 trace-cruncher: Refactor the examples
In reply to Yordan Karadzhov (VMware)For the moment we will keep only one example that is "sched_wakeup.py". "gpareto_fit.py" gets removed because it doesn't really demonstrate anything conceptually different compared to "sched_wakeup.py". The difference comes from the more advanced statistical analysis of the data, however this goes beyond the scope of trace-cruncher. "page_faults.py" gets removed only temporally, because it requires some functionalities that are not yet implemented in the ftrace libraries. Once those functionalities are made available, the example will be added to trace-cruncher again.
--- examples/gpareto_fit.py | 328 --------------------------------------- examples/page_faults.py | 120 -------------- examples/sched_wakeup.py | 70 ++++----- 3 files changed, 30 insertions(+), 488 deletions(-) delete mode 100755 examples/gpareto_fit.py delete mode 100755 examples/page_faults.py
examples/gpareto_fit.py deleted +0 -328
@@ -1,328 +0,0 @@ -#!/usr/bin/env python3 - -""" -SPDX-License-Identifier: LGPL-2.1 - -Copyright 2019 VMware Inc, Yordan Karadzhov <[email protected]> -""" - -import sys -import json - -import matplotlib.pyplot as plt -import scipy.stats as st -import numpy as np - -from scipy.stats import genpareto as gpareto -from scipy.optimize import curve_fit as cfit - -from ksharksetup import setup -# Always call setup() before importing ksharkpy!!! -setup() - -import ksharkpy as ks - -def chi2_test(hist, n_bins, c, loc, scale, norm): - """ Simple Chi^2 test for the goodness of the fit. - """ - chi2 = n_empty_bins = 0 - for i in range(len(hist[0])): - if hist[0][i] == 0: - # Ignore this empty bin. - n_empty_bins += 1 - continue - - # Get the center of bin i. - x = (hist[1][i] + hist[1][i + 1]) / 2 - fit_val = gpareto.pdf(x, c=c, loc=loc, scale=scale) - chi = (fit_val - hist[0][i]) / np.sqrt(hist[0][i]) - chi2 += chi**2 - - return norm * chi2 / (n_bins - n_empty_bins) - -def quantile(p, P, c, loc, scale): - """ The quantile function of the Generalized Pareto distribution. - """ - return loc + scale / c * ((P / p)**(c) - 1) - - -def dq_dscale(p, P, c, scale): - """ Partial derivative of the quantile function. - """ - return ((P / p)**c - 1) / c - - -def dq_dc(p, P, c, scale): - """ Partial derivative of the quantile function. - """ - return (scale * (np.log(P / p) * (P / p)**c ) / c - - scale * ((P / p)**c - 1) / (c**2)) - - -def dq_dP(p, P, c, scale): - """ Partial derivative of the quantile function. - """ - return scale / P * (P / p)**c - - -def error_P(n, N): - return np.sqrt(n) / N - - -def error(p, P, c, scale, err_P, err_c, err_scale): - return np.sqrt((dq_dP(p, P, c, scale) * err_P)**2 - + (dq_dc(p, P, c, scale) * err_c)**2 - + (dq_dscale(p, P, c, scale) * err_scale)**2) - - -def quantile_conf_bound(p, P, n, c, loc, scale, err_P, err_c, err_scale): - return (quantile(p=p, P=P, c=c, loc=loc, scale=scale) - + n * error(p=p, P=P, c=c, scale=scale, - err_P=err_P, err_c=err_c, err_scale=err_scale)); - - -def get_latency(t0, t1): - """ Get the value of the latency in microseconds - """ - return (t1 - t0) / 1000 - 1000 - - -def get_cpu_data(data, task_pid, start_id, stop_id, threshold): - """ Loop over the tracing data for a given CPU and find all latencies bigger - than the specified threshold. - """ - # Get the size of the data. - size = ks.data_size(data) - #print("data size:", size) - - time_start = -1 - dt_ot = [] - tot = 0 - i = 0 - i_start = 0; - - while i < size: - if data["pid"][i] == task_pid and data['event'][i] == start_id: - time_start = data['time'][i] - i_start = i; - i = i + 1 - - while i < size: - if data["pid"][i] == task_pid and data['event'][i] == stop_id: - delta = get_latency(time_start, data['time'][i]) - - if delta > threshold and tot != 0: - print('lat. over threshold: ', delta, i_start, i) - dt_ot.append([delta, i_start, i]) - - tot = tot + 1 - break - - i = i + 1 - i = i + 1 - - print(task_pid, 'tot:', len(dt_ot), '/', tot) - return dt_ot, tot - - -def make_ks_session(fname, data, start, stop): - """ Save a KernelShark session descriptor file (Json). - The sessions is zooming around the maximum observed latency. - """ - sname = 'max_lat.json' - ks.new_session(fname, sname) - i_start = int(start) - i_stop = int(stop) - - with open(sname, 'r+') as s: - session = json.load(s) - session['TaskPlots'] = [int(data['pid'][i_start])] - session['CPUPlots'] = [int(data['cpu'][i_start])] - - delta = data['time'][i_stop] - data['time'][i_start] - tmin = int(data['time'][i_start] - delta) - tmax = int(data['time'][i_stop] + delta) - session['Model']['range'] = [tmin, tmax] - - session['Markers']['markA']['isSet'] = True - session['Markers']['markA']['row'] = i_start) - - session['Markers']['markB']['isSet'] = True - session['Markers']['markB']['row'] = i_stop) - - session['ViewTop'] = i_start) - 5 - - ks.save_session(session, s) - - -fname = str(sys.argv[1]) -status = ks.open_file(fname) -if not status: - print ("Failed to open file ", fname) - sys.exit() - -ks.register_plugin('reg_pid') -data = ks.load_data() - -# Get the Event Ids of the hrtimer_start and print events. -start_id = ks.event_id('timer', 'hrtimer_start') -stop_id = ks.event_id('ftrace', 'print') -print("start_id", start_id) -print("stop_id", stop_id) - -tasks = ks.get_tasks() -jdb_pids = tasks['jitterdebugger'] -print('jitterdeburrer pids:', jdb_pids) -jdb_pids.pop(0) - -threshold = 10 -data_ot = [] -tot = 0 - -for task_pid in jdb_pids: - cpu_data, cpu_tot = get_cpu_data(data=data, - task_pid=task_pid, - start_id=start_id, - stop_id=stop_id, - threshold=threshold) - - data_ot.extend(cpu_data) - tot += cpu_tot - -ks.close() - -dt_ot = np.array(data_ot) -np.savetxt('peak_over_threshold_loaded.txt', dt_ot) - -make_ks_session(fname=fname, data=data, i_start=int(dt_ot[i_max_lat][1]), - i_stop=int(dt_ot[i_max_lat][2])) - -P = len(dt_ot) / tot -err_P = error_P(n=len(dt_ot), N=tot) -print('tot:', tot, ' P =', P) - -lat = dt_ot[:,0] -#print(lat) -i_max_lat = lat.argmax() -print('imax:', i_max_lat, int(dt_ot[i_max_lat][1])) - -print('max', np.amax(dt_ot)) - -start = threshold -stop = 31 -n_bins = (stop - start) * 2 - -bin_size = (stop - start) / n_bins - -x = np.linspace(start=start + bin_size / 2, - stop=stop - bin_size / 2, - num=n_bins) - -bins_ot = np.linspace(start=start, stop=stop, num=n_bins + 1) -#print(bins_ot) - -fig, ax = plt.subplots(nrows=2, ncols=2) -fig.tight_layout() -ax[-1, -1].axis('off') - -hist_ot = ax[0][0].hist(x=lat, bins=bins_ot, histtype='stepfilled', alpha=0.3) -ax[0][0].set_xlabel('latency [\u03BCs]', fontsize=8) -ax[0][0].set_yscale('log') -#print(hist_ot[0]) - -hist_ot_norm = ax[1][0].hist(x=lat, bins=bins_ot, - density=True, histtype='stepfilled', alpha=0.3) - -# Fit using the fitter of the genpareto class (shown in red). -ret = gpareto.fit(lat, loc=threshold) -ax[1][0].plot(x, gpareto.pdf(x, c=ret[0], loc=ret[1], scale=ret[2]), - 'r-', lw=1, color='red', alpha=0.8) - -ax[1][0].set_xlabel('latency [\u03BCs]', fontsize=8) -print(ret) -print('\ngoodness-of-fit: ' + '{:03.3f}'.format(chi2_test(hist_ot_norm, - n_bins=n_bins, - c=ret[0], - loc=ret[1], - scale=ret[2], - norm=len(lat)))) - -print("\n curve_fit:") -# Fit using the curve_fit fitter. Fix the value of the "loc" parameter. -popt, pcov = cfit(lambda x, c, scale: gpareto.pdf(x, c=c, loc=threshold, scale=scale), - x, hist_ot_norm[0], - p0=[ret[0], ret[2]]) - -print(popt) -print(pcov) - -ax[1][0].plot(x, gpareto.pdf(x, c=popt[0], loc=threshold, scale=popt[1]), - 'r-', lw=1, color='blue', alpha=0.8) - -fit_legend = str('\u03BE = ' + '{:05.3f}'.format(popt[0]) + - ' +- ' + '{:05.3f}'.format(pcov[0][0]**0.5) + - ' (' + '{:03.2f}'.format(pcov[0][0]**0.5 / abs(popt[0]) * 100) + '%)') - -fit_legend += str('\n\u03C3 = ' + '{:05.3f}'.format(popt[1]) + - ' +- ' + '{:05.3f}'.format(pcov[1][1]**0.5) + - ' (' + '{:03.2f}'.format(pcov[1][1]**0.5 / abs(popt[1]) * 100) + '%)') - -fit_legend += '\n\u03BC = ' + str(threshold) + ' (const)' - -fit_legend += '\ngoodness-of-fit: ' + '{:03.3f}'.format(chi2_test(hist_ot_norm, - n_bins=n_bins, - c=popt[0], - loc=threshold, - scale=popt[1], - norm=len(lat))) -print(fit_legend) - -ax[0][1].set_xscale('log') -##ax[0][1].set_yscale('log') -ax[0][1].set_xlabel('Return period', fontsize=8) -ax[0][1].set_ylabel('Return level [\u03BCs]', fontsize=6) -ax[0][1].grid(True, linestyle=":", which="both") - -y = np.linspace(200000, 5000000, 400) -ax[0][1].plot(y, - quantile(1 / y, - P=P, - c=popt[0], - loc=threshold, - scale=popt[1]), - 'r-', lw=1, color='blue', alpha=0.8) - -ax[0][1].plot(y, - quantile_conf_bound(1 / y, - P=P, - n=+1, - c=popt[0], - loc=threshold, - scale=popt[1], - err_P=err_P, - err_c= pcov[0][0]**0.5, - err_scale=pcov[1][1]**0.5), - 'r-', lw=1, color='green', alpha=0.8) - -ax[0][1].plot(y, - quantile_conf_bound(1 / y, - P=P, - n=-1, - c=popt[0], - loc=threshold, - scale=popt[1], - err_P=err_P, - err_c= pcov[0][0]**0.5, - err_scale=pcov[1][1]**0.5), - 'r-', lw=1, color='green', alpha=0.8) - -props = dict(boxstyle='round', color='black', alpha=0.05) - -ax[1][1].text(0.05, 0.85, - fit_legend, - fontsize=9, - verticalalignment='top', - bbox=props) - -plt.savefig('figfit-all-loaded.png') -#plt.show()
examples/page_faults.py deleted +0 -120
120 lines not on this page · raw message
examples/sched_wakeup.py +30 -40
132 lines not on this page · raw message
Signature · 1 line
2.27.0
PATCH 5/9 trace-cruncher: Add Makefile
In reply to Yordan Karadzhov (VMware)This will simplify the building procedure and will make it more intuitive.
--- Makefile | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 Makefile
Makefile new file +33 -0
@@ -0,0 +1,33 @@ +# +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright 2019 VMware Inc, Yordan Karadzhov (VMware) <[email protected]> +# + +UID := $(shell id -u) + +CYAN := '\e[36m' +PURPLE := '\e[35m' +NC := '\e[0m' + +all: + @ echo ${CYAN}Buildinging trace-cruncher:${NC}; + python3 setup.py build + +clean: + rm -f src/npdatawrapper.c + rm -rf build + +install: + @ echo ${CYAN}Installing trace-cruncher:${NC}; + python3 setup.py install --record install_manifest.txt + +uninstall: + @ if [ $(UID) -ne 0 ]; then \ + echo ${PURPLE}Permission denied${NC} 1>&2; \ + else \ + echo ${CYAN}Uninstalling trace-cruncher:${NC}; \ + xargs rm -v < install_manifest.txt; \ + rm -rfv dist tracecruncher.egg-info; \ + rm -fv install_manifest.txt; \ + fi
Signature · 1 line
2.27.0
PATCH 6/9 trace-cruncher: Update README.md
In reply to Yordan Karadzhov (VMware)Building instruction are updated in order to properly describe the refactored version and the installation of all third-party dependencies.
--- README.md | 84 +++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 54 insertions(+), 30 deletions(-)
README.md +54 -30
@@ -4,68 +4,92 @@ ## Overview -The Trace-Cruncher project aims to provide an interface between the existing instrumentation for collection and visualization of tracing data from the Linux kernel and the broad and very well developed ecosystem of instruments for data analysis available in Python. The interface will be based on NumPy. +The Trace-Cruncher project aims to provide an interface between the existing instrumentation for collection and visualization of tracing data from the Linux kernel and the broad and very well developed ecosystem of instruments for data analysis available in Python. The interface is based on NumPy. -NumPy implements an efficient multi-dimensional container of generic data and uses strong typing in order to provide fast data processing in Python. The Trace-Cruncher will allow for sophisticated analysis of kernel tracing data via scripts, but it will also opens the door for exposing the kernel tracing data to the instruments provided by the scientific toolkit of Python like MatPlotLib, Stats, Scikit-Learn and even to the nowadays most popular frameworks for Machine Learning like TensorFlow and PyTorch. The Trace-Cruncher is strongly coupled to the KernelShark project and is build on top of the C API of libkshark. +NumPy implements an efficient multi-dimensional container of generic data and uses strong typing in order to provide fast data processing in Python. The Trace-Cruncher allows for sophisticated analysis of kernel tracing data via scripts, but it also opens the door for exposing the kernel tracing data to the instruments provided by the scientific toolkit of Python like MatPlotLib, Stats, Scikit-Learn and even to the nowadays most popular frameworks for Machine Learning like TensorFlow and PyTorch. The Trace-Cruncher is strongly coupled to the KernelShark project and is build on top of the C API of libkshark. ## Try it out ### Prerequisites Trace-Cruncher has the following external dependencies: - trace-cmd / KernelShark, Json-C, Cython, NumPy, MatPlotLib. + libtraceevent, libtracefs, KernelShark, Json-C, Cython, NumPy. -1.1 In order to install the packages on Ubuntu do the following: +1.1 In order to install all packages on Ubuntu do the following: - sudo apt-get install libjson-c-dev libpython3-dev cython3 -y + > sudo apt-get update - sudo apt-get install python3-numpy python3-matplotlib -y + > sudo apt-get install build-essential git cmake libjson-c-dev -y -1.2 In order to install the packages on Fedora, as root do the following: + > sudo apt-get install libpython3-dev cython3 python3-numpy python3-pip -y - dnf install json-c-devel python3-devel python3-Cython -y + > sudo pip3 install --system pkgconfig GitPython - dnf install python3-numpy python3-matplotlib -y +1.2 In order to install all packages on Fedora, as root do the following: -2. In order to get the proper version of KernelShark / trace-cmd do the -following: + > dnf install gcc gcc-c++ git cmake json-c-devel -y - git clone git://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git --branch=kernelshark-v1.1 + > dnf install python3-devel python3-Cython python3-numpy python3-pip -y -or download a tarball from here: -https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git/snapshot/trace-cmd-kernelshark-v1.1.tar.gz + > sudo pip3 install --system pkgconfig GitPython -### Build & Run -1. Patch trace-cmd / KernelShark: +2 In order to install all third party libraries do the following: + + > git clone https://git.kernel.org/pub/scm/libs/libtrace/libtraceevent.git/ + + > cd libtraceevent + + > make + + > sudo make install + + > cd .. + + + > git clone https://git.kernel.org/pub/scm/libs/libtrace/libtracefs.git/ + + > cd libtracefs - cd path/to/trace-cmd/ + > make - git am ../path/to/trace-cruncher/0001-kernel-shark-Add-_DEVEL-build-flag.patch + > sudo make install - git am ../path/to/trace-cruncher/0002-kernel-shark-Add-reg_pid-plugin.patch + > cd .. -2. Install trace-cmd: - make + > git clone https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git - sudo make install_libs + > cd trace-cmd -3. Install KernelShark: + > make - cd kernel-shark/build + > sudo make install_libs - cmake -D_DEVEL=1 ../ + > cd .. - make - sudo make install + > git clone https://github.com/yordan-karadzhov/kernel-shark-v2.beta.git kernel-shark + + > cd kernel-shark/build + + > cmake .. + + > make + + > sudo make install + + > cd ../.. + +### Build & Run + +Installing trace-cruncher is very simple. After downloading the source code, you just have to run: -4. Build the NumPy API itself: + > cd trace-cruncher - cd path/to/trace-cruncher + > make - ./np_setup.py build_ext -i + > sudo make install ## Documentation
Signature · 1 line
2.27.0
PATCH 7/9 trace-cruncher: Remove all leftover files.
In reply to Yordan Karadzhov (VMware)This patch completes the refactoring of trace-cruncher into a Python module. All obsoleted source files are removed.
--- 0001-kernel-shark-Add-_DEVEL-build-flag.patch | 90 ----- 0002-kernel-shark-Add-reg_pid-plugin.patch | 231 ----------- clean.sh | 6 - examples/ksharksetup.py | 24 -- libkshark-py.c | 224 ----------- libkshark_wrapper.pyx | 361 ------------------ np_setup.py | 90 ----- 7 files changed, 1026 deletions(-) delete mode 100644 0001-kernel-shark-Add-_DEVEL-build-flag.patch delete mode 100644 0002-kernel-shark-Add-reg_pid-plugin.patch delete mode 100755 clean.sh delete mode 100644 examples/ksharksetup.py delete mode 100644 libkshark-py.c delete mode 100644 libkshark_wrapper.pyx delete mode 100755 np_setup.py
0001-kernel-shark-Add-_DEVEL-build-flag.patch deleted +0 -90
@@ -1,90 +0,0 @@ -From 6c9e3b3f29c8af4780bb46313c3af73fb5d852c7 Mon Sep 17 00:00:00 2001 -From: "Yordan Karadzhov (VMware)" <[email protected]> -Date: Fri, 20 Sep 2019 14:31:15 +0300 -Subject: [PATCH 1/2] kernel-shark: Add _DEVEL build flag - -KernelShark can be built with -D_DEVEL=1 as a command-line argument -for Cmake. In this case the headers of the libraries will be installed -as well and a symbolic link that points to the version of the library -being installed will be created. - -Signed-off-by: Yordan Karadzhov (VMware) <[email protected]> ---- - kernel-shark/README | 3 +++ - kernel-shark/src/CMakeLists.txt | 33 +++++++++++++++++++++++++++++++++ - 2 files changed, 36 insertions(+) - -diff --git a/kernel-shark/README b/kernel-shark/README -index 6c360bb..0f14212 100644 ---- a/kernel-shark/README -+++ b/kernel-shark/README -@@ -96,6 +96,9 @@ the dialog will derive the absolut path to the trace-cmd executable from - - If no build types is specified, the type will be "RelWithDebInfo". - -+2.1.4 In order to install a development version (including headers e.t.c) add -+-D_DEVEL=1 as a CMake Command-Line option. -+ - Examples: - - cmake -D_DOXYGEN_DOC=1 -D_INSTALL_PREFIX=/usr ../ -diff --git a/kernel-shark/src/CMakeLists.txt b/kernel-shark/src/CMakeLists.txt -index e20a030..305840b 100644 ---- a/kernel-shark/src/CMakeLists.txt -+++ b/kernel-shark/src/CMakeLists.txt -@@ -1,5 +1,13 @@ - message("\n src ...") - -+macro(install_symlink filepath sympath) -+ install(CODE "execute_process(COMMAND ${CMAKE_COMMAND} -E create_symlink ${filepath} ${sympath})") -+ install(CODE "LIST(APPEND CMAKE_INSTALL_MANIFEST_FILES ${sympath})") -+ install(CODE "message(\"-- Created symlink: ${sympath} -> ${filepath}\")") -+endmacro(install_symlink) -+ -+set(KS_INCLUDS_DESTINATION "${_INSTALL_PREFIX}/include/${KS_APP_NAME}") -+ - message(STATUS "libkshark") - add_library(kshark SHARED libkshark.c - libkshark-model.c -@@ -16,6 +24,19 @@ set_target_properties(kshark PROPERTIES SUFFIX ".so.${KS_VERSION_STRING}") - - install(TARGETS kshark LIBRARY DESTINATION ${_INSTALL_PREFIX}/lib/${KS_APP_NAME}) - -+if (_DEVEL) -+ -+ install_symlink("libkshark.so.${KS_VERSION_STRING}" -+ "${_INSTALL_PREFIX}/lib/${KS_APP_NAME}/libkshark.so") -+ -+ install(FILES "${KS_DIR}/src/libkshark.h" -+ "${KS_DIR}/src/libkshark-plugin.h" -+ "${KS_DIR}/src/libkshark-model.h" -+ DESTINATION ${KS_INCLUDS_DESTINATION} -+ COMPONENT devel) -+ -+endif (_DEVEL) -+ - if (OPENGL_FOUND AND GLUT_FOUND) - - message(STATUS "libkshark-plot") -@@ -30,6 +51,18 @@ if (OPENGL_FOUND AND GLUT_FOUND) - - install(TARGETS kshark-plot LIBRARY DESTINATION ${_INSTALL_PREFIX}/lib/${KS_APP_NAME}) - -+ if (_DEVEL) -+ -+ install_symlink("libkshark-plot.so.${KS_VERSION_STRING}" -+ "${_INSTALL_PREFIX}/lib/${KS_APP_NAME}/libkshark-plot.so") -+ -+ install(FILES "${KS_DIR}/src/KsPlotTools.hpp" -+ "${KS_DIR}/src/libkshark-plot.h" -+ DESTINATION ${KS_INCLUDS_DESTINATION} -+ COMPONENT devel) -+ -+ endif (_DEVEL) -+ - endif (OPENGL_FOUND AND GLUT_FOUND) - - if (Qt5Widgets_FOUND AND Qt5Network_FOUND) --- -2.20.1 -
0002-kernel-shark-Add-reg_pid-plugin.patch deleted +0 -231
231 lines not on this page · raw message
clean.sh deleted +0 -6
@@ -1,6 +0,0 @@ -#!/bin/bash - -rm libkshark_wrapper.c -rm ksharkpy.cpython-3*.so -rm -rf build/ -rm -rf examples/__pycache__/
examples/ksharksetup.py deleted +0 -24
@@ -1,24 +0,0 @@ -#!/usr/bin/env python3 - -""" -SPDX-License-Identifier: LGPL-2.1 - -Copyright 2019 VMware Inc, Yordan Karadzhov <[email protected]> -""" - -import os -import sys - -def setup(): - os.chdir(os.path.dirname(__file__)) - path = os.getcwd() + '/..' - sys.path.append(path) - - if 'LD_LIBRARY_PATH' not in os.environ: - os.environ['LD_LIBRARY_PATH'] = '/usr/local/lib/kernelshark:/usr/local/lib/traceevent:/usr/local/lib/trace-cmd' - try: - os.execv(sys.argv[0], sys.argv) - except e: - print('Failed re-exec:', e) - sys.exit(1) -
libkshark-py.c deleted +0 -224
224 lines not on this page · raw message
libkshark_wrapper.pyx deleted +0 -361
361 lines not on this page · raw message
np_setup.py deleted +0 -90
90 lines not on this page · raw message
Signature · 1 line
2.27.0
PATCH 8/9 trace-cruncher: Add testing
In reply to Yordan Karadzhov (VMware)Add basic infrastructure for CI testing.
--- tests/0_get_data/__init__.py | 0 tests/0_get_data/test_get_data.py | 26 + tests/1_unit/__init__.py | 0 tests/1_unit/test_01_ftracepy_unit.py | 454 ++++++++++++++++++ tests/1_unit/test_02_datawrapper_unit.py | 41 ++ tests/1_unit/test_03_ksharkpy_unit.py | 72 +++ tests/2_integration/__init__.py | 0 .../test_01_ftracepy_integration.py | 113 +++++ .../test_03_ksharkpy_integration.py | 25 + tests/__init__.py | 0 10 files changed, 731 insertions(+) create mode 100644 tests/0_get_data/__init__.py create mode 100755 tests/0_get_data/test_get_data.py create mode 100644 tests/1_unit/__init__.py create mode 100644 tests/1_unit/test_01_ftracepy_unit.py create mode 100755 tests/1_unit/test_02_datawrapper_unit.py create mode 100755 tests/1_unit/test_03_ksharkpy_unit.py create mode 100644 tests/2_integration/__init__.py create mode 100755 tests/2_integration/test_01_ftracepy_integration.py create mode 100755 tests/2_integration/test_03_ksharkpy_integration.py create mode 100644 tests/__init__.py
tests/0_get_data/__init__.py new file +0 -0
tests/0_get_data/test_get_data.py new file +26 -0
@@ -0,0 +1,26 @@ +#!/usr/bin/env python3 + +""" +SPDX-License-Identifier: LGPL-2.1 + +Copyright (C) 2021 VMware Inc, Yordan Karadzhov (VMware) <[email protected]> +""" + +import os +import sys +import shutil +import unittest +import git + +class GetTestData(unittest.TestCase): + def test_get_data(self): + data_dir = 'testdata' + if os.path.exists(data_dir) and os.path.isdir(data_dir): + shutil.rmtree(data_dir) + + github_repo = 'https://github.com/yordan-karadzhov/kernel-shark_testdata.git' + repo = git.Repo.clone_from(github_repo, data_dir) + + +if __name__ == '__main__': + unittest.main()
tests/1_unit/__init__.py new file +0 -0
tests/1_unit/test_01_ftracepy_unit.py new file +454 -0
454 lines not on this page · raw message
tests/1_unit/test_02_datawrapper_unit.py new file +41 -0
41 lines not on this page · raw message
tests/1_unit/test_03_ksharkpy_unit.py new file +72 -0
72 lines not on this page · raw message
tests/2_integration/__init__.py new file +0 -0
tests/2_integration/test_01_ftracepy_integration.py new file +113 -0
113 lines not on this page · raw message
tests/2_integration/test_03_ksharkpy_integration.py new file +25 -0
@@ -0,0 +1,25 @@ +#!/usr/bin/env python3 + +""" +SPDX-License-Identifier: LGPL-2.1 + +Copyright (C) 2021 VMware Inc, Yordan Karadzhov (VMware) <[email protected]> +""" + +import os +import sys +import shutil +import unittest +import tracecruncher.ks_utils as tc + +class GetTestData(unittest.TestCase): + def test_open_and_read(self): + f = tc.open_file(file_name='testdata/trace_test1.dat') + data = f.load(pid_data=False) + tasks = f.get_tasks() + self.assertEqual(len(tasks), 29) + self.assertEqual(tasks['zoom'], [28121, 28137, 28141, 28199, 28201, 205666]) + + +if __name__ == '__main__': + unittest.main()
tests/__init__.py new file +0 -0
Signature · 1 line
2.27.0
PATCH 9/9 trace-cruncher: Add github workflow for CI testing
In reply to Yordan Karadzhov (VMware)The CI workflow will run once a week, or on any push to branches "master" and "yordan_devel".
--- .github/workflows/main.yml | 58 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 .github/workflows/main.yml
.github/workflows/main.yml new file +58 -0
@@ -0,0 +1,58 @@ +name: trace-cruncher CI + +on: + push: + branches: [master, yordan_devel] + schedule: + - cron: '0 15 * * THU' + +jobs: + build: + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v2 + + - name: Install Dependencies + shell: bash + run: | + sudo apt-get update + sudo apt-get install build-essential git cmake libjson-c-dev -y + sudo apt-get install libpython3-dev cython3 python3-numpy -y + sudo apt install python3-pip + sudo pip3 install --system pkgconfig GitPython + git clone https://git.kernel.org/pub/scm/libs/libtrace/libtraceevent.git/ + cd libtraceevent + make + sudo make install + cd .. + git clone https://git.kernel.org/pub/scm/libs/libtrace/libtracefs.git/ + cd libtracefs + make + sudo make install + cd .. + git clone https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git + cd trace-cmd + make + sudo make install_libs + cd .. + git clone https://github.com/yordan-karadzhov/kernel-shark-v2.beta.git kernel-shark + cd kernel-shark/build + cmake .. + make + sudo make install + cd ../.. + + - name: Build + working-directory: ${{runner.workspace}}/trace-cruncher + shell: bash + # Build and install. + run: | + make + sudo make install + + - name: Test + working-directory: ${{runner.workspace}}/trace-cruncher/tests + shell: bash + # Execute tests defined by the CMake configuration. + run: sudo python3 -m unittest discover .
Signature · 1 line
2.27.0