Skip to contentSkip to conversation

Start KernelShark v2 transformation

PATCH Superseded Yordan Karadzhov (VMware) 2 participants 47 messages Oct 12, 2020 – Nov 9, 2020

v2 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v2 cover Start KernelShark v2 transformation

In this patch-set we are starting the introduction of the changes in the C API of libkshark. The goal of this transformation is to turn KernelShark into a general purpose toolkit for analysis of tracing data that is no longer strongly coupled to the Ftrace/trace-cmd data format. At the end of the transformation KernelShark will be able to merge and simultaneously visualize data from different tracers, recorded in different data formats. It will also significantly extend the possibilities for user customization.

This patch-set transforms the way KernelShark processes the raw tracing data. Later, in a consecutive patch-sets we will introduce the changes in the visualization instruments and in the KernelShark GUI itself.

Changes is v2:
* Tried to address review comments for "kernel-shark: Introduce libkshark-hash"

* The huge patch "kernel-shark: Integrate the stream definitions with
the C API" in the previous version is now split into 8 smaller patches. The last of these 8 new patches (kernel-shark: Complete the stream integration) is still a bit entangled. I can try to further break this patch in future versions of the patch-set, if this is needed.

Steven,
Please review very carefully patches:
kernel-shark: Introduce Data streams
kernel-shark: Add basic methods for Data streams
kernel-shark: Add stream interface for trace-cmd data
kernel-shark: Start using data streams
kernel-shark: Provide merging of multiple data streams
kernel-shark: Add methods for time calibration

Please review VERY carefully patche:
kernel-shark: Redesign the plugin interface

You may treat this one as a complete rewrite. Also it may be useful to see how the redesigned code is being used in the new version of the sched_events plugin here:
https://github.com/yordan-karadzhov/kernel-shark-v2.beta/blob/master/src/plugins/sched_events.c
https://github.com/yordan-karadzhov/kernel-shark-v2.beta/blob/master/src/plugins/SchedEvents.cpp

Thanks!
Yordan

Yordan Karadzhov (VMware) (20):
kernel-shark: Start introducing KernelShark 2.0
kernel-shark: Use only signed types in kshark_entry
kernel-shark: Introduce libkshark-hash
kernel-shark: Introduce Data streams
kernel-shark: Add stream_id to kshark_entry
kernel-shark: Rename static methods in libkshark
kernel-shark: Add basic methods for Data streams
kernel-shark: Housekeeping before implementing stream interface
kernel-shark: Add stream interface for trace-cmd data
kernel-shark: Start using data streams
kernel-shark: Remove dead code
kernel-shark: Redesign the plugin interface
kernel-shark: Complete the stream integration
kernel-shark: Provide merging of multiple data streams
kernel-shark: Integrate the stream definitions with data model
kernel-shark: Use only signed types for model defs
kernel-shark: Add ksmodel_get_bin()
kernel-shark: Protect ksmodel_set_in_range_bining()
kernel-shark: Add methods for time calibration
kernel-shark: Integrate streams with libkshark-configio

 CMakeLists.txt             |   10 +-
 build/deff.h.cmake         |   14 +-
 examples/CMakeLists.txt    |   34 +-
 examples/configio.c        |   20 +-
 examples/datafilter.c      |   67 +-
 examples/datahisto.c       |   38 +-
 examples/dataload.c        |   21 +-
 examples/multibufferload.c |   60 +
 src/CMakeLists.txt         |    7 +-
 src/libkshark-collection.c |  121 +-
 src/libkshark-configio.c   | 1173 +++++++++++++++---
 src/libkshark-hash.c       |  239 ++++
 src/libkshark-model.c      |  154 ++-
 src/libkshark-model.h      |   55 +-
 src/libkshark-plugin.c     |  597 ++++++++--
 src/libkshark-plugin.h     |  279 +++--
 src/libkshark-tepdata.c    | 1754 +++++++++++++++++++++++++++
 src/libkshark-tepdata.h    |  100 ++
 src/libkshark.c            | 2302 +++++++++++++++++-------------------
 src/libkshark.h            |  779 ++++++++++--
 20 files changed, 5908 insertions(+), 1916 deletions(-)
 create mode 100644 examples/multibufferload.c
 create mode 100644 src/libkshark-hash.c
 create mode 100644 src/libkshark-tepdata.c
 create mode 100644 src/libkshark-tepdata.h
Signature · 1 line
2.25.1
#1 lore raw

PATCH v2 1/20 kernel-shark: Start introducing KernelShark 2.0

This is the very first patch from a series of patches that will introduce KernelShark 2.0. The new version implies significant changes in the low level C API, used by the KernelShark GUI. As a consequence of the API modifications, significant changes are needed in every source file of the project. Because, we want to avoid having a huge and ugly single commit that will be extremely hard to review, we will introduce a regression. We will first disable the compilation of all sub-components of KernelShark, except the API itself. This is done by bypassing the search of the necessary third party components (OpenGL and Qt). Later we will introduce the changes of the API in several sub-steps and will re-enable one by one the modified components of the GUI.

---
 CMakeLists.txt          | 10 ++++----
 build/deff.h.cmake      | 14 ++---------
 examples/CMakeLists.txt | 54 ++++++++++++++++++++---------------------
 src/CMakeLists.txt      |  6 ++---
 4 files changed, 37 insertions(+), 47 deletions(-)
CMakeLists.txt +5 -5
index c95249e9..ccca95c1 100644
@@ -26,12 +26,12 @@ include(${KS_DIR}/build/FindJSONC.cmake)
 
 find_package(Doxygen)
 
-set(OpenGL_GL_PREFERENCE LEGACY)
-find_package(OpenGL)
-find_package(GLUT)
+# set(OpenGL_GL_PREFERENCE LEGACY)
+# find_package(OpenGL)
+# find_package(GLUT)
 
-find_package(Qt5Widgets 5.7.1)
-find_package(Qt5Network)
+# find_package(Qt5Widgets 5.7.1)
+# find_package(Qt5Network)
 if (Qt5Widgets_FOUND)
 
     message(STATUS "Found Qt5Widgets:  (version ${Qt5Widgets_VERSION})")
build/deff.h.cmake +2 -12
index 79726ff6..e398c0ca 100644
@@ -23,17 +23,7 @@
 /** "pkexec" executable. */
 #cmakedefine DO_AS_ROOT "@DO_AS_ROOT@"
 
-#ifdef __cplusplus
-
-	#include <QString>
-
-	/**
-	 * String containing semicolon-separated list of plugin names.
-	 * The plugins to be loaded when KernelShark starts are tagged
-	 * with "default".
-	 */
-	const QString plugins = "@PLUGINS@";
-
-#endif /* __cplusplus */
+/** Semicolon-separated list of plugin names. */
+#define KS_BUILTIN_PLUGINS "@PLUGINS@"
 
 #endif // _KS_CONFIG_H
examples/CMakeLists.txt +27 -27

57 lines not on this page · raw message

src/CMakeLists.txt +3 -3
index 457c1007..2e092b2e 100644
@@ -2,9 +2,9 @@ message("\n src ...")
 
 message(STATUS "libkshark")
 add_library(kshark SHARED libkshark.c
-                          libkshark-model.c
+#                           libkshark-model.c
                           libkshark-plugin.c
-                          libkshark-configio.c
+#                           libkshark-configio.c
                           libkshark-collection.c)
 
 target_link_libraries(kshark ${TRACECMD_LIBRARY}
@@ -99,7 +99,7 @@ if (Qt5Widgets_FOUND AND Qt5Network_FOUND)
 
 endif (Qt5Widgets_FOUND AND Qt5Network_FOUND)
 
-add_subdirectory(plugins)
+# add_subdirectory(plugins)
 
 find_program(DO_AS_ROOT pkexec)
 
Signature · 1 line
2.25.1
#2 lore raw

PATCH v2 2/20 kernel-shark: Use only signed types in kshark_entry

In reply to Yordan Karadzhov (VMware)

Using uint64_t for the value of the offset was just wrong. According to the POSIX standard off_t is a signed integer type with unspecified size. Here we stick to a 64 bit integer, because this size guaranties optimal packing of the kshark_entry structure.

Using unsigned values for the timestamps is also a source of problems and has been a reason for the introduction of multiple bugs in the past. In principal the value of the timestamps cannot be negative. However, this value must have the same type as the values used to define the state of the visualization model, like the range of the model or the size of the bin. The model state definitions should not take negative values as well, however their values are recalculated automatically when the user browses the data and those calculations may result in negative values in some corner cases. Because of this it is better to use a signed integer type and treat the negative values as an indicator of an error rather than have the negative result of the calculations casted into unsigned type which results into unpredictable behavior of the model.

---
 src/libkshark.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
src/libkshark.h +2 -2
index 0d6c50d1..9eecc2d0 100644
@@ -61,7 +61,7 @@ struct kshark_entry {
 	int32_t		event_id;
 
 	/** The offset into the trace file, used to find the record. */
-	uint64_t	offset;
+	int64_t		offset;
 
 	/**
 	 * The time of the record in nano seconds. The value is taken from
@@ -69,7 +69,7 @@ struct kshark_entry {
 	 * dependent. The time usually is the timestamp from when the system
 	 * started.
 	 */
-	uint64_t	ts;
+	int64_t		ts;
 };
 
 /** Size of the task's hash table. */
Signature · 1 line
2.25.1
#3 lore raw

PATCH v2 3/20 kernel-shark: Introduce libkshark-hash

In reply to Yordan Karadzhov (VMware)

So far KernelShark have been using an implementation of a hash table from trace-cmd/include/trace-cmd/trace-filter-hash.h. However it turns that KernelShark is the only user of trace-filter-hash, which means that it make more sense to make this implementation of the hash table part of KernelShark. In this patch we adapt the original trace-cmd implementation and change the naming convention used.

---
 src/CMakeLists.txt   |   1 +
 src/libkshark-hash.c | 239 +++++++++++++++++++++++++++++++++++++++++++
 src/libkshark.h      |  46 +++++++++
 3 files changed, 286 insertions(+)
 create mode 100644 src/libkshark-hash.c
src/CMakeLists.txt +1 -0
index 2e092b2e..39c4dcf0 100644
@@ -2,6 +2,7 @@ message("\n src ...")
 
 message(STATUS "libkshark")
 add_library(kshark SHARED libkshark.c
+                          libkshark-hash.c
 #                           libkshark-model.c
                           libkshark-plugin.c
 #                           libkshark-configio.c
src/libkshark-hash.c new file +239 -0

239 lines not on this page · raw message

src/libkshark.h +46 -0

52 lines not on this page · raw message

Signature · 1 line
2.25.1
#4 lore raw

Re: [PATCH v2 03/20] kernel-shark: Introduce libkshark-hash

Quoted 9 lines · On Mon, 12 Oct 2020 16:35:06 +0300 "Yordan Karadzhov (VMware)"…

So far KernelShark have been using an implementation of a hash table from trace-cmd/include/trace-cmd/trace-filter-hash.h. However it turns that KernelShark is the only user of trace-filter-hash, which means that it make more sense to make this implementation of the hash table part of KernelShark. In this patch we adapt the original trace-cmd implementation and change the naming convention used.

---

I feel at least the first two patches do not need to be in the "disabled" section. Can we move them before patch 1?

-- Steve

#5 lore raw

Re: [PATCH v2 03/20] kernel-shark: Introduce libkshark-hash

Quoted 10 lines · On Mon, 12 Oct 2020 10:05:16 -0400 Steven Rostedt <[email protected]> wrote:
Quoted 7 lines · On Mon, 12 Oct 2020 16:35:06 +0300 "Yordan Karadzhov (VMware)"…

So far KernelShark have been using an implementation of a hash table from trace-cmd/include/trace-cmd/trace-filter-hash.h. However it turns that KernelShark is the only user of trace-filter-hash, which means that it make more sense to make this implementation of the hash table part of KernelShark. In this patch we adapt the original trace-cmd implementation and change the naming convention used.

Quoted 5 lines
Quoted 2 lines

---

I feel at least the first two patches do not need to be in the "disabled" section. Can we move them before patch 1?

I meant patch 2 and 3 before patch 1.

-- Steve

#6 lore raw

Re: [PATCH v2 03/20] kernel-shark: Introduce libkshark-hash

Quoted 13 lines · Steven Rostedt
Quoted 10 lines · On Mon, 12 Oct 2020 10:05:16 -0400 Steven Rostedt <[email protected]> wrote:
Quoted 7 lines · On Mon, 12 Oct 2020 16:35:06 +0300 "Yordan Karadzhov (VMware)"…

So far KernelShark have been using an implementation of a hash table from trace-cmd/include/trace-cmd/trace-filter-hash.h. However it turns that KernelShark is the only user of trace-filter-hash, which means that it make more sense to make this implementation of the hash table part of KernelShark. In this patch we adapt the original trace-cmd implementation and change the naming convention used.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

---

I feel at least the first two patches do not need to be in the "disabled" section. Can we move them before patch 1?

I meant patch 2 and 3 before patch 1.

OK, thanks!

I will change this in v3.
Y.

Quoted 2 lines

-- Steve

#7 lore raw

PATCH v2 4/20 kernel-shark: Introduce Data streams

In reply to Yordan Karadzhov (VMware)

With the help of Data stream, KernelShark will be able to load and merge multiple trace files (streams). Each stream can have different plugins or filters, registered for it, which means that the raw trace data of the streams can have different formats, and will allow for a great degree of customization of the provided data visualization. In this patch we only provide the basic definitions. The actual integration of the Data streams into the C API of KernelShark will happen in the following patches.

---
 src/libkshark.h | 201 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 201 insertions(+)
src/libkshark.h +201 -0
index 0182cf6d..1165c512 100644
@@ -118,6 +118,207 @@ void kshark_hash_id_free(struct kshark_hash_id *hash);
 
 int *kshark_hash_ids(struct kshark_hash_id *hash);
 
+struct kshark_data_stream;
+
+/** A function type to be used to initialize the interface of the data stream. */
+typedef int (*interface_init_func) (struct kshark_data_stream *,
+				    const char *);
+
+/** A function type to be used to initialize the interface of the data stream. */
+typedef int (*interface_close_func) (struct kshark_data_stream *,
+				     const char *);
+
+/** A function type to be used by the method interface of the data stream. */
+typedef char *(*stream_get_str_func) (struct kshark_data_stream *,
+				      const struct kshark_entry *);
+
+/** A function type to be used by the method interface of the data stream. */
+typedef const int (*stream_get_int_func) (struct kshark_data_stream *,
+					  const struct kshark_entry *);
+
+/** A function type to be used by the method interface of the data stream. */
+typedef int (*stream_find_id_func) (struct kshark_data_stream *,
+				    const char *);
+
+/** A function type to be used by the method interface of the data stream. */
+typedef int *(*stream_get_ids_func) (struct kshark_data_stream *);
+
+/** A function type to be used by the method interface of the data stream. */
+typedef int (*stream_get_names_func) (struct kshark_data_stream *,
+				      const struct kshark_entry *,
+				      char ***);
+
+/** Event field format identifier. */
+typedef enum kshark_event_field_format {
+	/** A field of unknown type. */
+	KS_INVALIDE_FIELD,
+
+	/** Integer number */
+	KS_INTEGER_FIELD,
+
+	/** Floating-point number */
+	KS_FLOAT_FIELD
+} kshark_event_field_format;
+
+/** A function type to be used by the method interface of the data stream. */
+typedef kshark_event_field_format
+(*stream_event_field_type) (struct kshark_data_stream *,
+			    const struct kshark_entry *,
+			    const char *);
+
+/** A function type to be used by the method interface of the data stream. */
+typedef const int (*stream_read_event_field) (struct kshark_data_stream *,
+					      const struct kshark_entry *,
+					      const char *,
+					      int64_t *);
+
+/** A function type to be used by the method interface of the data stream. */
+typedef const int (*stream_read_record_field) (struct kshark_data_stream *,
+					       void *,
+					       const char *,
+					       int64_t *);
+
+struct kshark_context;
+
+/** A function type to be used by the method interface of the data stream. */
+typedef ssize_t (*load_entries_func) (struct kshark_data_stream *,
+				      struct kshark_context *,
+				      struct kshark_entry ***);
+
+/** A function type to be used by the method interface of the data stream. */
+typedef ssize_t (*load_matrix_func) (struct kshark_data_stream *,
+				     struct kshark_context *,
+				     int16_t **cpu_array,
+				     int32_t **pid_array,
+				     int32_t **event_array,
+				     int64_t **offset_array,
+				     int64_t **ts_array);
+
+/** Data format identifier. */
+typedef enum kshark_data_format {
+	/** A data of unknown type. */
+	KS_INVALIDE_DATA,
+
+	/** Ftrace data. */
+	KS_TEP_DATA,
+
+	/** VMware SchedTrace data. */
+	KS_VMW_ST_DATA,
+} kshark_data_format;
+
+/**
+ * Structure representing the interface of methods used to operate over
+ * the data from a given stream.
+ */
+struct kshark_data_stream_interface {
+	/** Method used to retrieve the Process Id of the entry. */
+	stream_get_int_func	get_pid;
+
+	/** Method used to retrieve the Event Id of the entry. */
+	stream_get_int_func	get_event_id;
+
+	/** Method used to retrieve the Event name of the entry. */
+	stream_get_str_func	get_event_name;
+
+	/** Method used to retrieve the Task name of the entry. */
+	stream_get_str_func	get_task;
+
+	/** Method used to retrieve the Latency of the entry. */
+	stream_get_str_func	get_latency;
+
+	/** Method used to retrieve the Info string of the entry. */
+	stream_get_str_func	get_info;
+
+	/** Method used to retrieve Id of the Event from its name. */
+	stream_find_id_func	find_event_id;
+
+	/** Method used to retrieve the array of Ids of all Events. */
+	stream_get_ids_func	get_all_event_ids;
+
+	/** Method used to dump the entry's content to string. */
+	stream_get_str_func	dump_entry;
+
+	/**
+	 * Method used to retrieve the array of all field names of a given
+	 * event.
+	 */
+	stream_get_names_func	get_all_field_names;
+
+	/** Method used to access the type of an event's data field. */
+	stream_event_field_type		get_event_field_type;
+
+	/** Method used to access the value of an event's data field. */
+	stream_read_event_field		read_event_field_int64;
+
+	/** Method used to access the value of an event's data field. */
+	stream_read_record_field	read_record_field_int64;
+
+	/** Method used to load the data in the form of entries. */
+	load_entries_func	load_entries;
+
+	/** Method used to load the data in matrix form. */
+	load_matrix_func	load_matrix;
+
+	/** Generic data handle. */
+	void			*handle;
+};
+
+/** Structure representing a stream of trace data. */
+struct kshark_data_stream {
+	/** Data stream identifier. */
+	uint8_t			stream_id;
+
+	/** The number of CPUs presented in this data stream. */
+	int			n_cpus;
+
+	/**
+	 * The number of distinct event types presented in this data stream.
+	 */
+	int 			n_events;
+
+	/** The Process Id of the Idle task. */
+	int			idle_pid;
+
+	/** Trace data file pathname. */
+	char			*file;
+
+	/** Stream name. */
+	char			*name;
+
+	/** Hash table of task PIDs. */
+	struct kshark_hash_id	*tasks;
+
+	/** A mutex, used to protect the access to the input file. */
+	pthread_mutex_t		input_mutex;
+
+	/** Hash of tasks to filter on. */
+	struct kshark_hash_id	*show_task_filter;
+
+	/** Hash of tasks to not display. */
+	struct kshark_hash_id	*hide_task_filter;
+
+	/** Hash of events to filter on. */
+	struct kshark_hash_id	*show_event_filter;
+
+	/** Hash of events to not display. */
+	struct kshark_hash_id	*hide_event_filter;
+
+	/** Hash of CPUs to filter on. */
+	struct kshark_hash_id	*show_cpu_filter;
+
+	/** Hash of CPUs to not display. */
+	struct kshark_hash_id	*hide_cpu_filter;
+
+	/** The type of the data. */
+	enum kshark_data_format		format;
+
+	/**
+	 * The interface of methods used to operate over the data from a given
+	 * stream.
+	 */
+	struct kshark_data_stream_interface	interface;
+};
+
 /** Size of the task's hash table. */
 #define KS_TASK_HASH_SHIFT 16
 #define KS_TASK_HASH_SIZE (1 << KS_TASK_HASH_SHIFT)
Signature · 1 line
2.25.1
#8 lore raw

PATCH v2 5/20 kernel-shark: Add stream_id to kshark_entry

In reply to Yordan Karadzhov (VMware)

kshark_entry contains all information from one trace record, needed in order to visualize the time-series of trace records. Here we reorganize data fields to kshark_entry in order to make room for the unique identifier of the Data stream this entry belongs to.

---
 src/libkshark.c |  2 +-
 src/libkshark.h | 12 +++++++++---
 2 files changed, 10 insertions(+), 4 deletions(-)
src/libkshark.c +1 -1
index 52aacd30..92e2450c 100644
@@ -489,7 +489,7 @@ static inline void unset_event_filter_flag(struct kshark_context *kshark_ctx,
 	e->visible &= ~event_mask;
 }
 
-static void set_all_visible(uint16_t *v) {
+static void set_all_visible(uint8_t *v) {
 	/*  Keep the original value of the PLUGIN_UNTOUCHED bit flag. */
 	*v |= 0xFF & ~KS_PLUGIN_UNTOUCHED_MASK;
 }
src/libkshark.h +9 -3
index 1165c512..fe0ba7f2 100644
@@ -1,7 +1,7 @@
 /* SPDX-License-Identifier: LGPL-2.1 */
 
 /*
- * Copyright (C) 2017 VMware Inc, Yordan Karadzhov <[email protected]>
+ * Copyright (C) 2017 VMware Inc, Yordan Karadzhov (VMware) <[email protected]>
  */
 
  /**
@@ -49,7 +49,10 @@ struct kshark_entry {
 	 * kshark_filter_masks to check the level of visibility/invisibility
 	 * of the entry.
 	 */
-	uint16_t	visible;
+	uint8_t		visible;
+
+	/** Data stream identifier. */
+	int8_t		stream_id;
 
 	/** The CPU core of the record. */
 	int16_t		cpu;
@@ -57,7 +60,7 @@ struct kshark_entry {
 	/** The PID of the task the record was generated. */
 	int32_t		pid;
 
-	/** Unique Id ot the trace event type. */
+	/** Unique Id of the trace event type. */
 	int32_t		event_id;
 
 	/** The offset into the trace file, used to find the record. */
@@ -319,6 +322,9 @@ struct kshark_data_stream {
 	struct kshark_data_stream_interface	interface;
 };
 
+/** Hard-coded maximum number of data stream. */
+#define KS_MAX_NUM_STREAMS	127
+
 /** Size of the task's hash table. */
 #define KS_TASK_HASH_SHIFT 16
 #define KS_TASK_HASH_SIZE (1 << KS_TASK_HASH_SHIFT)
Signature · 1 line
2.25.1
#9 lore raw

Re: [PATCH v2 05/20] kernel-shark: Add stream_id to kshark_entry

Quoted 38 lines · On Mon, 12 Oct 2020 16:35:08 +0300 "Yordan Karadzhov (VMware)"…

kshark_entry contains all information from one trace record, needed in order to visualize the time-series of trace records. Here we reorganize data fields to kshark_entry in order to make room for the unique identifier of the Data stream this entry belongs to.

---
 src/libkshark.c |  2 +-
 src/libkshark.h | 12 +++++++++---
 2 files changed, 10 insertions(+), 4 deletions(-)
src/libkshark.c +1 -1
index 52aacd30..92e2450c 100644
@@ -489,7 +489,7 @@ static inline void unset_event_filter_flag(struct kshark_context *kshark_ctx,
 	e->visible &= ~event_mask;
 }
 
-static void set_all_visible(uint16_t *v) {
+static void set_all_visible(uint8_t *v) {
 	/*  Keep the original value of the PLUGIN_UNTOUCHED bit flag. */
 	*v |= 0xFF & ~KS_PLUGIN_UNTOUCHED_MASK;
 }
src/libkshark.h +1 -1
index 1165c512..fe0ba7f2 100644
@@ -1,7 +1,7 @@
 /* SPDX-License-Identifier: LGPL-2.1 */
 
 /*
- * Copyright (C) 2017 VMware Inc, Yordan Karadzhov <[email protected]>
+ * Copyright (C) 2017 VMware Inc, Yordan Karadzhov (VMware) <[email protected]>
  */
 
  /**

src/libkshark.h struct kshark_entry { · line 52

Quoted 5 lines
src/libkshark.h +1 -1
@@ -49,7 +49,10 @@ struct kshark_entry {
 	 * kshark_filter_masks to check the level of visibility/invisibility
 	 * of the entry.
 	 */
-	uint16_t	visible;
+	uint8_t		visible;

Why did the visible variable turn from 16bits to 8 bits?

That should be stated in the change log.

Quoted 16 lines
src/libkshark.h +4 -1
+
+	/** Data stream identifier. */
+	int8_t		stream_id;
 
 	/** The CPU core of the record. */
 	int16_t		cpu;
@@ -57,7 +60,7 @@ struct kshark_entry {
 	/** The PID of the task the record was generated. */
 	int32_t		pid;
 
-	/** Unique Id ot the trace event type. */
+	/** Unique Id of the trace event type. */
 	int32_t		event_id;
 
 	/** The offset into the trace file, used to find the record. */

src/libkshark.h struct kshark_data_stream { · line 326

Quoted 5 lines
src/libkshark.h +2 -0
@@ -319,6 +322,9 @@ struct kshark_data_stream {
 	struct kshark_data_stream_interface	interface;
 };
 
+/** Hard-coded maximum number of data stream. */
+#define KS_MAX_NUM_STREAMS	127

Will this become a variable?

Let's say we have a really powerful machine with a lot of memory. And we have a data file with 500 CPUs traced, and we want to look at all of them? Will this be a hard limit to change in the future without recompiling.

I've been hit by restrictions like this in code in the past, and know that things like this can stay in the code for a long time, to a point where they are no longer reasonable. I just want to make sure that we don't get stuck with that. I've learned to look at all hard limits with extra care, knowing that they are always a pain point in the future.

-- Steve

Quoted 4 lines
src/libkshark.h +1 -0
+
 /** Size of the task's hash table. */
 #define KS_TASK_HASH_SHIFT 16
 #define KS_TASK_HASH_SIZE (1 << KS_TASK_HASH_SHIFT)
#10 lore raw

Re: [PATCH v2 05/20] kernel-shark: Add stream_id to kshark_entry

Quoted 49 lines · Steven Rostedt
Quoted 38 lines · On Mon, 12 Oct 2020 16:35:08 +0300 "Yordan Karadzhov (VMware)"…

kshark_entry contains all information from one trace record, needed in order to visualize the time-series of trace records. Here we reorganize data fields to kshark_entry in order to make room for the unique identifier of the Data stream this entry belongs to.

---
  src/libkshark.c |  2 +-
  src/libkshark.h | 12 +++++++++---
  2 files changed, 10 insertions(+), 4 deletions(-)
src/libkshark.c +1 -1
index 52aacd30..92e2450c 100644
@@ -489,7 +489,7 @@ static inline void unset_event_filter_flag(struct kshark_context *kshark_ctx,
  	e->visible &= ~event_mask;
  }
 
-static void set_all_visible(uint16_t *v) {
+static void set_all_visible(uint8_t *v) {
  	/*  Keep the original value of the PLUGIN_UNTOUCHED bit flag. */
  	*v |= 0xFF & ~KS_PLUGIN_UNTOUCHED_MASK;
  }
src/libkshark.h +1 -1
index 1165c512..fe0ba7f2 100644
@@ -1,7 +1,7 @@
  /* SPDX-License-Identifier: LGPL-2.1 */
 
  /*
- * Copyright (C) 2017 VMware Inc, Yordan Karadzhov <[email protected]>
+ * Copyright (C) 2017 VMware Inc, Yordan Karadzhov (VMware) <[email protected]>
   */
 
   /**

src/libkshark.h struct kshark_entry { · line 52

Quoted 5 lines
src/libkshark.h +1 -1
@@ -49,7 +49,10 @@ struct kshark_entry {
  	 * kshark_filter_masks to check the level of visibility/invisibility
  	 * of the entry.
  	 */
-	uint16_t	visible;
+	uint8_t		visible;

Why did the visible variable turn from 16bits to 8 bits?

Quoted 2 lines

That should be stated in the change log.

After further discussing this on a call with Steven we decided to keep the "visible" field 16 bit and to define the "stream_id" to be a 16 bit integer as well. Those 16 bit will be taken from the event_id. This is OK because the Id of the trace event is coded with only 16 bit in the kernel anyway.

Quoted 28 lines
Quoted 16 lines
src/libkshark.h +4 -1
+
+	/** Data stream identifier. */
+	int8_t		stream_id;
 
  	/** The CPU core of the record. */
  	int16_t		cpu;
@@ -57,7 +60,7 @@ struct kshark_entry {
  	/** The PID of the task the record was generated. */
  	int32_t		pid;
 
-	/** Unique Id ot the trace event type. */
+	/** Unique Id of the trace event type. */
  	int32_t		event_id;
 
  	/** The offset into the trace file, used to find the record. */

src/libkshark.h struct kshark_data_stream { · line 326

Quoted 5 lines
src/libkshark.h +2 -0
@@ -319,6 +322,9 @@ struct kshark_data_stream {
  	struct kshark_data_stream_interface	interface;
  };
 
+/** Hard-coded maximum number of data stream. */
+#define KS_MAX_NUM_STREAMS	127

Will this become a variable?

Let's say we have a really powerful machine with a lot of memory. And we have a data file with 500 CPUs traced, and we want to look at all of them? Will this be a hard limit to change in the future without recompiling.

Quoted 5 lines

I've been hit by restrictions like this in code in the past, and know that things like this can stay in the code for a long time, to a point where they are no longer reasonable. I just want to make sure that we don't get stuck with that. I've learned to look at all hard limits with extra care, knowing that they are always a pain point in the future.

Now when we have 16 bit available for coding the stream Id, we no longer need to enforce such a small limit for the total number of streams loaded.

Thanks Steven!

Yordan

Quoted 9 lines

-- Steve

Quoted 4 lines
src/libkshark.h +1 -0
+
  /** Size of the task's hash table. */
  #define KS_TASK_HASH_SHIFT 16
  #define KS_TASK_HASH_SIZE (1 << KS_TASK_HASH_SHIFT)
#11 lore raw

PATCH v2 6/20 kernel-shark: Rename static methods in libkshark

In reply to Yordan Karadzhov (VMware)

"kshark" prefix is needed only for the public methods of the API. We are changing the names of those static methods because we need to use those names when integrating the data streams into the API.i

---
 src/libkshark.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
src/libkshark.c +7 -7
index 92e2450c..6a1258dc 100644
@@ -1097,8 +1097,8 @@ size_t kshark_load_data_matrix(struct kshark_context *kshark_ctx,
 	return -ENOMEM;
 }
 
-static const char *kshark_get_latency(struct tep_handle *pe,
-				      struct tep_record *record)
+static const char *get_latency(struct tep_handle *pe,
+			       struct tep_record *record)
 {
 	if (!record)
 		return NULL;
@@ -1108,7 +1108,7 @@ static const char *kshark_get_latency(struct tep_handle *pe,
 	return seq.buffer;
 }
 
-static const char *kshark_get_info(struct tep_handle *pe,
+static const char *get_info(struct tep_handle *pe,
 				   struct tep_record *record,
 				   struct tep_event *event)
 {
@@ -1233,7 +1233,7 @@ const char *kshark_get_latency_easy(struct kshark_entry *entry)
 	pthread_mutex_lock(&kshark_ctx->input_mutex);
 
 	data = tracecmd_read_at(kshark_ctx->handle, entry->offset, NULL);
-	lat = kshark_get_latency(kshark_ctx->pevent, data);
+	lat = get_latency(kshark_ctx->pevent, data);
 	free_record(data);
 
 	pthread_mutex_unlock(&kshark_ctx->input_mutex);
@@ -1376,7 +1376,7 @@ const char *kshark_get_info_easy(struct kshark_entry *entry)
 	event_id = tep_data_type(kshark_ctx->pevent, data);
 	event = tep_find_event(kshark_ctx->pevent, event_id);
 	if (event)
-		info = kshark_get_info(kshark_ctx->pevent, data, event);
+		info = get_info(kshark_ctx->pevent, data, event);
 
 	free_record(data);
 
@@ -1471,7 +1471,7 @@ char* kshark_dump_entry(const struct kshark_entry *entry)
 		event = tep_find_event(kshark_ctx->pevent, entry->event_id);
 
 		event_name = event? event->name : "[UNKNOWN EVENT]";
-		lat = kshark_get_latency(kshark_ctx->pevent, data);
+		lat = get_latency(kshark_ctx->pevent, data);
 
 		size = asprintf(&temp_str, "%" PRIu64 "; %s-%i; CPU %i; %s;",
 				entry->ts,
@@ -1480,7 +1480,7 @@ char* kshark_dump_entry(const struct kshark_entry *entry)
 				entry->cpu,
 				lat);
 
-		info = kshark_get_info(kshark_ctx->pevent, data, event);
+		info = get_info(kshark_ctx->pevent, data, event);
 
 		if (size > 0) {
 			size = asprintf(&entry_str, "%s %s; %s; 0x%x",
Signature · 1 line
2.25.1
#12 lore raw

PATCH v2 7/20 kernel-shark: Add basic methods for Data streams

In reply to Yordan Karadzhov (VMware)

Here we introduce the basic mechanisms for using data streams. For the moment these are just stand alone definitions and the integration with the API is yet to be introduced in the following patches.

---
 src/libkshark.c | 197 ++++++++++++++++++++++++++++++++++++++++++++++++
 src/libkshark.h | 171 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 368 insertions(+)
src/libkshark.c +197 -0
index 6a1258dc..ffeb66a7 100644
@@ -166,6 +166,153 @@ bool kshark_open(struct kshark_context *kshark_ctx, const char *file)
 	return true;
 }
 
+static void kshark_stream_free(struct kshark_data_stream *stream)
+{
+	if (!stream)
+		return;
+
+	kshark_hash_id_free(stream->show_task_filter);
+	kshark_hash_id_free(stream->hide_task_filter);
+
+	kshark_hash_id_free(stream->show_event_filter);
+	kshark_hash_id_free(stream->hide_event_filter);
+
+	kshark_hash_id_free(stream->show_cpu_filter);
+	kshark_hash_id_free(stream->hide_cpu_filter);
+
+	kshark_hash_id_free(stream->tasks);
+
+	free(stream->file);
+	free(stream->name);
+	free(stream);
+}
+
+static struct kshark_data_stream *kshark_stream_alloc()
+{
+	struct kshark_data_stream *stream;
+
+	stream = calloc(1, sizeof(*stream));
+	if (!stream)
+		goto fail;
+
+	stream->show_task_filter = kshark_hash_id_alloc(KS_FILTER_HASH_NBITS);
+	stream->hide_task_filter = kshark_hash_id_alloc(KS_FILTER_HASH_NBITS);
+
+	stream->show_event_filter = kshark_hash_id_alloc(KS_FILTER_HASH_NBITS);
+	stream->hide_event_filter = kshark_hash_id_alloc(KS_FILTER_HASH_NBITS);
+
+	stream->show_cpu_filter = kshark_hash_id_alloc(KS_FILTER_HASH_NBITS);
+	stream->hide_cpu_filter = kshark_hash_id_alloc(KS_FILTER_HASH_NBITS);
+
+	stream->tasks = kshark_hash_id_alloc(KS_TASK_HASH_NBITS);
+
+	if (!stream->show_task_filter ||
+	    !stream->hide_task_filter ||
+	    !stream->show_event_filter ||
+	    !stream->hide_event_filter ||
+	    !stream->tasks) {
+		    goto fail;
+	}
+
+	stream->format = KS_INVALIDE_DATA;
+
+	return stream;
+
+ fail:
+	fprintf(stderr, "Failed to allocate memory for data stream.\n");
+	if (stream)
+		kshark_stream_free(stream);
+
+	return NULL;
+}
+
+/**
+ * @brief Add new Data stream.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ *
+ * @returns Zero on success or a negative errno code on failure.
+ */
+int kshark_add_stream(struct kshark_context *kshark_ctx)
+{
+	struct kshark_data_stream *stream;
+
+	if (kshark_ctx->n_streams == KS_MAX_NUM_STREAMS)
+		return -EMFILE;
+
+	stream = kshark_stream_alloc();
+	stream->stream_id = kshark_ctx->n_streams;
+
+	if (pthread_mutex_init(&stream->input_mutex, NULL) != 0) {
+		kshark_stream_free(stream);
+		return -EAGAIN;
+	}
+
+	kshark_ctx->stream[kshark_ctx->n_streams++] = stream;
+
+	return stream->stream_id;
+}
+
+/**
+ * @brief Get the Data stream object having given Id.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ * @param sd: Data stream identifier.
+ *
+ * @returns Pointer to a Data stream object if the sream exists. Otherwise
+ *	    NULL.
+ */
+struct kshark_data_stream *
+kshark_get_data_stream(struct kshark_context *kshark_ctx, int sd)
+{
+	if (sd >= 0 && sd < KS_MAX_NUM_STREAMS)
+		return kshark_ctx->stream[sd];
+
+	return NULL;
+}
+
+/**
+ * @brief Get the Data stream object corresponding to a given entry
+ *
+ * @param entry: Input location for the KernelShark entry.
+ *
+ * @returns Pointer to a Data stream object on success. Otherwise NULL.
+ */
+struct kshark_data_stream *
+kshark_get_stream_from_entry(const struct kshark_entry *entry)
+{
+	struct kshark_context *kshark_ctx = NULL;
+
+	if (!kshark_instance(&kshark_ctx))
+		return NULL;
+
+	return kshark_get_data_stream(kshark_ctx, entry->stream_id);
+}
+
+/**
+ * @brief Get an array containing the Ids of all opened Trace data streams.
+ * 	  The User is responsible for freeing the array.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ */
+int *kshark_all_streams(struct kshark_context *kshark_ctx)
+{
+	int *ids, i, count = 0;
+
+	ids = malloc(kshark_ctx->n_streams * (sizeof(*ids)));
+	if (!ids) {
+		fprintf(stderr,
+			"Failed to allocate memory for stream array.\n");
+		return NULL;
+	}
+
+	for (i = 0; i < KS_MAX_NUM_STREAMS; ++i)
+		if (kshark_ctx->stream[i])
+			ids[count++] = i;
+
+	return ids;
+}
+
 /**
  * @brief Close the trace data file and free the trace data handle.
  *
@@ -252,6 +399,56 @@ void kshark_free(struct kshark_context *kshark_ctx)
 	free(kshark_ctx);
 }
 
+/**
+ * @brief Get the name of the command/task from its Process Id.
+ *
+ * @param sd: Data stream identifier.
+ * @param pid: Process Id of the command/task.
+ */
+char *kshark_comm_from_pid(int sd, int pid)
+{
+	struct kshark_context *kshark_ctx = NULL;
+	struct kshark_data_stream *stream;
+	struct kshark_entry e;
+
+	if (!kshark_instance(&kshark_ctx))
+		return NULL;
+
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
+		return NULL;
+
+	e.visible = KS_PLUGIN_UNTOUCHED_MASK;
+	e.pid = pid;
+
+	return stream->interface.get_task(stream, &e);
+}
+
+/**
+ * @brief Get the name of the event from its Id.
+ *
+ * @param sd: Data stream identifier.
+ * @param event_id: The unique Id of the event type.
+ */
+char *kshark_event_from_id(int sd, int event_id)
+{
+	struct kshark_context *kshark_ctx = NULL;
+	struct kshark_data_stream *stream;
+	struct kshark_entry e;
+
+	if (!kshark_instance(&kshark_ctx))
+		return NULL;
+
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
+		return NULL;
+
+	e.visible = KS_PLUGIN_UNTOUCHED_MASK;
+	e.event_id = event_id;
+
+	return stream->interface.get_event_name(stream, &e);
+}
+
 static struct kshark_task_list *
 kshark_find_task(struct kshark_context *kshark_ctx, uint32_t key, int pid)
 {
src/libkshark.h +171 -0

195 lines not on this page · raw message

Signature · 1 line
2.25.1
#13 lore raw

Re: [PATCH v2 07/20] kernel-shark: Add basic methods for Data streams

Quoted 23 lines · On Mon, 12 Oct 2020 16:35:10 +0300 "Yordan Karadzhov (VMware)"…
src/libkshark.c +23 -0
+static struct kshark_data_stream *kshark_stream_alloc()
+{
+	struct kshark_data_stream *stream;
+
+	stream = calloc(1, sizeof(*stream));
+	if (!stream)
+		goto fail;
+
+	stream->show_task_filter = kshark_hash_id_alloc(KS_FILTER_HASH_NBITS);
+	stream->hide_task_filter = kshark_hash_id_alloc(KS_FILTER_HASH_NBITS);
+
+	stream->show_event_filter = kshark_hash_id_alloc(KS_FILTER_HASH_NBITS);
+	stream->hide_event_filter = kshark_hash_id_alloc(KS_FILTER_HASH_NBITS);
+
+	stream->show_cpu_filter = kshark_hash_id_alloc(KS_FILTER_HASH_NBITS);
+	stream->hide_cpu_filter = kshark_hash_id_alloc(KS_FILTER_HASH_NBITS);
+
+	stream->tasks = kshark_hash_id_alloc(KS_TASK_HASH_NBITS);
+
+	if (!stream->show_task_filter ||
+	    !stream->hide_task_filter ||
+	    !stream->show_event_filter ||
+	    !stream->hide_event_filter ||

src/libkshark.c bool kshark_open(struct kshark_context *kshark_ctx, const char *file) · line 222

Quoted 10 lines
src/libkshark.c +10 -0
+	    !stream->tasks) {
+		    goto fail;
+	}
+
+	stream->format = KS_INVALIDE_DATA;
+
+	return stream;
+
+ fail:
+	fprintf(stderr, "Failed to allocate memory for data stream.\n");

I don't think we need the print. Not if this is a library routine. The calloc failure should set the errno that the caller should be able to figure out what happened.

Quoted 11 lines
src/libkshark.c +11 -0
+	if (stream)
+		kshark_stream_free(stream);
+
+	return NULL;
+}
+
+/**
+ * @brief Add new Data stream.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ *

src/libkshark.c bool kshark_open(struct kshark_context *kshark_ctx, const char *file) · line 243

Quoted 10 lines
src/libkshark.c +10 -0
+ * @returns Zero on success or a negative errno code on failure.
+ */
+int kshark_add_stream(struct kshark_context *kshark_ctx)
+{
+	struct kshark_data_stream *stream;
+
+	if (kshark_ctx->n_streams == KS_MAX_NUM_STREAMS)
+		return -EMFILE;
+
+	stream = kshark_stream_alloc();

Need to check for success.

Quoted 49 lines
src/libkshark.c +49 -0
+	stream->stream_id = kshark_ctx->n_streams;
+
+	if (pthread_mutex_init(&stream->input_mutex, NULL) != 0) {
+		kshark_stream_free(stream);
+		return -EAGAIN;
+	}
+
+	kshark_ctx->stream[kshark_ctx->n_streams++] = stream;
+
+	return stream->stream_id;
+}
+
+/**
+ * @brief Get the Data stream object having given Id.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ * @param sd: Data stream identifier.
+ *
+ * @returns Pointer to a Data stream object if the sream exists. Otherwise
+ *	    NULL.
+ */
+struct kshark_data_stream *
+kshark_get_data_stream(struct kshark_context *kshark_ctx, int sd)
+{
+	if (sd >= 0 && sd < KS_MAX_NUM_STREAMS)
+		return kshark_ctx->stream[sd];
+
+	return NULL;
+}
+
+/**
+ * @brief Get the Data stream object corresponding to a given entry
+ *
+ * @param entry: Input location for the KernelShark entry.
+ *
+ * @returns Pointer to a Data stream object on success. Otherwise NULL.
+ */
+struct kshark_data_stream *
+kshark_get_stream_from_entry(const struct kshark_entry *entry)
+{
+	struct kshark_context *kshark_ctx = NULL;
+
+	if (!kshark_instance(&kshark_ctx))
+		return NULL;
+
+	return kshark_get_data_stream(kshark_ctx, entry->stream_id);
+}
+
+/**

src/libkshark.c bool kshark_open(struct kshark_context *kshark_ctx, const char *file) · line 302

Quoted 10 lines
src/libkshark.c +10 -0
+ * @brief Get an array containing the Ids of all opened Trace data streams.
+ * 	  The User is responsible for freeing the array.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ */
+int *kshark_all_streams(struct kshark_context *kshark_ctx)
+{
+	int *ids, i, count = 0;
+
+	ids = malloc(kshark_ctx->n_streams * (sizeof(*ids)));

I think calloc is more appropriate for the above.

	ids = calloc(kshark_ctx->n_streams, sizeof(*ids));

src/libkshark.c bool kshark_open(struct kshark_context *kshark_ctx, const char *file) · line 305

Quoted 3 lines
src/libkshark.c +3 -0
+	if (!ids) {
+		fprintf(stderr,
+			"Failed to allocate memory for stream array.\n");

Probably don't need the print.

src/libkshark.c bool kshark_open(struct kshark_context *kshark_ctx, const char *file) · line 311

Quoted 6 lines
src/libkshark.c +6 -0
+		return NULL;
+	}
+
+	for (i = 0; i < KS_MAX_NUM_STREAMS; ++i)
+		if (kshark_ctx->stream[i])
+			ids[count++] = i;

Definitely need the calloc, as malloc doesn't initialize the array to zero. Thus some ids[] will not be initialized.

Quoted 8 lines
src/libkshark.c +4 -0
+
+	return ids;
+}
+
 /**
  * @brief Close the trace data file and free the trace data handle.
  *

src/libkshark.c void kshark_free(struct kshark_context *kshark_ctx) · line 408

Quoted 10 lines
src/libkshark.c +7 -0
@@ -252,6 +399,56 @@ void kshark_free(struct kshark_context *kshark_ctx)
 	free(kshark_ctx);
 }
 
+/**
+ * @brief Get the name of the command/task from its Process Id.
+ *
+ * @param sd: Data stream identifier.
+ * @param pid: Process Id of the command/task.
+ */
+char *kshark_comm_from_pid(int sd, int pid)

I wonder if we should abstract this further, and call it "kshark_name_from_id()", as comm and pid are specific to processes, and we may have a stream that will represent something other than processes.

Quoted 202 lines
src/libkshark.c +43 -0
+{
+	struct kshark_context *kshark_ctx = NULL;
+	struct kshark_data_stream *stream;
+	struct kshark_entry e;
+
+	if (!kshark_instance(&kshark_ctx))
+		return NULL;
+
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
+		return NULL;
+
+	e.visible = KS_PLUGIN_UNTOUCHED_MASK;
+	e.pid = pid;
+
+	return stream->interface.get_task(stream, &e);
+}
+
+/**
+ * @brief Get the name of the event from its Id.
+ *
+ * @param sd: Data stream identifier.
+ * @param event_id: The unique Id of the event type.
+ */
+char *kshark_event_from_id(int sd, int event_id)
+{
+	struct kshark_context *kshark_ctx = NULL;
+	struct kshark_data_stream *stream;
+	struct kshark_entry e;
+
+	if (!kshark_instance(&kshark_ctx))
+		return NULL;
+
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
+		return NULL;
+
+	e.visible = KS_PLUGIN_UNTOUCHED_MASK;
+	e.event_id = event_id;
+
+	return stream->interface.get_event_name(stream, &e);
+}
+
 static struct kshark_task_list *
 kshark_find_task(struct kshark_context *kshark_ctx, uint32_t key, int pid)
 {
src/libkshark.h +127 -0
index fe0ba7f2..e299d067 100644
@@ -340,6 +340,12 @@ struct kshark_task_list {
 
 /** Structure representing a kshark session. */
 struct kshark_context {
+	/** Array of data stream descriptors. */
+	struct kshark_data_stream	**stream;
+
+	/** The number of data streams. */
+	int				n_streams;
+
 	/** Input handle for the trace data file. */
 	struct tracecmd_input	*handle;
 
@@ -397,6 +403,16 @@ bool kshark_instance(struct kshark_context **kshark_ctx);
 
 bool kshark_open(struct kshark_context *kshark_ctx, const char *file);
 
+int kshark_add_stream(struct kshark_context *kshark_ctx);
+
+struct kshark_data_stream *
+kshark_get_data_stream(struct kshark_context *kshark_ctx, int sd);
+
+struct kshark_data_stream *
+kshark_get_stream_from_entry(const struct kshark_entry *entry);
+
+int *kshark_all_streams(struct kshark_context *kshark_ctx);
+
 ssize_t kshark_load_data_entries(struct kshark_context *kshark_ctx,
 				 struct kshark_entry ***data_rows);
 
@@ -416,6 +432,10 @@ void kshark_close(struct kshark_context *kshark_ctx);
 
 void kshark_free(struct kshark_context *kshark_ctx);
 
+char *kshark_comm_from_pid(int sd, int pid);
+
+char *kshark_event_from_id(int sd, int event_id);
+
 int kshark_get_pid_easy(struct kshark_entry *entry);
 
 const char *kshark_get_task_easy(struct kshark_entry *entry);
@@ -432,6 +452,157 @@ void kshark_convert_nano(uint64_t time, uint64_t *sec, uint64_t *usec);
 
 char* kshark_dump_entry(const struct kshark_entry *entry);
 
+static inline int kshark_get_pid(const struct kshark_entry *entry)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return -1;
+
+	return stream->interface.get_pid(stream, entry);
+}
+
+static inline int kshark_get_event_id(const struct kshark_entry *entry)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return -1;
+
+	return stream->interface.get_event_id(stream, entry);
+}
+
+static inline int *kshark_get_all_event_ids(struct kshark_data_stream *stream)
+{
+	return stream->interface.get_all_event_ids(stream);
+}
+
+static inline char *kshark_get_event_name(const struct kshark_entry *entry)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return NULL;
+
+	return stream->interface.get_event_name(stream, entry);
+}
+
+static inline char *kshark_get_task(const struct kshark_entry *entry)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return NULL;
+
+	return stream->interface.get_task(stream, entry);
+}
+
+static inline char *kshark_get_latency(const struct kshark_entry *entry)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return NULL;
+
+	return stream->interface.get_latency(stream, entry);
+}
+
+static inline char *kshark_get_info(const struct kshark_entry *entry)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return NULL;
+
+	return stream->interface.get_info(stream, entry);
+}
+
+static inline int kshark_read_event_field(const struct kshark_entry *entry,
+					  const char* field, int64_t *val)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return -1;
+
+	return stream->interface.read_event_field_int64(stream, entry,
+							field, val);
+}
+
+/**
+ * @brief Load the content of the trace data file asociated with a given
+ *	  Data stream identifie into an array of kshark_entries.
+ *	  If one or more filters are set, the "visible" fields of each entry
+ *	  is updated according to the criteria provided by the filters. The
+ *	  field "filter_mask" of the session's context is used to control the
+ *	  level of visibility/invisibility of the filtered entries.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ * @param sd: Data stream identifier.
+ * @param data_rows: Output location for the trace data. The user is
+ *		     responsible for freeing the elements of the outputted
+ *		     array.
+ *
+ * @returns The size of the outputted data in the case of success, or a
+ *	    negative error code on failure.
+ */
+static inline ssize_t kshark_load_entries(struct kshark_context *kshark_ctx,
+					  int sd,
+					  struct kshark_entry ***data_rows)
+{
+	struct kshark_data_stream *stream;
+

src/libkshark.h void kshark_convert_nano(uint64_t time, uint64_t *sec, uint64_t *usec); · line 571

Quoted 10 lines
src/libkshark.h +10 -0
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
+		return -EBADF;
+
+	return stream->interface.load_entries(stream, kshark_ctx, data_rows);
+}
+
+/**
+ * @brief Load the content of the trace data file asociated with a given
+ *	  Data stream identifie into a data matrix. The user is responsible

identifie?

src/libkshark.h void kshark_convert_nano(uint64_t time, uint64_t *sec, uint64_t *usec); · line 583

Quoted 12 lines
src/libkshark.h +12 -0
+ *	  for freeing the outputted data.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ * @param sd: Data stream identifier.
+ * @param cpu_array: Output location for the CPU column (array) of the matrix.
+ * @param event_array: Output location for the Event Id column (array) of the
+ *		       matrix.
+ * @param _array: Output location for the  column (array) of the matrix.
+ * @param offset_array: Output location for the offset column (array) of the
+ *			matrix.
+ * @param ts_array: Output location for the time stamp column (array) of the
+ *		    matrix.

Hmm, how is this matrix composed? How do each realate, via the ts_array?

-- Steve

Quoted 25 lines
src/libkshark.h +22 -0
+ */
+static inline ssize_t kshark_load_matrix(struct kshark_context *kshark_ctx,
+					 int sd,
+					 int16_t **cpu_array,
+					 int32_t **pid_array,
+					 int32_t **event_array,
+					 int64_t **offset_array,
+					 int64_t **ts_array)
+{
+	struct kshark_data_stream *stream;
+
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
+		return -EBADF;
+
+	return stream->interface.load_matrix(stream, kshark_ctx, cpu_array,
+								 pid_array,
+								 event_array,
+								 offset_array,
+								 ts_array);
+}
+
 /**
  * Custom entry info function type. To be user for dumping info for custom
  * KernelShark entryes.
#14 lore raw

Re: [PATCH v2 07/20] kernel-shark: Add basic methods for Data streams

Quoted 16 lines · Steven Rostedt

src/libkshark.c bool kshark_open(struct kshark_context *kshark_ctx, const char *file) · line 222

Quoted 9 lines · On Mon, 12 Oct 2020 16:35:10 +0300 "Yordan Karadzhov (VMware)"…
src/libkshark.c +9 -0
+static struct kshark_data_stream *kshark_stream_alloc()
+{
+	struct kshark_data_stream *stream;
+
+
+	return stream;
+
+ fail:
+	fprintf(stderr, "Failed to allocate memory for data stream.\n");

I don't think we need the print. Not if this is a library routine. The calloc failure should set the errno that the caller should be able to figure out what happened.

Agree. Will be fixed in v3.

Quoted 15 lines
Quoted 15 lines
src/libkshark.c +15 -0
+	if (stream)
+		kshark_stream_free(stream);
+
+	return NULL;
+}
+
+/**
+ * @brief Add new Data stream.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ *
+ * @returns Zero on success or a negative errno code on failure.
+ */
+int kshark_add_stream(struct kshark_context *kshark_ctx)
+{
Quoted 8 lines

src/libkshark.c bool kshark_open(struct kshark_context *kshark_ctx, const char *file) · line 243

Quoted 6 lines
src/libkshark.c +6 -0
+	struct kshark_data_stream *stream;
+
+	if (kshark_ctx->n_streams == KS_MAX_NUM_STREAMS)
+		return -EMFILE;
+
+	stream = kshark_stream_alloc();

Need to check for success.

OK. Fix in v3.

Quoted 21 lines
Quoted 8 lines
src/libkshark.c +6 -0
+	stream->stream_id = kshark_ctx->n_streams;
+
+	if (pthread_mutex_init(&stream->input_mutex, NULL) != 0) {
+		kshark_stream_free(stream);
+		return -EAGAIN;
 
+/**

src/libkshark.c bool kshark_open(struct kshark_context *kshark_ctx, const char *file) · line 302

Quoted 10 lines
src/libkshark.c +10 -0
+ * @brief Get an array containing the Ids of all opened Trace data streams.
+ * 	  The User is responsible for freeing the array.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ */
+int *kshark_all_streams(struct kshark_context *kshark_ctx)
+{
+	int *ids, i, count = 0;
+
+	ids = malloc(kshark_ctx->n_streams * (sizeof(*ids)));

I think calloc is more appropriate for the above.

Quoted 2 lines
	ids = calloc(kshark_ctx->n_streams, sizeof(*ids));

Fix in v3.

Quoted 5 lines

src/libkshark.c bool kshark_open(struct kshark_context *kshark_ctx, const char *file) · line 305

Quoted 3 lines
src/libkshark.c +3 -0
+	if (!ids) {
+		fprintf(stderr,
+			"Failed to allocate memory for stream array.\n");

Probably don't need the print.

Fix in v3.

Quoted 22 lines

src/libkshark.c bool kshark_open(struct kshark_context *kshark_ctx, const char *file) · line 311

Quoted 7 lines
src/libkshark.c +6 -0
+		return NULL;
+	}
+
+	for (i = 0; i < KS_MAX_NUM_STREAMS; ++i)
+		if (kshark_ctx->stream[i])
+			ids[count++] = i;

Definitely need the calloc, as malloc doesn't initialize the array to zero. Thus some ids[] will not be initialized.

Quoted 11 lines
src/libkshark.c +4 -0
+
+	return ids;
+}
+
  /**
   * @brief Close the trace data file and free the trace data handle.
   *
@@ -252,6 +399,56 @@ void kshark_free(struct kshark_context *kshark_ctx)
  	free(kshark_ctx);
  }
 
Quoted 12 lines

src/libkshark.c void kshark_free(struct kshark_context *kshark_ctx) · line 408

Quoted 7 lines
src/libkshark.c +7 -0
+/**
+ * @brief Get the name of the command/task from its Process Id.
+ *
+ * @param sd: Data stream identifier.
+ * @param pid: Process Id of the command/task.
+ */
+char *kshark_comm_from_pid(int sd, int pid)

I wonder if we should abstract this further, and call it "kshark_name_from_id()", as comm and pid are specific to processes, and we may have a stream that will represent something other than processes.

This is just a helper function that wraps the corresponding method of the interface. In the future we can add a similar helper/wrapper function with different name if we have streams that contain no processes.

However, you are actually making a very good point. It may be even better if we abstract the interface itself, instead of trying to have more abstract method names. We can keep the existing interface of methods unchanged, but in the definition of "struct kshark_data_stream" make the interface void*

struct kshark_data_stream {
	/** Data stream identifier. */
	uint8_t			stream_id;

....

	/** List of Plugin's Draw handlers. */
	struct kshark_draw_handler		*draw_handlers;

	/**
	 * Abstract interface of methods used to operate over the data
	 * from a given stream. An implementation must be provided.
	 */
	void	*interface;
};

and then the wrapping functions may look like this

char *kshark_comm_from_pid(int sd, int pid)

{
	struct kshark_data_stream_interface *interface;
	struct kshark_context *kshark_ctx = NULL;
	struct kshark_data_stream *stream;
	struct kshark_entry e;

	if (!kshark_instance(&kshark_ctx))
		return NULL;

	stream = kshark_get_data_stream(kshark_ctx, sd);
	if (!stream)
		return NULL;

	interface = stream->interface;
	if (!interface || !interface->get_task)
		return NULL;

	e.visible = KS_PLUGIN_UNTOUCHED_MASK;
	e.pid = pid;

	return interface->get_task(stream, &e);
}

And in the future we can add more interface implementations and more helper functions.

What do you think?

Thanks!
Yordan

Quoted 259 lines
Quoted 202 lines
src/libkshark.c +43 -0
+{
+	struct kshark_context *kshark_ctx = NULL;
+	struct kshark_data_stream *stream;
+	struct kshark_entry e;
+
+	if (!kshark_instance(&kshark_ctx))
+		return NULL;
+
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
+		return NULL;
+
+	e.visible = KS_PLUGIN_UNTOUCHED_MASK;
+	e.pid = pid;
+
+	return stream->interface.get_task(stream, &e);
+}
+
+/**
+ * @brief Get the name of the event from its Id.
+ *
+ * @param sd: Data stream identifier.
+ * @param event_id: The unique Id of the event type.
+ */
+char *kshark_event_from_id(int sd, int event_id)
+{
+	struct kshark_context *kshark_ctx = NULL;
+	struct kshark_data_stream *stream;
+	struct kshark_entry e;
+
+	if (!kshark_instance(&kshark_ctx))
+		return NULL;
+
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
+		return NULL;
+
+	e.visible = KS_PLUGIN_UNTOUCHED_MASK;
+	e.event_id = event_id;
+
+	return stream->interface.get_event_name(stream, &e);
+}
+
  static struct kshark_task_list *
  kshark_find_task(struct kshark_context *kshark_ctx, uint32_t key, int pid)
  {
src/libkshark.h +127 -0
index fe0ba7f2..e299d067 100644
@@ -340,6 +340,12 @@ struct kshark_task_list {
 
  /** Structure representing a kshark session. */
  struct kshark_context {
+	/** Array of data stream descriptors. */
+	struct kshark_data_stream	**stream;
+
+	/** The number of data streams. */
+	int				n_streams;
+
  	/** Input handle for the trace data file. */
  	struct tracecmd_input	*handle;
 
@@ -397,6 +403,16 @@ bool kshark_instance(struct kshark_context **kshark_ctx);
 
  bool kshark_open(struct kshark_context *kshark_ctx, const char *file);
 
+int kshark_add_stream(struct kshark_context *kshark_ctx);
+
+struct kshark_data_stream *
+kshark_get_data_stream(struct kshark_context *kshark_ctx, int sd);
+
+struct kshark_data_stream *
+kshark_get_stream_from_entry(const struct kshark_entry *entry);
+
+int *kshark_all_streams(struct kshark_context *kshark_ctx);
+
  ssize_t kshark_load_data_entries(struct kshark_context *kshark_ctx,
  				 struct kshark_entry ***data_rows);
 
@@ -416,6 +432,10 @@ void kshark_close(struct kshark_context *kshark_ctx);
 
  void kshark_free(struct kshark_context *kshark_ctx);
 
+char *kshark_comm_from_pid(int sd, int pid);
+
+char *kshark_event_from_id(int sd, int event_id);
+
  int kshark_get_pid_easy(struct kshark_entry *entry);
 
  const char *kshark_get_task_easy(struct kshark_entry *entry);
@@ -432,6 +452,157 @@ void kshark_convert_nano(uint64_t time, uint64_t *sec, uint64_t *usec);
 
  char* kshark_dump_entry(const struct kshark_entry *entry);
 
+static inline int kshark_get_pid(const struct kshark_entry *entry)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return -1;
+
+	return stream->interface.get_pid(stream, entry);
+}
+
+static inline int kshark_get_event_id(const struct kshark_entry *entry)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return -1;
+
+	return stream->interface.get_event_id(stream, entry);
+}
+
+static inline int *kshark_get_all_event_ids(struct kshark_data_stream *stream)
+{
+	return stream->interface.get_all_event_ids(stream);
+}
+
+static inline char *kshark_get_event_name(const struct kshark_entry *entry)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return NULL;
+
+	return stream->interface.get_event_name(stream, entry);
+}
+
+static inline char *kshark_get_task(const struct kshark_entry *entry)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return NULL;
+
+	return stream->interface.get_task(stream, entry);
+}
+
+static inline char *kshark_get_latency(const struct kshark_entry *entry)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return NULL;
+
+	return stream->interface.get_latency(stream, entry);
+}
+
+static inline char *kshark_get_info(const struct kshark_entry *entry)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return NULL;
+
+	return stream->interface.get_info(stream, entry);
+}
+
+static inline int kshark_read_event_field(const struct kshark_entry *entry,
+					  const char* field, int64_t *val)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return -1;
+
+	return stream->interface.read_event_field_int64(stream, entry,
+							field, val);
+}
+
+/**
+ * @brief Load the content of the trace data file asociated with a given
+ *	  Data stream identifie into an array of kshark_entries.
+ *	  If one or more filters are set, the "visible" fields of each entry
+ *	  is updated according to the criteria provided by the filters. The
+ *	  field "filter_mask" of the session's context is used to control the
+ *	  level of visibility/invisibility of the filtered entries.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ * @param sd: Data stream identifier.
+ * @param data_rows: Output location for the trace data. The user is
+ *		     responsible for freeing the elements of the outputted
+ *		     array.
+ *
+ * @returns The size of the outputted data in the case of success, or a
+ *	    negative error code on failure.
+ */
+static inline ssize_t kshark_load_entries(struct kshark_context *kshark_ctx,
+					  int sd,
+					  struct kshark_entry ***data_rows)
+{
+	struct kshark_data_stream *stream;
+

src/libkshark.h void kshark_convert_nano(uint64_t time, uint64_t *sec, uint64_t *usec); · line 571

Quoted 10 lines
src/libkshark.h +10 -0
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
+		return -EBADF;
+
+	return stream->interface.load_entries(stream, kshark_ctx, data_rows);
+}
+
+/**
+ * @brief Load the content of the trace data file asociated with a given
+ *	  Data stream identifie into a data matrix. The user is responsible

identifie?

src/libkshark.h void kshark_convert_nano(uint64_t time, uint64_t *sec, uint64_t *usec); · line 583

Quoted 12 lines
src/libkshark.h +12 -0
+ *	  for freeing the outputted data.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ * @param sd: Data stream identifier.
+ * @param cpu_array: Output location for the CPU column (array) of the matrix.
+ * @param event_array: Output location for the Event Id column (array) of the
+ *		       matrix.
+ * @param _array: Output location for the  column (array) of the matrix.
+ * @param offset_array: Output location for the offset column (array) of the
+ *			matrix.
+ * @param ts_array: Output location for the time stamp column (array) of the
+ *		    matrix.

Hmm, how is this matrix composed? How do each realate, via the ts_array?

-- Steve

Quoted 25 lines
src/libkshark.h +22 -0
+ */
+static inline ssize_t kshark_load_matrix(struct kshark_context *kshark_ctx,
+					 int sd,
+					 int16_t **cpu_array,
+					 int32_t **pid_array,
+					 int32_t **event_array,
+					 int64_t **offset_array,
+					 int64_t **ts_array)
+{
+	struct kshark_data_stream *stream;
+
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
+		return -EBADF;
+
+	return stream->interface.load_matrix(stream, kshark_ctx, cpu_array,
+								 pid_array,
+								 event_array,
+								 offset_array,
+								 ts_array);
+}
+
  /**
   * Custom entry info function type. To be user for dumping info for custom
   * KernelShark entryes.
#15 lore raw

Re: [PATCH v2 07/20] kernel-shark: Add basic methods for Data streams

Quoted 65 lines · On Thu, 29 Oct 2020 12:10:36 +0200 "Yordan Karadzhov (VMware)"…
Quoted 12 lines

src/libkshark.c void kshark_free(struct kshark_context *kshark_ctx) · line 408

Quoted 7 lines
src/libkshark.c +7 -0
+/**
+ * @brief Get the name of the command/task from its Process Id.
+ *
+ * @param sd: Data stream identifier.
+ * @param pid: Process Id of the command/task.
+ */
+char *kshark_comm_from_pid(int sd, int pid)  

I wonder if we should abstract this further, and call it "kshark_name_from_id()", as comm and pid are specific to processes, and we may have a stream that will represent something other than processes.

This is just a helper function that wraps the corresponding method of the interface. In the future we can add a similar helper/wrapper function with different name if we have streams that contain no processes.

However, you are actually making a very good point. It may be even better if we abstract the interface itself, instead of trying to have more abstract method names. We can keep the existing interface of methods unchanged, but in the definition of "struct kshark_data_stream" make the interface void*

struct kshark_data_stream {
	/** Data stream identifier. */
	uint8_t			stream_id;

....

	/** List of Plugin's Draw handlers. */
	struct kshark_draw_handler		*draw_handlers;

	/**
	 * Abstract interface of methods used to operate over the data
	 * from a given stream. An implementation must be provided.
	 */
	void	*interface;
};

and then the wrapping functions may look like this

char *kshark_comm_from_pid(int sd, int pid)

{
	struct kshark_data_stream_interface *interface;
	struct kshark_context *kshark_ctx = NULL;
	struct kshark_data_stream *stream;
	struct kshark_entry e;

	if (!kshark_instance(&kshark_ctx))
		return NULL;

	stream = kshark_get_data_stream(kshark_ctx, sd);
	if (!stream)
		return NULL;

	interface = stream->interface;
	if (!interface || !interface->get_task)
		return NULL;

	e.visible = KS_PLUGIN_UNTOUCHED_MASK;
	e.pid = pid;

	return interface->get_task(stream, &e);
}
Quoted 4 lines

And in the future we can add more interface implementations and more helper functions.

What do you think?

Do we need to make the interface "void *" and not just a non defined type "struct kshark_data_stream_interface *"?

Just have:

struct kshark_data_stream_interface;

And then reference it without defining it, and have the streams define it. If you need this to have inheritance and a bit of polymorphism you can do that too :-) That is, if you want this interface to have something common among all streams.

struct kshark_data_stream_interface {
	int		type;
	int		common_data;
};

then local to the file that implements it:

struct data_stream_interface {
	struct kshark_data_stream_interface	kinterface;
	int					unique_data;
};
{
	struct data_stream_interface	*interface;


	interface = (struct data_stream_interface *)stream->interface;

	if (interface->kinterface.type != my_type)
		return (or error);

	unique_data = interface->unique_data;

}

This is even how the trace events work in the kernel. For example:

struct trace_entry {
	unsigned short		type;
	unsigned char		flags;
	unsigned char		preempt_count;
	int			pid;
};
struct kprobe_trace_entry_head {
	struct trace_entry	ent;
	unsigned long		ip;
};
struct kretprobe_trace_entry_head {
	struct trace_entry	ent;
	unsigned long		func;
	unsigned long		ret_ip;
};

-- Steve

#16 lore raw

Re: [PATCH v2 07/20] kernel-shark: Add basic methods for Data streams

Quoted 77 lines · Steven Rostedt
Quoted 65 lines · On Thu, 29 Oct 2020 12:10:36 +0200 "Yordan Karadzhov (VMware)"…
Quoted 12 lines

src/libkshark.c void kshark_free(struct kshark_context *kshark_ctx) · line 408

Quoted 7 lines
src/libkshark.c +7 -0
+/**
+ * @brief Get the name of the command/task from its Process Id.
+ *
+ * @param sd: Data stream identifier.
+ * @param pid: Process Id of the command/task.
+ */
+char *kshark_comm_from_pid(int sd, int pid)

I wonder if we should abstract this further, and call it "kshark_name_from_id()", as comm and pid are specific to processes, and we may have a stream that will represent something other than processes.

This is just a helper function that wraps the corresponding method of the interface. In the future we can add a similar helper/wrapper function with different name if we have streams that contain no processes.

However, you are actually making a very good point. It may be even better if we abstract the interface itself, instead of trying to have more abstract method names. We can keep the existing interface of methods unchanged, but in the definition of "struct kshark_data_stream" make the interface void*

struct kshark_data_stream {
	/** Data stream identifier. */
	uint8_t			stream_id;

....

	/** List of Plugin's Draw handlers. */
	struct kshark_draw_handler		*draw_handlers;

	/**
	 * Abstract interface of methods used to operate over the data
	 * from a given stream. An implementation must be provided.
	 */
	void	*interface;
};

and then the wrapping functions may look like this

char *kshark_comm_from_pid(int sd, int pid)

{
	struct kshark_data_stream_interface *interface;
	struct kshark_context *kshark_ctx = NULL;
	struct kshark_data_stream *stream;
	struct kshark_entry e;

	if (!kshark_instance(&kshark_ctx))
		return NULL;

	stream = kshark_get_data_stream(kshark_ctx, sd);
	if (!stream)
		return NULL;

	interface = stream->interface;
	if (!interface || !interface->get_task)
		return NULL;

	e.visible = KS_PLUGIN_UNTOUCHED_MASK;
	e.pid = pid;

	return interface->get_task(stream, &e);
}
Quoted 4 lines

And in the future we can add more interface implementations and more helper functions.

What do you think?

Do we need to make the interface "void *" and not just a non defined type "struct kshark_data_stream_interface *"?

Quoted 9 lines

Just have:

struct kshark_data_stream_interface;

And then reference it without defining it, and have the streams define it. If you need this to have inheritance and a bit of polymorphism you can do that too :-) That is, if you want this interface to have something common among all streams.

Maybe I don't understand your idea very well, but I think what you suggest has very different behavior. What I want is the implementation of the interface to stay in the same header file (libkshark.h). In the future we can add more interfaces but this will be again in the same header (libkshark.h).

The readout plugins will include libkshark.h and will have to chose one of the available interfaces and implement its methods like this:

static int kshark_tep_stream_init(struct kshark_data_stream *stream,
struct tracecmd_input *input)

{
	struct kshark_data_stream_interface *interface;

	stream->interface = interface = calloc(1, sizeof(*interface));
	if (!interface)
		return -ENOMEM;

....

	interface->get_pid = tepdata_get_pid;
	interface->get_task = tepdata_get_task;
	interface->get_event_id = tepdata_get_event_id;

....

Note that the plugins will include libkshark.h but libkshark will never include headers from plugins.

Does it make any sense, or maybe I just don't understand your suggestion?

Thanks a lot!
Yordan

Quoted 49 lines
struct kshark_data_stream_interface {
	int		type;
	int		common_data;
};

then local to the file that implements it:

struct data_stream_interface {
	struct kshark_data_stream_interface	kinterface;
	int					unique_data;
};
{
	struct data_stream_interface	*interface;


	interface = (struct data_stream_interface *)stream->interface;

	if (interface->kinterface.type != my_type)
		return (or error);

	unique_data = interface->unique_data;

}

This is even how the trace events work in the kernel. For example:

struct trace_entry {
	unsigned short		type;
	unsigned char		flags;
	unsigned char		preempt_count;
	int			pid;
};
struct kprobe_trace_entry_head {
	struct trace_entry	ent;
	unsigned long		ip;
};
struct kretprobe_trace_entry_head {
	struct trace_entry	ent;
	unsigned long		func;
	unsigned long		ret_ip;
};

-- Steve

#17 lore raw

Re: [PATCH v2 07/20] kernel-shark: Add basic methods for Data streams

Quoted 5 lines · On Thu, 29 Oct 2020 16:49:03 +0200 "Yordan Karadzhov (VMware)"…

Maybe I don't understand your idea very well, but I think what you suggest has very different behavior. What I want is the implementation of the interface to stay in the same header file (libkshark.h). In the future we can add more interfaces but this will be again in the same header (libkshark.h).

Maybe I got confused ;-)

Is there going to be different interface structures? Why the "void *" and not just supply a "struct kshark_data_stream *"?

That way at least you have some kind of type checking when tasks move things around. I try to avoid using "void *" because it can easily be the source of unwanted bugs, due to the lack of type checking.

-- Steve

#18 lore raw

Re: [PATCH v2 07/20] kernel-shark: Add basic methods for Data streams

Quoted 9 lines · Steven Rostedt
Quoted 5 lines · On Thu, 29 Oct 2020 16:49:03 +0200 "Yordan Karadzhov (VMware)"…

Maybe I don't understand your idea very well, but I think what you suggest has very different behavior. What I want is the implementation of the interface to stay in the same header file (libkshark.h). In the future we can add more interfaces but this will be again in the same header (libkshark.h).

Quoted 4 lines

Maybe I got confused ;-)

Is there going to be different interface structures? Why the "void *" and not just supply a "struct kshark_data_stream *"?

Hi Steven,

Yes, my idea is that in the future we may decide to change something in the interface, or to have a second completely different interface, while we may still need to keep the interface version that we have now for backward compatibility.

Quoted 4 lines

That way at least you have some kind of type checking when tasks move things around. I try to avoid using "void *" because it can easily be the source of unwanted bugs, due to the lack of type checking.

What if we define the interface to start with an integer identifier?

/** Data interface identifier. */

typedef enum kshark_data_interface_id {
	/** An interface with unknown type. */
	KS_INVALIDE_INTERFACE,

	/** Generic interface suitable Ftrace data. */
	KS_GENERIC_DATA_INTERFACE,
} kshark_data_interface_id;

/**
* Structure representing the interface of methods used to operate over
* the data from a given stream.
*/

struct kshark_generic_stream_interface {
	/** Interface version identifier. */
	int			type; /* MUST BE FIRST ENTRY */

	/** Method used to retrieve the Process Id of the entry. */
	stream_get_int_func	get_pid;

	/** Method used to retrieve the Event Id of the entry. */
	stream_get_int_func	get_event_id;

....

and it can be used like this:

char *kshark_get_aux_field(const struct kshark_entry *entry)

{
	struct kshark_generic_stream_interface *interface;
	struct kshark_data_stream *stream =
		kshark_get_stream_from_entry(entry);

	....

	interface = stream->interface;
	if (interface->type == KS_GENERIC_DATA_INTERFACE &&
	    interface->aux_field)
		return interface->aux_field(stream, entry);

	return NULL;
}

What do you think?

Thanks a lot!
Yordan

Quoted 3 lines

-- Steve

#19 lore raw

Re: [PATCH v2 07/20] kernel-shark: Add basic methods for Data streams

Quoted 45 lines · On Tue, 3 Nov 2020 15:38:33 +0200 "Yordan Karadzhov (VMware)"…

What if we define the interface to start with an integer identifier?

/** Data interface identifier. */

typedef enum kshark_data_interface_id {
	/** An interface with unknown type. */
	KS_INVALIDE_INTERFACE,

	/** Generic interface suitable Ftrace data. */
	KS_GENERIC_DATA_INTERFACE,
} kshark_data_interface_id;

/**
* Structure representing the interface of methods used to operate over
* the data from a given stream.
*/

struct kshark_generic_stream_interface {
	/** Interface version identifier. */
	int			type; /* MUST BE FIRST ENTRY */

	/** Method used to retrieve the Process Id of the entry. */
	stream_get_int_func	get_pid;

	/** Method used to retrieve the Event Id of the entry. */
	stream_get_int_func	get_event_id;

....

and it can be used like this:

char *kshark_get_aux_field(const struct kshark_entry *entry)

{
	struct kshark_generic_stream_interface *interface;
	struct kshark_data_stream *stream =
		kshark_get_stream_from_entry(entry);

	....

	interface = stream->interface;
	if (interface->type == KS_GENERIC_DATA_INTERFACE &&
	    interface->aux_field)
		return interface->aux_field(stream, entry);

	return NULL;
}
Quoted 1 line

What do you think?

This was actually what I was getting at ;-)

That is a common practice in the Linux kernel.

-- Steve

#20 lore raw

Re: [PATCH v2 07/20] kernel-shark: Add basic methods for Data streams

Quoted 48 lines · Steven Rostedt
Quoted 45 lines · On Tue, 3 Nov 2020 15:38:33 +0200 "Yordan Karadzhov (VMware)"…

What if we define the interface to start with an integer identifier?

/** Data interface identifier. */

typedef enum kshark_data_interface_id {
	/** An interface with unknown type. */
	KS_INVALIDE_INTERFACE,

	/** Generic interface suitable Ftrace data. */
	KS_GENERIC_DATA_INTERFACE,
} kshark_data_interface_id;

/**
* Structure representing the interface of methods used to operate over
* the data from a given stream.
*/

struct kshark_generic_stream_interface {
	/** Interface version identifier. */
	int			type; /* MUST BE FIRST ENTRY */

	/** Method used to retrieve the Process Id of the entry. */
	stream_get_int_func	get_pid;

	/** Method used to retrieve the Event Id of the entry. */
	stream_get_int_func	get_event_id;

....

and it can be used like this:

char *kshark_get_aux_field(const struct kshark_entry *entry)

{
	struct kshark_generic_stream_interface *interface;
	struct kshark_data_stream *stream =
		kshark_get_stream_from_entry(entry);

	....

	interface = stream->interface;
	if (interface->type == KS_GENERIC_DATA_INTERFACE &&
	    interface->aux_field)
		return interface->aux_field(stream, entry);

	return NULL;
}
Quoted 6 lines
Quoted 1 line

What do you think?

This was actually what I was getting at ;-)

That is a common practice in the Linux kernel.

OK, thanks a lot!
Will be implemented in v3.

cheers,
Yordan

Quoted 3 lines

-- Steve

#21 lore raw

PATCH v2 8/20 kernel-shark: Housekeeping before implementing stream interface

In reply to Yordan Karadzhov (VMware)

kshark_load data_matrix() is a method that is used only in trace-cruncher prototype. The version of this function that is currently part of the API is anyway not compatible with the latest version of trace-cruncher because of the mismatch of the argument types, so we can directly remove the existing implementation. An equivalent functionality will be provided as part of the implantation of the Data stream interface in the following patch. Here we just make kshark_data_matrix_alloc() and unset_event_filter_flag() public, because the methods will be used in the stream interface implementation.

---
 src/libkshark.c | 121 ++++++++----------------------------------------
 src/libkshark.h |  21 +++++++++
 2 files changed, 40 insertions(+), 102 deletions(-)
src/libkshark.c +19 -102
index ffeb66a7..56721717 100644
@@ -671,21 +671,6 @@ bool kshark_filter_is_set(struct kshark_context *kshark_ctx)
 -              kshark_this_filter_is_set(kshark_ctx->hide_event_filter);
 }
 
-static inline void unset_event_filter_flag(struct kshark_context *kshark_ctx,
-					   struct kshark_entry *e)
-{
-	/*
-	 * All entries, filtered-out by the event filters, will be treated
-	 * differently, when visualized. Because of this, ignore the value
-	 * of the GRAPH_VIEW flag provided by the user via
-	 * kshark_ctx->filter_mask. The value of the EVENT_VIEW flag in
-	 * kshark_ctx->filter_mask will be used instead.
-	 */
-	int event_mask = kshark_ctx->filter_mask & ~KS_GRAPH_VIEW_FILTER_MASK;
-
-	e->visible &= ~event_mask;
-}
-
 static void set_all_visible(uint8_t *v) {
 	/*  Keep the original value of the PLUGIN_UNTOUCHED bit flag. */
 	*v |= 0xFF & ~KS_PLUGIN_UNTOUCHED_MASK;
@@ -1161,11 +1146,25 @@ static inline void free_ptr(void *ptr)
 		free(*(void **)ptr);
 }
 
-static bool data_matrix_alloc(size_t n_rows, uint64_t **offset_array,
-					     uint16_t **cpu_array,
-					     uint64_t **ts_array,
-					     uint16_t **pid_array,
-					     int **event_array)
+/**
+ * @brief Allocate data arrays (matrix columns) to be used to load the tracing
+ *	  data into a data matrix form.
+ *
+ * @param n_rows: Number matrix rows to be allocated. Must be equal to the
+ *	 	  number of trace records.
+ * @param cpu_array: Output location for the CPU Id column.
+ * @param pid_array: Output location for the PID column.
+ * @param event_array: Output location for the Event Id column.
+ * @param offset_array: Output location for the record offset column.
+ * @param ts_array: Output location for the timestamp column.
+ *
+ * @returns True on success. Else false.
+ */
+bool kshark_data_matrix_alloc(size_t n_rows, int16_t **cpu_array,
+					     int32_t **pid_array,
+					     int32_t **event_array,
+					     int64_t **offset_array,
+					     int64_t **ts_array)
 {
 	if (offset_array) {
 		*offset_array = calloc(n_rows, sizeof(**offset_array));
@@ -1212,88 +1211,6 @@ static bool data_matrix_alloc(size_t n_rows, uint64_t **offset_array,
 	return false;
 }
 
-/**
- * @brief Load the content of the trace data file into a table / matrix made
- *	  of columns / arrays of data. The user is responsible for freeing the
- *	  elements of the outputted array
- *
- * @param kshark_ctx: Input location for the session context pointer.
- * @param offset_array: Output location for the array of record offsets.
- * @param cpu_array: Output location for the array of CPU Ids.
- * @param ts_array: Output location for the array of timestamps.
- * @param pid_array: Output location for the array of Process Ids.
- * @param event_array: Output location for the array of Event Ids.
- *
- * @returns The size of the outputted arrays in the case of success, or a
- *	    negative error code on failure.
- */
-size_t kshark_load_data_matrix(struct kshark_context *kshark_ctx,
-			       uint64_t **offset_array,
-			       uint16_t **cpu_array,
-			       uint64_t **ts_array,
-			       uint16_t **pid_array,
-			       int **event_array)
-{
-	enum rec_type type = REC_ENTRY;
-	struct rec_list **rec_list;
-	ssize_t count, total = 0;
-	bool status;
-	int n_cpus;
-
-	total = get_records(kshark_ctx, &rec_list, type);
-	if (total < 0)
-		goto fail;
-
-	n_cpus = tracecmd_cpus(kshark_ctx->handle);
-
-	status = data_matrix_alloc(total, offset_array,
-					  cpu_array,
-					  ts_array,
-					  pid_array,
-					  event_array);
-	if (!status)
-		goto fail_free;
-
-	for (count = 0; count < total; count++) {
-		int next_cpu;
-
-		next_cpu = pick_next_cpu(rec_list, n_cpus, type);
-		if (next_cpu >= 0) {
-			struct rec_list *rec = rec_list[next_cpu];
-			struct kshark_entry *e = &rec->entry;
-
-			if (offset_array)
-				(*offset_array)[count] = e->offset;
-
-			if (cpu_array)
-				(*cpu_array)[count] = e->cpu;
-
-			if (ts_array)
-				(*ts_array)[count] = e->ts;
-
-			if (pid_array)
-				(*pid_array)[count] = e->pid;
-
-			if (event_array)
-				(*event_array)[count] = e->event_id;
-
-			rec_list[next_cpu] = rec_list[next_cpu]->next;
-			free(rec);
-		}
-	}
-
-	/* There should be no entries left in rec_list. */
-	free_rec_list(rec_list, n_cpus, type);
-	return total;
-
- fail_free:
-	free_rec_list(rec_list, n_cpus, type);
-
- fail:
-	fprintf(stderr, "Failed to allocate memory during data loading.\n");
-	return -ENOMEM;
-}
-
 static const char *get_latency(struct tep_handle *pe,
 			       struct tep_record *record)
 {
src/libkshark.h +21 -0
index e299d067..f26fad7c 100644
@@ -689,6 +689,21 @@ bool kshark_this_filter_is_set(struct tracecmd_filter_id *filter);
 
 bool kshark_filter_is_set(struct kshark_context *kshark_ctx);
 
+static inline void unset_event_filter_flag(struct kshark_context *kshark_ctx,
+					   struct kshark_entry *e)
+{
+	/*
+	 * All entries, filtered-out by the event filters, will be treated
+	 * differently, when visualized. Because of this, ignore the value
+	 * of the GRAPH_VIEW flag provided by the user via
+	 * stream->filter_mask. The value of the EVENT_VIEW flag in
+	 * stream->filter_mask will be used instead.
+	 */
+	int event_mask = kshark_ctx->filter_mask & ~KS_GRAPH_VIEW_FILTER_MASK;
+
+	e->visible &= ~event_mask;
+}
+
 void kshark_filter_entries(struct kshark_context *kshark_ctx,
 			   struct kshark_entry **data,
 			   size_t n_entries);
@@ -1067,6 +1082,12 @@ struct kshark_config_doc *kshark_open_config_file(const char *file_name,
 
 struct kshark_config_doc *kshark_json_to_conf(struct json_object *jobj);
 
+bool kshark_data_matrix_alloc(size_t n_rows, int16_t **cpu_array,
+					     int32_t **pid_array,
+					     int32_t **event_array,
+					     int64_t **offset_array,
+					     int64_t **ts_array);
+
 #ifdef __cplusplus
 }
 #endif
Signature · 1 line
2.25.1
#22 lore raw

PATCH v2 9/20 kernel-shark: Add stream interface for trace-cmd data

In reply to Yordan Karadzhov (VMware)

Here we provide an implementation of the Data stream interface that will process the trace-cmd (Ftrace) data. This can be considered the most essential change in the transformation of the C API towards version 2. However, for the moment we only have stand alone definitions that are not made functional yet. The actual integration with the API will be introduced in the following patches.

---
 src/CMakeLists.txt      |    1 +
 src/libkshark-tepdata.c | 1391 +++++++++++++++++++++++++++++++++++++++
 src/libkshark-tepdata.h |   51 ++
 3 files changed, 1443 insertions(+)
 create mode 100644 src/libkshark-tepdata.c
 create mode 100644 src/libkshark-tepdata.h
src/CMakeLists.txt +1 -0
index 39c4dcf0..2bdab1a5 100644
@@ -5,6 +5,7 @@ add_library(kshark SHARED libkshark.c
                           libkshark-hash.c
 #                           libkshark-model.c
                           libkshark-plugin.c
+                          libkshark-tepdata.c
 #                           libkshark-configio.c
                           libkshark-collection.c)
 
src/libkshark-tepdata.c new file +1391 -0

1,391 lines not on this page · raw message

src/libkshark-tepdata.h new file +51 -0

51 lines not on this page · raw message

Signature · 1 line
2.25.1
#23 lore raw

Re: [PATCH v2 09/20] kernel-shark: Add stream interface for trace-cmd data

Quoted 216 lines · On Mon, 12 Oct 2020 16:35:12 +0300 "Yordan Karadzhov (VMware)"…

Here we provide an implementation of the Data stream interface that will process the trace-cmd (Ftrace) data. This can be considered the most essential change in the transformation of the C API towards version 2. However, for the moment we only have stand alone definitions that are not made functional yet. The actual integration with the API will be introduced in the following patches.

---

src/libkshark-tepdata.c new file +205 -0
+
+static ssize_t get_records(struct kshark_context *kshark_ctx,
+			   struct kshark_data_stream *stream,
+			   struct rec_list ***rec_list,
+			   enum rec_type type)
+{
+	struct tep_event_filter *adv_filter = NULL;
+	struct rec_list **temp_next;
+	struct rec_list **cpu_list;
+	struct rec_list *temp_rec;
+	struct tep_record *rec;
+	ssize_t count, total = 0;
+	int pid, next_pid, cpu;
+
+	cpu_list = calloc(stream->n_cpus, sizeof(*cpu_list));
+	if (!cpu_list)
+		return -ENOMEM;
+
+	if (type == REC_ENTRY)
+		adv_filter = get_adv_filter(stream);
+
+	for (cpu = 0; cpu < stream->n_cpus; ++cpu) {
+		count = 0;
+		cpu_list[cpu] = NULL;
+		temp_next = &cpu_list[cpu];
+
+		rec = tracecmd_read_cpu_first(kshark_get_tep_input(stream), cpu);
+		while (rec) {
+			*temp_next = temp_rec = calloc(1, sizeof(*temp_rec));
+			if (!temp_rec)
+				goto fail;
+
+			temp_rec->next = NULL;
+
+			switch (type) {
+			case REC_RECORD:
+				temp_rec->rec = rec;
+				pid = tep_data_pid(kshark_get_tep(stream), rec);
+				break;
+			case REC_ENTRY: {
+				struct kshark_entry *entry;
+
+				if (rec->missed_events) {
+					/*
+					 * Insert a custom "missed_events" entry just
+					 * befor this record.
+					 */
+					entry = &temp_rec->entry;
+					missed_events_action(stream, rec, entry);
+
+					entry->stream_id = stream->stream_id;
+
+					temp_next = &temp_rec->next;
+					++count;
+
+					/* Now allocate a new rec_list node and comtinue. */
+					*temp_next = temp_rec = calloc(1, sizeof(*temp_rec));
+				}
+
+				entry = &temp_rec->entry;
+				set_entry_values(stream, rec, entry);
+
+				if(entry->event_id == get_sched_switch_id(stream)) {
+					next_pid = get_next_pid(stream, rec);
+					if (next_pid >= 0)
+						register_command(stream, rec, next_pid);
+				}
+
+				entry->stream_id = stream->stream_id;
+
+				pid = entry->pid;
+
+				/* Apply advanced event filtering. */
+				if (adv_filter && adv_filter->filters &&
+				    tep_filter_match(adv_filter, rec) != FILTER_MATCH)
+					unset_event_filter_flag(kshark_ctx, entry);
+
+				free_record(rec);
+				break;
+			} /* REC_ENTRY */
+			}
+
+			kshark_hash_id_add(stream->tasks, pid);
+
+			temp_next = &temp_rec->next;
+
+			++count;
+			rec = tracecmd_read_data(kshark_get_tep_input(stream), cpu);
+		}
+
+		total += count;
+	}
+
+	*rec_list = cpu_list;
+	return total;
+
+ fail:
+	free_rec_list(cpu_list, stream->n_cpus, type);
+	return -ENOMEM;
+}
+
+static int pick_next_cpu(struct rec_list **rec_list, int n_cpus,
+			 enum rec_type type)
+{
+	uint64_t ts = 0;
+	uint64_t rec_ts;
+	int next_cpu = -1;
+	int cpu;
+
+	for (cpu = 0; cpu < n_cpus; ++cpu) {
+		if (!rec_list[cpu])
+			continue;
+
+		switch (type) {
+		case REC_RECORD:
+			rec_ts = rec_list[cpu]->rec->ts;
+			break;
+		case REC_ENTRY:
+			rec_ts = rec_list[cpu]->entry.ts;
+			break;
+		}
+		if (!ts || rec_ts < ts) {
+			ts = rec_ts;
+			next_cpu = cpu;
+		}
+	}
+
+	return next_cpu;
+}
+
+/**
+ * @brief Load the content of the trace data file asociated with a given
+ *	  Data stream into an array of kshark_entries. This function
+ *	  provides an abstraction of the entries from the raw data
+ *	  that is read, however the "latency" and the "info" fields can be
+ *	  accessed only via the offset into the file. This makes the access
+ *	  to these two fields much slower.
+ *	  If one or more filters are set, the "visible" fields of each entry
+ *	  is updated according to the criteria provided by the filters. The
+ *	  field "filter_mask" of the session's context is used to control the
+ *	  level of visibility/invisibility of the filtered entries.
+ *
+ * @param stream: Input location for the FTRACE data stream pointer.
+ * @param kshark_ctx: Input location for context pointer.
+ * @param data_rows: Output location for the trace data. The user is
+ *		     responsible for freeing the elements of the outputted
+ *		     array.
+ *
+ * @returns The size of the outputted data in the case of success, or a
+ *	    negative error code on failure.
+ */
+ssize_t tepdata_load_entries(struct kshark_data_stream *stream,
+				struct kshark_context *kshark_ctx,
+				struct kshark_entry ***data_rows)
+{
+	enum rec_type type = REC_ENTRY;
+	struct kshark_entry **rows;
+	struct rec_list **rec_list;
+	ssize_t count, total = 0;
+
+	total = get_records(kshark_ctx, stream, &rec_list, type);
+	if (total < 0)
+		goto fail;
+
+	rows = calloc(total, sizeof(struct kshark_entry *));
+	if (!rows)
+		goto fail_free;
+
+	for (count = 0; count < total; count++) {
+		int next_cpu;
+
+		next_cpu = pick_next_cpu(rec_list, stream->n_cpus, type);
+
+		if (next_cpu >= 0) {
+			rows[count] = &rec_list[next_cpu]->entry;
+			rec_list[next_cpu] = rec_list[next_cpu]->next;
+		}
+	}
+
+	/* There should be no entries left in rec_list. */
+	free_rec_list(rec_list, stream->n_cpus, type);
+	*data_rows = rows;
+
+	return total;
+
+ fail_free:
+	free_rec_list(rec_list, stream->n_cpus, type);
+
+ fail:
+	fprintf(stderr, "Failed to allocate memory during data loading.\n");
+	return -ENOMEM;
+}
+
+static ssize_t tepdata_load_matrix(struct kshark_data_stream *stream,
+				   struct kshark_context *kshark_ctx,
+				   int16_t **cpu_array,
+				   int32_t **pid_array,
+				   int32_t **event_array,
+				   int64_t **offset_array,
+				   int64_t **ts_array)
+{
+	enum rec_type type = REC_ENTRY;
+	struct rec_list **rec_list;
+	ssize_t count, total = 0;
+	bool status;

src/libkshark-tepdata.c line 463

Quoted 10 lines
src/libkshark-tepdata.c new file +10 -0
+
+	total = get_records(kshark_ctx, stream, &rec_list, type);
+	if (total < 0)
+		goto fail;
+
+	status = kshark_data_matrix_alloc(total, cpu_array,
+						 pid_array,
+						 event_array,
+						 offset_array,
+						 ts_array);

So if we have a billion events, we'll be allocating more than 5 arrays of a billion items?

Will this scale well?

Quoted 344 lines
src/libkshark-tepdata.c new file +344 -0
+	if (!status)
+		goto fail_free;
+
+	for (count = 0; count < total; count++) {
+		int next_cpu;
+
+		next_cpu = pick_next_cpu(rec_list, stream->n_cpus, type);
+		if (next_cpu >= 0) {
+			struct rec_list *rec = rec_list[next_cpu];
+			struct kshark_entry *e = &rec->entry;
+
+			if (offset_array)
+				(*offset_array)[count] = e->offset;
+
+			if (cpu_array)
+				(*cpu_array)[count] = e->cpu;
+
+			if (ts_array)
+				(*ts_array)[count] = e->ts;
+
+			if (pid_array)
+				(*pid_array)[count] = e->pid;
+
+			if (event_array)
+				(*event_array)[count] = e->event_id;
+
+			rec_list[next_cpu] = rec_list[next_cpu]->next;
+			free(rec);
+		}
+	}
+
+	/* There should be no entries left in rec_list. */
+	free_rec_list(rec_list, stream->n_cpus, type);
+	return total;
+
+ fail_free:
+	free_rec_list(rec_list, stream->n_cpus, type);
+
+ fail:
+	fprintf(stderr, "Failed to allocate memory during data loading.\n");
+	return -ENOMEM;
+}
+
+/**
+ * @brief Load the content of the trace data file into an array of
+ *	  tep_records. Use this function only if you need fast access
+ *	  to all fields of the record.
+ *
+ * @param kshark_ctx: Input location for the session context pointer.
+ * @param sd: Data stream identifier.
+ * @param data_rows: Output location for the trace data. Use free_record()
+ *	 	     to free the elements of the outputted array.
+ *
+ * @returns The size of the outputted data in the case of success, or a
+ *	    negative error code on failure.
+ */
+ssize_t kshark_load_tep_records(struct kshark_context *kshark_ctx, int sd,
+				struct tep_record ***data_rows)
+{
+	struct kshark_data_stream *stream;
+	enum rec_type type = REC_RECORD;
+	struct rec_list **rec_list;
+	struct rec_list *temp_rec;
+	struct tep_record **rows;
+	struct tep_record *rec;
+	ssize_t count, total = 0;
+
+	if (*data_rows)
+		free(*data_rows);
+
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
+		return -EBADF;
+
+	total = get_records(kshark_ctx, stream, &rec_list, type);
+	if (total < 0)
+		goto fail;
+
+	rows = calloc(total, sizeof(struct tep_record *));
+	if (!rows)
+		goto fail_free;
+
+	for (count = 0; count < total; count++) {
+		int next_cpu;
+
+		next_cpu = pick_next_cpu(rec_list, stream->n_cpus, type);
+
+		if (next_cpu >= 0) {
+			rec = rec_list[next_cpu]->rec;
+			rows[count] = rec;
+
+			temp_rec = rec_list[next_cpu];
+			rec_list[next_cpu] = rec_list[next_cpu]->next;
+			free(temp_rec);
+			/* The record is still referenced in rows */
+		}
+	}
+
+	/* There should be no records left in rec_list */
+	free_rec_list(rec_list, stream->n_cpus, type);
+	*data_rows = rows;
+	return total;
+
+ fail_free:
+	free_rec_list(rec_list, stream->n_cpus, type);
+
+ fail:
+	fprintf(stderr, "Failed to allocate memory during data loading.\n");
+	return -ENOMEM;
+}
+
+static const int tepdata_get_event_id(struct kshark_data_stream *stream,
+				      const struct kshark_entry *entry)
+{
+	int event_id = KS_EMPTY_BIN;
+	struct tep_record *record;
+
+	if (entry->visible & KS_PLUGIN_UNTOUCHED_MASK) {
+		event_id = entry->event_id;
+	} else {
+		/*
+		 * The entry has been touched by a plugin callback function.
+		 * Because of this we do not trust the value of
+		 * "entry->event_id".
+		 *
+		 * Currently the data reading operations are not thread-safe.
+		 * Use a mutex to protect the access.
+		 */
+		pthread_mutex_lock(&stream->input_mutex);
+
+		record = tracecmd_read_at(kshark_get_tep_input(stream),
+					  entry->offset, NULL);
+
+		if (record)
+			event_id = tep_data_type(kshark_get_tep(stream), record);
+
+		free_record(record);
+
+		pthread_mutex_unlock(&stream->input_mutex);
+	}
+
+	return (event_id == -1)? -EFAULT : event_id;
+}
+
+static char* missed_events_dump(struct kshark_data_stream *stream,
+				      const struct kshark_entry *entry,
+				      bool get_info)
+{
+	char *buffer;
+	int size = 0;
+
+	if (get_info)
+		size = asprintf(&buffer, "missed_events=%i",
+				(int) entry->offset);
+	else
+		size = asprintf(&buffer, "missed_events");
+
+	if (size > 0)
+		return buffer;
+
+	return NULL;
+}
+
+static char *tepdata_get_event_name(struct kshark_data_stream *stream,
+				    const struct kshark_entry *entry)
+{
+	struct tep_event *event;
+	char *buffer;
+
+	int event_id = stream->interface.get_event_id(stream, entry);
+	if (event_id == -EFAULT)
+		return NULL;
+
+	if (event_id < 0) {
+		switch (event_id) {
+		case KS_EVENT_OVERFLOW:
+			return missed_events_dump(stream, entry, false);
+		default:
+			return NULL;
+		}
+	}
+
+	/*
+	 * Currently the data reading operations are not thread-safe.
+	 * Use a mutex to protect the access.
+	 */
+	pthread_mutex_lock(&stream->input_mutex);
+
+	event = tep_find_event(kshark_get_tep(stream), event_id);
+
+	pthread_mutex_unlock(&stream->input_mutex);
+
+	if (!event ||
+            asprintf(&buffer, "%s/%s", event->system, event->name) <= 0)
+		return NULL;
+
+	return buffer;
+}
+
+static const int tepdata_get_pid(struct kshark_data_stream *stream,
+				 const struct kshark_entry *entry)
+{
+	struct tep_record *record;
+	int pid = KS_EMPTY_BIN;
+
+	if (entry->visible & KS_PLUGIN_UNTOUCHED_MASK) {
+		pid = entry->pid;
+	} else {
+		/*
+		 * The entry has been touched by a plugin callback function.
+		 * Because of this we do not trust the value of "entry->pid".
+		 *
+		 * Currently the data reading operations are not thread-safe.
+		 * Use a mutex to protect the access.
+		 */
+		pthread_mutex_lock(&stream->input_mutex);
+
+		record = tracecmd_read_at(kshark_get_tep_input(stream),
+					  entry->offset, NULL);
+
+		if (record)
+			pid = tep_data_pid(kshark_get_tep(stream), record);
+
+		free_record(record);
+
+		pthread_mutex_unlock(&stream->input_mutex);
+	}
+
+	return pid;
+}
+
+static char *tepdata_get_task(struct kshark_data_stream *stream,
+			      const struct kshark_entry *entry)
+{
+	int pid = stream->interface.get_pid(stream, entry);
+	const char *task;
+	char *buffer;
+
+	task = tep_data_comm_from_pid(kshark_get_tep(stream), pid);
+	if (asprintf(&buffer, "%s", task)  <= 0)
+		return NULL;
+
+	return buffer;
+}
+
+static char *tepdata_get_latency(struct kshark_data_stream *stream,
+				 const struct kshark_entry *entry)
+{
+	struct tep_record *record;
+	char *buffer;
+
+	/* Check if this is a "Missed event" (event_id < 0). */
+	if (!init_thread_seq() || entry->event_id < 0)
+		return NULL;
+
+	/*
+	 * Currently the data reading operations are not thread-safe.
+	 * Use a mutex to protect the access.
+	 */
+	pthread_mutex_lock(&stream->input_mutex);
+
+	record = tracecmd_read_at(kshark_get_tep_input(stream), entry->offset, NULL);
+
+	if (!record)
+		return NULL;
+
+	trace_seq_reset(&seq);
+	tep_print_event(kshark_get_tep(stream), &seq, record,
+			"%s", TEP_PRINT_LATENCY);
+
+	free_record(record);
+
+	pthread_mutex_unlock(&stream->input_mutex);
+
+	if (asprintf(&buffer, "%s", seq.buffer)  <= 0)
+		return NULL;
+
+	return buffer;
+}
+
+static char *get_info_str(struct kshark_data_stream *stream,
+			  struct tep_record *record,
+			  struct tep_event *event)
+{
+	char *pos, *buffer;
+
+	if (!init_thread_seq() || !record || !event)
+		return NULL;
+
+	trace_seq_reset(&seq);
+	tep_print_event(kshark_get_tep(stream), &seq, record,
+			"%s", TEP_PRINT_INFO);
+
+	/*
+	 * The event info string contains a trailing newline.
+	 * Remove this newline.
+	 */
+	if ((pos = strchr(seq.buffer, '\n')) != NULL)
+		*pos = '\0';
+
+	if (asprintf(&buffer, "%s", seq.buffer)  <= 0)
+		return NULL;
+
+	return buffer;
+}
+
+static char *tepdata_get_info(struct kshark_data_stream *stream,
+			      const struct kshark_entry *entry)
+{
+	struct tep_record *record;
+	struct tep_event *event;
+	char *info = NULL;
+	int event_id;
+
+	if (entry->event_id < 0) {
+		switch (entry->event_id) {
+		case KS_EVENT_OVERFLOW:
+			return missed_events_dump(stream, entry, true);
+		default:
+			return NULL;
+		}
+	}
+
+	/*
+	 * Currently the data reading operations are not thread-safe.
+	 * Use a mutex to protect the access.
+	 */
+	pthread_mutex_lock(&stream->input_mutex);
+
+	record = tracecmd_read_at(kshark_get_tep_input(stream), entry->offset, NULL);
+	if (!record) {
+		pthread_mutex_unlock(&stream->input_mutex);
+		return NULL;
+	}
+
+	event_id = tep_data_type(kshark_get_tep(stream), record);
+	event = tep_find_event(kshark_get_tep(stream), event_id);
+
+	if (event)
+		info = get_info_str(stream, record, event);
+
+	free_record(record);
+
+	pthread_mutex_unlock(&stream->input_mutex);

src/libkshark-tepdata.c line 817

Quoted 10 lines
src/libkshark-tepdata.c new file +10 -0
+
+	return info;
+}
+
+static int *tepdata_get_event_ids(struct kshark_data_stream *stream)
+{
+	struct tep_event **events;
+	int i, *evt_ids;
+
+	events = tep_list_events(kshark_get_tep(stream), TEP_EVENT_SORT_SYSTEM);

I believe that tep_list_events() can fail. Need to check.

src/libkshark-tepdata.c line 818

Quoted 1 line
src/libkshark-tepdata.c new file +1 -0
+	evt_ids = malloc(stream->n_events * sizeof(*evt_ids));

And need to check evt_ids.

Quoted 13 lines
src/libkshark-tepdata.c new file +13 -0
+
+	for (i = 0; i < stream->n_events ; ++i)
+		evt_ids[i] = events[i]->id;
+
+	return evt_ids;
+}
+
+static int tepdata_get_field_names(struct kshark_data_stream *stream,
+				   const struct kshark_entry *entry,
+				   char ***fields_str)
+{
+	struct tep_format_field *field, **fields;
+	struct tep_event *event;

src/libkshark-tepdata.c line 841

Quoted 10 lines
src/libkshark-tepdata.c new file +10 -0
+	int i= 0, nr_fields;
+	char **buffer;
+
+	*fields_str = NULL;
+	event = tep_find_event(kshark_get_tep(stream), entry->event_id);
+	if (!event)
+		return 0;
+
+	nr_fields = event->format.nr_fields + event->format.nr_common;
+	buffer = calloc(nr_fields, sizeof(**fields_str));

Need to check buffer.

src/libkshark-tepdata.c line 843

Quoted 2 lines
src/libkshark-tepdata.c new file +2 -0
+
+	fields = tep_event_common_fields(event);

And fields.

Quoted 254 lines
src/libkshark-tepdata.c new file +254 -0
+	for (field = *fields; field; field = field->next)
+		if (asprintf(&buffer[i++], "%s", field->name) <= 0)
+			goto fail;
+
+	free(fields);
+
+	fields = tep_event_fields(event);
+	for (field = *fields; field; field = field->next)
+		if (asprintf(&buffer[i++], "%s", field->name) <= 0)
+			goto fail;
+
+	free(fields);
+
+	*fields_str = buffer;
+	return nr_fields;
+
+ fail:
+	for (i = 0; i < nr_fields; ++i)
+		free(buffer[i]);
+
+	return -EFAULT;
+}
+
+/**
+ * Custom entry info function type. To be user for dumping info for custom
+ * KernelShark entryes.
+ */
+typedef char *(tepdata_custom_info_func)(struct kshark_data_stream *,
+					const struct kshark_entry *,
+					bool);
+
+static char* tepdata_dump_custom_entry(struct kshark_data_stream *stream,
+				       const struct kshark_entry *entry,
+				       tepdata_custom_info_func info_func)
+{
+	char *entry_str;
+	int size = 0;
+
+	size = asprintf(&entry_str, "%" PRIu64 "; %s-%i; CPU %i; ; %s; %s; 0x%x",
+			entry->ts,
+			tep_data_comm_from_pid(kshark_get_tep(stream), entry->pid),
+			entry->pid,
+			entry->cpu,
+			info_func(stream, entry, false),
+			info_func(stream, entry, true),
+			entry->visible);
+
+	if (size > 0)
+		return entry_str;
+
+	return NULL;
+}
+
+/**
+ * @brief Dump into a string the content of one entry. The function allocates
+ *	  a null terminated string and returns a pointer to this string.
+ *
+ * @param stream: Input location for the FTRACE data stream pointer.
+ * @param entry: A Kernel Shark entry to be printed.
+ *
+ * @returns The returned string contains a semicolon-separated list of data
+ *	    fields. The user has to free the returned string.
+ */
+static char *tepdata_dump_entry(struct kshark_data_stream *stream,
+				const struct kshark_entry *entry)
+{
+	char *entry_str, *task, *latency, *event, *info;
+	struct kshark_context *kshark_ctx = NULL;
+	int n = 0;
+
+	if (!kshark_instance(&kshark_ctx) || !init_thread_seq())
+		return NULL;
+
+	if (entry->event_id >= 0) {
+		if (kshark_get_tep(stream)) {
+			task = stream->interface.get_task(stream, entry);
+			latency = stream->interface.get_latency(stream, entry);
+			event = stream->interface.get_event_name(stream, entry);
+			info = stream->interface.get_info(stream, entry);
+
+			n = asprintf(&entry_str,
+				     "%i; %" PRIu64 "; %s-%i; CPU %i; %s; %s; %s; 0x%x",
+				     entry->stream_id,
+				     entry->ts,
+				     task,
+				     stream->interface.get_pid(stream, entry),
+				     entry->cpu,
+				     latency,
+				     event,
+				     info,
+				     entry->visible);
+
+			free(task);
+			free(latency);
+			free(event);
+			free(info);
+		} else {
+			n = asprintf(&entry_str,
+				     "%i; %li; [UNKNOWN TASK]-%i; CPU %i; ; [UNKNOWN EVENT]; [NO INFO]; 0x%x",
+				     entry->stream_id,
+				     entry->ts,
+				     stream->interface.get_pid(stream, entry),
+				     entry->cpu,
+				     entry->visible);
+		}
+
+		if (n < 1)
+			return NULL;
+	} else {
+		switch (entry->event_id) {
+		case KS_EVENT_OVERFLOW:
+			entry_str = tepdata_dump_custom_entry(stream, entry,
+							     missed_events_dump);
+		default:
+			return NULL;
+		}
+	}
+
+	return entry_str;
+}
+
+static const int tepdata_find_event_id(struct kshark_data_stream *stream,
+				       const char *event_name)
+{
+	struct tep_event *event;
+	char *buffer, *system, *name;
+
+	if (asprintf(&buffer, "%s", event_name) < 1)
+		return -1;
+
+	system = strtok(buffer, "/");
+	name = strtok(NULL, "");
+	if (!system || !name)
+		return -1;
+
+	event = tep_find_event_by_name(kshark_get_tep(stream), system, name);
+
+	free(buffer);
+
+	if (!event)
+		return -1;
+
+	return event->id;
+}
+
+static struct tep_format_field *
+get_evt_field(struct kshark_data_stream *stream,
+	      int event_id, const char *field_name)
+{
+	struct tep_event *event = tep_find_event(kshark_get_tep(stream),
+						 event_id);
+	if (!event)
+		return NULL;
+
+	return tep_find_any_field(event, field_name);
+}
+
+/**
+ * @brief  Get the type of a trace record field. For the moment only integer
+ *	   fields are supported.
+ *
+ * @param stream: Input location for the FTRACE data stream pointer.
+ * @param entry: Input location for the Kernel Shark entry asociated with thes
+ *		 record.
+ * @param field: The name of the field.
+ *
+ * @returns KS_INTEGER_FIELD in case the field has an integer type. Otherwise
+ *	    KS_INVALIDE_FIELD.
+ */
+kshark_event_field_format
+tepdata_get_field_type(struct kshark_data_stream *stream,
+		       const struct kshark_entry *entry,
+		       const char *field)
+{
+	struct tep_format_field *evt_field;
+	int mask = ~(TEP_FIELD_IS_SIGNED |
+		     TEP_FIELD_IS_LONG |
+		     TEP_FIELD_IS_FLAG);
+
+	evt_field = get_evt_field(stream, entry->event_id, field);
+	if (!evt_field)
+		return KS_INVALIDE_FIELD;
+
+	if (mask & evt_field->flags)
+		return KS_INVALIDE_FIELD;
+
+	return KS_INTEGER_FIELD;
+}
+
+/**
+ * @brief  Get the value of a trace record field.
+ *
+ * @param stream: Input location for the FTRACE data stream pointer.
+ * @param rec: Input location for the trace record.
+ * @param field: The name of the field.
+ * @param val: Output location for the field value.
+ *
+ * @returns Returns 0 on success, otherwise a negative error code..
+ */
+int tepdata_read_record_field(struct kshark_data_stream *stream,
+			      void *rec,
+			      const char *field, int64_t *val)
+{
+	struct tep_format_field *evt_field;
+	struct tep_record *record = rec;
+	int event_id, ret;
+
+	if (!record)
+		return -EFAULT;
+
+	event_id = tep_data_type(kshark_get_tep(stream), record);
+	evt_field = get_evt_field(stream, event_id, field);
+	if (!evt_field)
+		return -EINVAL;
+
+	ret = tep_read_number_field(evt_field, record->data,
+				    (unsigned long long *) val);
+
+	return ret;
+}
+
+/**
+ * @brief  Get the value of a trace record field.
+ *
+ * @param stream: Input location for the FTRACE data stream pointer.
+ * @param entry: Input location for the Kernel Shark entry asociated with thes
+ *		 record.
+ * @param field: The name of the field.
+ * @param val: Output location for the field value.
+ *
+ * @returns Returns 0 on success, otherwise a negative error code.
+ */
+int tepdata_read_event_field(struct kshark_data_stream *stream,
+			     const struct kshark_entry *entry,
+			     const char *field, int64_t *val)
+{
+	struct tep_format_field *evt_field;
+	struct tep_record *record;
+	int ret;
+
+	evt_field = get_evt_field(stream, entry->event_id, field);
+	if (!evt_field)
+		return -EINVAL;
+
+	record = tracecmd_read_at(kshark_get_tep_input(stream),
+				  entry->offset, NULL);
+	if (!record)
+		return -EFAULT;
+
+	ret = tep_read_number_field(evt_field, record->data,
+				    (unsigned long long *) val);
+	free_record(record);
+
+	return ret;

src/libkshark-tepdata.c line 1107

Quoted 10 lines
src/libkshark-tepdata.c new file +10 -0
+}
+
+/** Initialize all methods used by a stream of FTRACE data. */
+static void kshark_tep_init_methods(struct kshark_data_stream *stream)
+{
+	stream->interface.get_pid = tepdata_get_pid;
+	stream->interface.get_task = tepdata_get_task;
+	stream->interface.get_event_id = tepdata_get_event_id;
+	stream->interface.get_event_name = tepdata_get_event_name;
+	stream->interface.get_latency = tepdata_get_latency;

Since "latency" is a very specific field, I wonder if we should call this "aux_field", and have a way to express what to show in the table. That is, if there's not a "aux_field" defined, it ignores it. Not all streams may have this.

-- Steve

Quoted 341 lines
src/libkshark-tepdata.c new file +284 -0
+	stream->interface.get_info = tepdata_get_info;
+	stream->interface.find_event_id = tepdata_find_event_id;
+	stream->interface.get_all_event_ids = tepdata_get_event_ids;
+	stream->interface.dump_entry = tepdata_dump_entry;
+	stream->interface.get_all_field_names = tepdata_get_field_names;
+	stream->interface.get_event_field_type = tepdata_get_field_type;
+	stream->interface.read_record_field_int64 = tepdata_read_record_field;
+	stream->interface.read_event_field_int64 = tepdata_read_event_field;
+	stream->interface.load_entries = tepdata_load_entries;
+	stream->interface.load_matrix = tepdata_load_matrix;
+}
+
+/** Find a host stream from the same tracing session, that has guest information */
+static struct tracecmd_input *
+kshark_tep_find_merge_peer(struct kshark_context *kshark_ctx,
+			   struct tracecmd_input *handle)
+{
+	struct tracecmd_input *peer_handle = NULL;
+	struct kshark_data_stream *peer_stream;
+	unsigned long long trace_id;
+	int *stream_ids = NULL;
+	int ret;
+	int i;
+
+	trace_id = tracecmd_get_traceid(handle);
+	if (!trace_id)
+		goto out;
+
+	stream_ids = kshark_all_streams(kshark_ctx);
+	if (!stream_ids)
+		goto out;
+
+	for (i = 0; i < kshark_ctx->n_streams - 1; i++) {
+		peer_stream = kshark_get_data_stream(kshark_ctx, stream_ids[i]);
+		if (!peer_stream || peer_stream->format != KS_TEP_DATA)
+			continue;
+
+		peer_handle = kshark_get_tep_input(peer_stream);
+		if (!peer_handle)
+			continue;
+
+		ret = tracecmd_get_guest_cpumap(peer_handle, trace_id,
+						NULL, NULL, NULL);
+		if (!ret)
+			break;
+	}
+
+	if (i == kshark_ctx->n_streams)
+		peer_handle = NULL;
+
+out:
+	free(stream_ids);
+	return peer_handle;
+}
+
+/** The Process Id of the Idle tasks is zero. */
+#define LINUX_IDLE_TASK_PID	0
+
+static int kshark_tep_stream_init(struct kshark_data_stream *stream,
+				  struct tracecmd_input *input)
+{
+	struct tepdata_handle *tep_handle;
+	struct tep_event *event;
+
+	tep_handle = calloc(1, sizeof(*tep_handle));
+	if (!tep_handle)
+		goto fail;
+
+	tep_handle->input = input;
+	tep_handle->tep = tracecmd_get_pevent(tep_handle->input);
+	if (!tep_handle->tep)
+		goto fail;
+
+	tep_handle->sched_switch_event_id = -EINVAL;
+	event = tep_find_event_by_name(tep_handle->tep,
+				       "sched", "sched_switch");
+	if (event) {
+		tep_handle->sched_switch_event_id = event->id;
+
+		tep_handle->sched_switch_next_field =
+			tep_find_any_field(event, "next_pid");
+
+		tep_handle->sched_switch_comm_field =
+			tep_find_field(event, "next_comm");
+	}
+
+	stream->n_cpus = tep_get_cpus(tep_handle->tep);
+	stream->n_events = tep_get_events_count(tep_handle->tep);
+	stream->idle_pid = LINUX_IDLE_TASK_PID;
+
+	tep_handle->advanced_event_filter =
+		tep_filter_alloc(tep_handle->tep);
+
+	kshark_tep_init_methods(stream);
+
+	stream->interface.handle = tep_handle;
+
+	return 0;
+
+ fail:
+	free(tep_handle);
+	stream->interface.handle = NULL;
+	return -EFAULT;
+}
+
+/** Initialize the FTRACE data input (from file). */
+int kshark_tep_init_input(struct kshark_data_stream *stream,
+			  const char *file)
+{
+	struct kshark_context *kshark_ctx = NULL;
+	struct tracecmd_input *merge_peer;
+	struct tracecmd_input *input;
+
+	if (!kshark_instance(&kshark_ctx) || !init_thread_seq())
+		return -EEXIST;
+
+	/*
+	 * Turn off function trace indent and turn on show parent
+	 * if possible.
+	 */
+	tep_plugin_add_option("ftrace:parent", "1");
+	tep_plugin_add_option("ftrace:indent", "0");
+
+	input = tracecmd_open_head(file);
+	if (!input)
+		return -EEXIST;
+
+	/* Find a merge peer from the same tracing session. */
+	merge_peer = kshark_tep_find_merge_peer(kshark_ctx, input);
+	if (merge_peer)
+		tracecmd_pair_peer(input, merge_peer);
+
+	/* Read the tracing data from the file. */
+	if (tracecmd_init_data(input) < 0)
+		goto fail;
+
+	/* Initialize the stream asociated with the main buffer. */
+	if (kshark_tep_stream_init(stream, input) < 0)
+		goto fail;
+
+	stream->name = strdup("top");
+
+	return 0;
+
+ fail:
+	tracecmd_close(input);
+	return -EFAULT;
+}
+
+/** Initialize using the locally available tracing events. */
+int kshark_tep_init_local(struct kshark_data_stream *stream)
+{
+	struct tepdata_handle *tep_handle;
+
+	tep_handle = calloc(1, sizeof(*tep_handle));
+	if (!tep_handle)
+		return -EFAULT;
+
+	tep_handle->tep = tracefs_local_events(tracefs_get_tracing_dir());
+	if (!tep_handle->tep)
+		goto fail;
+
+	stream->n_events = tep_get_events_count(tep_handle->tep);
+	stream->n_cpus =  tep_get_cpus(tep_handle->tep);
+	stream->format = KS_TEP_DATA;
+	if (asprintf(&stream->file, "Local system") <= 0)
+		goto fail;
+
+	stream->interface.handle = tep_handle;
+	kshark_tep_init_methods(stream);
+
+	return 0;
+
+ fail:
+	free(tep_handle);
+	stream->interface.handle = NULL;
+	return -EFAULT;
+}
+
+/** Method used to close a stream of FTRACE data. */
+void kshark_tep_close_interface(struct kshark_data_stream *stream)
+{
+	struct tepdata_handle *tep_handle = stream->interface.handle;
+
+	if (seq.buffer)
+		trace_seq_destroy(&seq);
+
+	if (tep_handle->advanced_event_filter) {
+		tep_filter_reset(tep_handle->advanced_event_filter);
+		tep_filter_free(tep_handle->advanced_event_filter);
+		tep_handle->advanced_event_filter = NULL;
+	}
+
+	if (tep_handle->input)
+		tracecmd_close(tep_handle->input);
+
+	free(tep_handle);
+	stream->interface.handle = NULL;
+}
+
+/** Check if the filter any filter is set. */
+bool kshark_tep_filter_is_set(struct kshark_data_stream *stream)
+{
+	struct tep_event_filter *adv_filter = get_adv_filter(stream);
+
+	if (adv_filter && adv_filter->filters)
+		return true;
+
+	return false;
+}
+
+/**
+ * @brief Add a filter based on the content of the event.
+ *
+ * @param stream: Input location for the FTRACE data stream pointer.
+ * @param filter_str: The definition of the filter.
+ *
+ * @returns 0 if the filter was successfully added or a negative error code.
+ */
+int kshark_tep_add_filter_str(struct kshark_data_stream *stream,
+			       const char *filter_str)
+{
+	struct tep_event_filter *adv_filter = get_adv_filter(stream);
+	int ret = tep_filter_add_filter_str(adv_filter, filter_str);
+
+	if (ret < 0) {
+		char error_str[200];
+		int error_status =
+			tep_strerror(kshark_get_tep(stream), ret, error_str,
+				     sizeof(error_str));
+
+		if (error_status == 0)
+			fprintf(stderr, "filter failed due to: %s\n",
+					error_str);
+	}
+
+	return ret;
+}
+
+/**
+ * @brief Get a string showing the filter definition.
+ *
+ * @param stream: Input location for the FTRACE data stream pointer.
+ * @param event_id: The unique Id of the event type of the filter.
+ *
+ * @returns A string that displays the filter contents. This string must be
+ *	    freed with free(str). NULL is returned if no filter is found or
+ *	    allocation failed.
+ */
+char *kshark_tep_filter_make_string(struct kshark_data_stream *stream,
+				    int event_id)
+{
+	struct tep_event_filter *adv_filter = get_adv_filter(stream);
+
+	return tep_filter_make_string(adv_filter, event_id);
+}
+
+/**
+ * @brief Remove a filter based on the content of the event.
+ *
+ * @param stream: Input location for the FTRACE data stream pointer.
+ * @param event_id: The unique Id of the event type of the filter.
+ *
+ * @return 1: if an event was removed or 0 if the event was not found.
+ */
+int kshark_tep_filter_remove_event(struct kshark_data_stream *stream,
+				   int event_id)
+{
+	struct tep_event_filter *adv_filter = get_adv_filter(stream);
+
+	return tep_filter_remove_event(adv_filter, event_id);
+}
+
+/** Reset all filters based on the content of the event. */
+void kshark_tep_filter_reset(struct kshark_data_stream *stream)
+{
+	return tep_filter_reset(get_adv_filter(stream));
+}
+
+/** Get an array of available tracer plugins. */
+char **kshark_tracecmd_local_plugins()
+{
+	return tracefs_tracers(tracefs_get_tracing_dir());
+}
src/libkshark-tepdata.h new file +51 -0
new file mode 100644
index 00000000..a2bd211e
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+
+/*
+ * Copyright (C) 2019 VMware Inc, Yordan Karadzhov (VMware) <[email protected]>
+ */
+
+/**
+ *  @file    libkshark-tepdata.h
+ *  @brief   Interface for processing of FTRACE (trace-cmd) data.
+ */
+
+#ifndef _KSHARK_TEPDATA_H
+#define _KSHARK_TEPDATA_H
+
+// KernelShark
+#include "libkshark.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+int kshark_tep_init_input(struct kshark_data_stream *stream,
+			  const char *file);
+
+int kshark_tep_init_local(struct kshark_data_stream *stream);
+
+void kshark_tep_close_interface(struct kshark_data_stream *stream);
+
+bool kshark_tep_filter_is_set(struct kshark_data_stream *stream);
+
+int kshark_tep_add_filter_str(struct kshark_data_stream *stream,
+			      const char *filter_str);
+
+char *kshark_tep_filter_make_string(struct kshark_data_stream *stream,
+				    int event_id);
+
+int kshark_tep_filter_remove_event(struct kshark_data_stream *stream,
+				   int event_id);
+
+void kshark_tep_filter_reset(struct kshark_data_stream *stream);
+
+char **kshark_tracecmd_local_plugins();
+
+ssize_t kshark_load_tep_records(struct kshark_context *kshark_ctx, int sd,
+				struct tep_record ***data_rows);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif // _KSHARK_TEPDATA_H
#24 lore raw

Re: [PATCH v2 09/20] kernel-shark: Add stream interface for trace-cmd data

Quoted 22 lines · Steven Rostedt
Quoted 19 lines · On Mon, 12 Oct 2020 16:35:12 +0300 "Yordan Karadzhov (VMware)"…

Here we provide an implementation of the Data stream interface that will process the trace-cmd (Ftrace) data. This can be considered the most essential change in the transformation of the C API towards version 2. However, for the moment we only have stand alone definitions that are not made functional yet. The actual integration with the API will be introduced in the following patches.

---

src/libkshark-tepdata.c new file +8 -0
+
+static ssize_t get_records(struct kshark_context *kshark_ctx,
+			   struct kshark_data_stream *stream,
+			   struct rec_list ***rec_list,
+			   enum rec_type type)
+{
+	struct tep_event_filter *adv_filter = NULL;
+	struct rec_list **temp_next;

....

Quoted 12 lines
Quoted 12 lines
src/libkshark-tepdata.c new file +12 -0
+	ret = tep_read_number_field(evt_field, record->data,
+				    (unsigned long long *) val);
+	free_record(record);
+
+	return ret;
+}
+
+/** Initialize all methods used by a stream of FTRACE data. */
+static void kshark_tep_init_methods(struct kshark_data_stream *stream)
+{
+	stream->interface.get_pid = tepdata_get_pid;
+	stream->interface.get_task = tepdata_get_task;
Quoted 9 lines

src/libkshark-tepdata.c line 1107

Quoted 3 lines
src/libkshark-tepdata.c new file +3 -0
+	stream->interface.get_event_id = tepdata_get_event_id;
+	stream->interface.get_event_name = tepdata_get_event_name;
+	stream->interface.get_latency = tepdata_get_latency;

Since "latency" is a very specific field, I wonder if we should call this "aux_field", and have a way to express what to show in the table. That is, if there's not a "aux_field" defined, it ignores it. Not all streams may have this.

This is a good idea. I will change it in v3 and will fix all the issues you pointed out above.

Thanks!
Yordan

Quoted 14 lines

-- Steve

Quoted 12 lines
src/libkshark-tepdata.c new file +12 -0
+	stream->interface.get_info = tepdata_get_info;
+	stream->interface.find_event_id = tepdata_find_event_id;
+	stream->interface.get_all_event_ids = tepdata_get_event_ids;
+	stream->interface.dump_entry = tepdata_dump_entry;
+	stream->interface.get_all_field_names = tepdata_get_field_names;
+	stream->interface.get_event_field_type = tepdata_get_field_type;
+	stream->interface.read_record_field_int64 = tepdata_read_record_field;
+	stream->interface.read_event_field_int64 = tepdata_read_event_field;
+	stream->interface.load_entries = tepdata_load_entries;
+	stream->interface.load_matrix = tepdata_load_matrix;
+}
+
#25 lore raw

PATCH v2 10/20 kernel-shark: Start using data streams

In reply to Yordan Karadzhov (VMware)

Here we switch to using the trace data readout, provided by the Data stream interface. The actual implementation of the new readout was done in the previous commits.

---
 examples/dataload.c |  21 ++-
 src/libkshark.c     | 336 ++++++++++++++++++--------------------------
 src/libkshark.h     |  24 +++-
 3 files changed, 164 insertions(+), 217 deletions(-)
examples/dataload.c +9 -12
index 15c5de05..c13acbcb 100644
@@ -19,8 +19,7 @@ int main(int argc, char **argv)
 	struct kshark_entry **data = NULL;
 	ssize_t r, n_rows, n_tasks;
 	char *entry_str;
-	bool status;
-	int *pids;
+	int sd, *pids;
 
 	/* Create a new kshark session. */
 	kshark_ctx = NULL;
@@ -29,30 +28,28 @@ int main(int argc, char **argv)
 
 	/* Open a trace data file produced by trace-cmd. */
 	if (argc > 1)
-		status = kshark_open(kshark_ctx, argv[1]);
+		sd = kshark_open(kshark_ctx, argv[1]);
 	else
-		status = kshark_open(kshark_ctx, default_file);
+		sd = kshark_open(kshark_ctx, default_file);
 
-	if (!status) {
+	if (sd < 0) {
 		kshark_free(kshark_ctx);
 		return 1;
 	}
 
 	/* Load the content of the file into an array of entries. */
-	n_rows = kshark_load_data_entries(kshark_ctx, &data);
+	n_rows = kshark_load_entries(kshark_ctx, sd, &data);
 	if (n_rows < 1) {
 		kshark_free(kshark_ctx);
 		return 1;
 	}
 
 	/* Print to the screen the list of all tasks. */
-	n_tasks = kshark_get_task_pids(kshark_ctx, &pids);
+	n_tasks = kshark_get_task_pids(kshark_ctx, sd, &pids);
 	for (r = 0; r < n_tasks; ++r) {
-		const char *task_str =
-			tep_data_comm_from_pid(kshark_ctx->pevent,
-					       pids[r]);
-
+		char *task_str = kshark_comm_from_pid(sd, pids[r]);
 		printf("task: %s-%i\n", task_str, pids[r]);
+		free(task_str);
 	}
 
 	free(pids);
@@ -82,7 +79,7 @@ int main(int argc, char **argv)
 	free(data);
 
 	/* Close the file. */
-	kshark_close(kshark_ctx);
+	kshark_close(kshark_ctx, sd);
 
 	/* Close the session. */
 	kshark_free(kshark_ctx);
src/libkshark.c +136 -200

425 lines not on this page · raw message

src/libkshark.h +19 -5
index f26fad7c..ef48d94b 100644
@@ -401,7 +401,9 @@ struct kshark_context {
 
 bool kshark_instance(struct kshark_context **kshark_ctx);
 
-bool kshark_open(struct kshark_context *kshark_ctx, const char *file);
+int kshark_open(struct kshark_context *kshark_ctx, const char *file);
+
+int kshark_stream_open(struct kshark_data_stream *stream, const char *file);
 
 int kshark_add_stream(struct kshark_context *kshark_ctx);
 
@@ -426,9 +428,12 @@ size_t kshark_load_data_matrix(struct kshark_context *kshark_ctx,
 			       uint16_t **pid_array,
 			       int **event_array);
 
-ssize_t kshark_get_task_pids(struct kshark_context *kshark_ctx, int **pids);
+ssize_t kshark_get_task_pids(struct kshark_context *kshark_ctx, int sd,
+			     int **pids);
+
+void kshark_close(struct kshark_context *kshark_ctx, int sd);
 
-void kshark_close(struct kshark_context *kshark_ctx);
+void kshark_close_all(struct kshark_context *kshark_ctx);
 
 void kshark_free(struct kshark_context *kshark_ctx);
 
@@ -450,8 +455,6 @@ const char *kshark_get_info_easy(struct kshark_entry *entry);
 
 void kshark_convert_nano(uint64_t time, uint64_t *sec, uint64_t *usec);
 
-char* kshark_dump_entry(const struct kshark_entry *entry);
-
 static inline int kshark_get_pid(const struct kshark_entry *entry)
 {
 	struct kshark_data_stream *stream =
@@ -536,6 +539,17 @@ static inline int kshark_read_event_field(const struct kshark_entry *entry,
 							field, val);
 }
 
+static inline char *kshark_dump_entry(const struct kshark_entry *entry)
+{
+	struct kshark_data_stream *stream =
+		kshark_get_stream_from_entry(entry);
+
+	if (!stream)
+		return NULL;
+
+	return stream->interface.dump_entry(stream, entry);
+}
+
 /**
  * @brief Load the content of the trace data file asociated with a given
  *	  Data stream identifie into an array of kshark_entries.
Signature · 1 line
2.25.1
#26 lore raw

Re: [PATCH v2 10/20] kernel-shark: Start using data streams

Quoted 83 lines · On Mon, 12 Oct 2020 16:35:13 +0300 "Yordan Karadzhov (VMware)"…

Here we switch to using the trace data readout, provided by the Data stream interface. The actual implementation of the new readout was done in the previous commits.

---

src/libkshark.c +10 -38
@@ -19,6 +19,7 @@
 
 // KernelShark
 #include "libkshark.h"
+#include "libkshark-tepdata.h"
 
 static __thread struct trace_seq seq;
 
@@ -32,6 +33,9 @@ static bool kshark_default_context(struct kshark_context **context)
 	if (!kshark_ctx)
 		return false;
 
+	kshark_ctx->stream = calloc(KS_MAX_NUM_STREAMS,
+				    sizeof(*kshark_ctx->stream));
+
 	kshark_ctx->event_handlers = NULL;
 	kshark_ctx->collections = NULL;
 	kshark_ctx->plugins = NULL;
@@ -108,62 +112,28 @@ bool kshark_instance(struct kshark_context **kshark_ctx)
 	return true;
 }
 
-static void kshark_free_task_list(struct kshark_context *kshark_ctx)
-{
-	struct kshark_task_list *task;
-	int i;
-
-	if (!kshark_ctx)
-		return;
-
-	for (i = 0; i < KS_TASK_HASH_SIZE; ++i) {
-		while (kshark_ctx->tasks[i]) {
-			task = kshark_ctx->tasks[i];
-			kshark_ctx->tasks[i] = task->next;
-			free(task);
-		}
-	}
-}
-
 /**
  * @brief Open and prepare for reading a trace data file specified by "file".
- *	  If the specified file does not exist, or contains no trace data,
- *	  the function returns false.
  *
  * @param kshark_ctx: Input location for context pointer.
  * @param file: The file to load.
  *
- * @returns True on success, or false on failure.
+ * @returns The Id number of the data stream associated with this file on success.
+ * 	    Otherwise a negative errno code.
  */
-bool kshark_open(struct kshark_context *kshark_ctx, const char *file)
+int kshark_open(struct kshark_context *kshark_ctx, const char *file)
 {
-	struct tracecmd_input *handle;
-
-	kshark_free_task_list(kshark_ctx);
-
-	handle = tracecmd_open(file);
-	if (!handle)
-		return false;
-
-	if (pthread_mutex_init(&kshark_ctx->input_mutex, NULL) != 0) {
-		tracecmd_close(handle);
-		return false;
-	}
-
-	kshark_ctx->handle = handle;
-	kshark_ctx->pevent = tracecmd_get_pevent(handle);
+	int sd, rt;
 
-	kshark_ctx->advanced_event_filter =
-		tep_filter_alloc(kshark_ctx->pevent);
+	sd = kshark_add_stream(kshark_ctx);
+	if (sd < 0)

src/libkshark.c bool kshark_instance(struct kshark_context **kshark_ctx) · line 133

Quoted 10 lines
src/libkshark.c +3 -6
+		return sd;
 
-	/*
-	 * Turn off function trace indent and turn on show parent
-	 * if possible.
-	 */
-	tep_plugin_add_option("ftrace:parent", "1");
-	tep_plugin_add_option("ftrace:indent", "0");
+	rt = kshark_stream_open(kshark_ctx->stream[sd], file);
+	if (rt < 0)

On failure, we should probably destroy the stream that was created.

Quoted 124 lines
src/libkshark.c +88 -13
+		return rt;
 
-	return true;
+	return sd;
 }
 
 static void kshark_stream_free(struct kshark_data_stream *stream)
@@ -253,6 +223,56 @@ int kshark_add_stream(struct kshark_context *kshark_ctx)
 	return stream->stream_id;
 }
 
+static bool is_tep(const char *filename)
+{
+	/*
+	 * TODO: This is very naive. Implement more appropriate check. Ideally
+	 * it should be part of the trace-cmd library.
+	 */
+	char *ext = strrchr(filename, '.');
+	return ext && strcmp(ext, ".dat") == 0;
+}
+
+static void set_format(struct kshark_context *kshark_ctx,
+		       struct kshark_data_stream *stream,
+		       const char *filename)
+{
+	stream->format = KS_INVALIDE_DATA;
+
+	if (is_tep(filename)) {
+		stream->format = KS_TEP_DATA;
+		return;
+	}
+}
+
+/**
+ * @brief Use an existing Trace data stream to open and prepare for reading
+ *	  a trace data file specified by "file".
+ *
+ * @param stream: Input location for a Trace data stream pointer.
+ * @param file: The file to load.
+ *
+ * @returns Zero on success or a negative error code in the case of an errno.
+ */
+int kshark_stream_open(struct kshark_data_stream *stream, const char *file)
+{
+	struct kshark_context *kshark_ctx = NULL;
+
+	if (!stream || !kshark_instance(&kshark_ctx))
+		return -EFAULT;
+
+	stream->file = strdup(file);
+	set_format(kshark_ctx, stream, file);
+
+	switch (stream->format) {
+	case KS_TEP_DATA:
+		return kshark_tep_init_input(stream, file);
+
+	default:
+		return -ENODATA;
+	}
+}
+
 /**
  * @brief Get the Data stream object having given Id.
  *
@@ -313,45 +333,76 @@ int *kshark_all_streams(struct kshark_context *kshark_ctx)
 	return ids;
 }
 
+static void kshark_stream_close(struct kshark_data_stream *stream)
+{
+	struct kshark_context *kshark_ctx = NULL;
+
+	if (!stream || !kshark_instance(&kshark_ctx))
+		return;
+
+	/*
+	 * All filters are file specific. Make sure that all Process Ids and
+	 * Event Ids from this file are not going to be used with another file.
+	 */
+	kshark_hash_id_clear(stream->show_task_filter);
+	kshark_hash_id_clear(stream->hide_task_filter);
+	kshark_hash_id_clear(stream->show_event_filter);
+	kshark_hash_id_clear(stream->hide_event_filter);
+	kshark_hash_id_clear(stream->show_cpu_filter);
+	kshark_hash_id_clear(stream->hide_cpu_filter);
+
+	switch (stream->format) {
+	case KS_TEP_DATA:
+		kshark_tep_close_interface(stream);
+		break;
+
+	default:
+		break;
+	}
+
+	pthread_mutex_destroy(&stream->input_mutex);
+}
+
 /**
  * @brief Close the trace data file and free the trace data handle.
  *
  * @param kshark_ctx: Input location for the session context pointer.
+ * @param sd: Data stream identifier.
  */
-void kshark_close(struct kshark_context *kshark_ctx)
+void kshark_close(struct kshark_context *kshark_ctx, int sd)
 {
-	if (!kshark_ctx || !kshark_ctx->handle)
+	struct kshark_data_stream *stream;
+
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
 		return;
 
-	/*
-	 * All filters are file specific. Make sure that the Pids and Event Ids
-	 * from this file are not going to be used with another file.
-	 */
-	tracecmd_filter_id_clear(kshark_ctx->show_task_filter);
-	tracecmd_filter_id_clear(kshark_ctx->hide_task_filter);
-	tracecmd_filter_id_clear(kshark_ctx->show_event_filter);
-	tracecmd_filter_id_clear(kshark_ctx->hide_event_filter);
-	tracecmd_filter_id_clear(kshark_ctx->show_cpu_filter);
-	tracecmd_filter_id_clear(kshark_ctx->hide_cpu_filter);

src/libkshark.c int *kshark_all_streams(struct kshark_context *kshark_ctx) · line 383

Quoted 10 lines
src/libkshark.c +4 -6
-
-	if (kshark_ctx->advanced_event_filter) {
-		tep_filter_reset(kshark_ctx->advanced_event_filter);
-		tep_filter_free(kshark_ctx->advanced_event_filter);
-		kshark_ctx->advanced_event_filter = NULL;
-	}
+	kshark_stream_close(stream);
+	kshark_stream_free(stream);
+	kshark_ctx->stream[sd] = NULL;
+	kshark_ctx->n_streams--;

So, if you have multiple streams, and you close one that's not the last, and then add a new one, this will cause the new one to be overwritten.

As add_stream has:

	kshark_ctx->stream[kshark_ctx->n_streams++] = stream;

You may want to do instead:

	kshark_ctx->stream[sd] = NULL;
	while (!kshark->streams[kshark_ctx->n_streams - 1])
		kshark_ctx->n_streams--;

You can also add a free store, where you store an index in the stream[sd] field of the next free item. Then for adding you have:

	if (kshark_ctx->free >= 0) {
		sd = kshark_ctx->free;
		kshark_ctx->free = (int)kshark_ctx->stream[kshark_ctx->free];
	} else {
		sd = kshark_ctx->n_streams++;
	}

and on freeing:

	kshark_ctx->streams[sd] = (void *)kshark_ctx->free;
	kshark_ctx->free = sd;

Just need to initialize kshark_ctx->free to -1.

And never decrement n_streams. If you do any loops over the stream, you could verify that it is a real stream by:

	if ((unsigned long)kshark_ctx->streams[sd] > kshark_ctx->n_streams)
		/* pointer to a stream */
	else
		/* a free item. */

Places like kshark_close() would need the above (if doing a free store).

-- Steve

Quoted 34 lines
src/libkshark.c +18 -9
+}
+
+/**
+ * @brief Close all currently open trace data file and free the trace data handle.
+ *
+ * @param kshark_ctx: Input location for the session context pointer.
+ */
+void kshark_close_all(struct kshark_context *kshark_ctx)
+{
+	int i, *stream_ids, n_streams;
+
+	stream_ids = kshark_all_streams(kshark_ctx);
 
 	/*
-	 * All data collections are file specific. Make sure that collections
-	 * from this file are not going to be used with another file.
+	 * Get a copy of shark_ctx->n_streams befor you start closing. Be aware
+	 * that kshark_close() will decrement shark_ctx->n_streams.
 	 */
-	kshark_free_collection_list(kshark_ctx->collections);
-	kshark_ctx->collections = NULL;
-
-	tracecmd_close(kshark_ctx->handle);
-	kshark_ctx->handle = NULL;
-	kshark_ctx->pevent = NULL;
+	n_streams = kshark_ctx->n_streams;
+	for (i = 0; i < n_streams; ++i)
+		kshark_close(kshark_ctx, stream_ids[i]);
 
-	pthread_mutex_destroy(&kshark_ctx->input_mutex);
+	free(stream_ids);
 }
 
 /**
#27 lore raw

Re: [PATCH v2 10/20] kernel-shark: Start using data streams

Quoted 88 lines · Steven Rostedt
Quoted 85 lines · On Mon, 12 Oct 2020 16:35:13 +0300 "Yordan Karadzhov (VMware)"…

Here we switch to using the trace data readout, provided by the Data stream interface. The actual implementation of the new readout was done in the previous commits.

---

src/libkshark.c +11 -38
@@ -19,6 +19,7 @@
 
  // KernelShark
  #include "libkshark.h"
+#include "libkshark-tepdata.h"
 
  static __thread struct trace_seq seq;
 
@@ -32,6 +33,9 @@ static bool kshark_default_context(struct kshark_context **context)
  	if (!kshark_ctx)
  		return false;
 
+	kshark_ctx->stream = calloc(KS_MAX_NUM_STREAMS,
+				    sizeof(*kshark_ctx->stream));
+
  	kshark_ctx->event_handlers = NULL;
  	kshark_ctx->collections = NULL;
  	kshark_ctx->plugins = NULL;
@@ -108,62 +112,28 @@ bool kshark_instance(struct kshark_context **kshark_ctx)
  	return true;
  }
 
-static void kshark_free_task_list(struct kshark_context *kshark_ctx)
-{
-	struct kshark_task_list *task;
-	int i;
-
-	if (!kshark_ctx)
-		return;
-
-	for (i = 0; i < KS_TASK_HASH_SIZE; ++i) {
-		while (kshark_ctx->tasks[i]) {
-			task = kshark_ctx->tasks[i];
-			kshark_ctx->tasks[i] = task->next;
-			free(task);
-		}
-	}
-}
-
  /**
   * @brief Open and prepare for reading a trace data file specified by "file".
- *	  If the specified file does not exist, or contains no trace data,
- *	  the function returns false.
   *
   * @param kshark_ctx: Input location for context pointer.
   * @param file: The file to load.
   *
- * @returns True on success, or false on failure.
+ * @returns The Id number of the data stream associated with this file on success.
+ * 	    Otherwise a negative errno code.
   */
-bool kshark_open(struct kshark_context *kshark_ctx, const char *file)
+int kshark_open(struct kshark_context *kshark_ctx, const char *file)
  {
-	struct tracecmd_input *handle;
-
-	kshark_free_task_list(kshark_ctx);
-
-	handle = tracecmd_open(file);
-	if (!handle)
-		return false;
-
-	if (pthread_mutex_init(&kshark_ctx->input_mutex, NULL) != 0) {
-		tracecmd_close(handle);
-		return false;
-	}
-
-	kshark_ctx->handle = handle;
-	kshark_ctx->pevent = tracecmd_get_pevent(handle);
+	int sd, rt;
 
-	kshark_ctx->advanced_event_filter =
-		tep_filter_alloc(kshark_ctx->pevent);
+	sd = kshark_add_stream(kshark_ctx);
+	if (sd < 0)
+		return sd;
 
Quoted 10 lines

src/libkshark.c bool kshark_instance(struct kshark_context **kshark_ctx) · line 133

Quoted 8 lines
src/libkshark.c +2 -6
-	/*
-	 * Turn off function trace indent and turn on show parent
-	 * if possible.
-	 */
-	tep_plugin_add_option("ftrace:parent", "1");
-	tep_plugin_add_option("ftrace:indent", "0");
+	rt = kshark_stream_open(kshark_ctx->stream[sd], file);
+	if (rt < 0)

On failure, we should probably destroy the stream that was created.

Correct, will be fixed in v3.

Quoted 139 lines
Quoted 125 lines
src/libkshark.c +88 -13
+		return rt;
 
-	return true;
+	return sd;
  }
 
  static void kshark_stream_free(struct kshark_data_stream *stream)
@@ -253,6 +223,56 @@ int kshark_add_stream(struct kshark_context *kshark_ctx)
  	return stream->stream_id;
  }
 
+static bool is_tep(const char *filename)
+{
+	/*
+	 * TODO: This is very naive. Implement more appropriate check. Ideally
+	 * it should be part of the trace-cmd library.
+	 */
+	char *ext = strrchr(filename, '.');
+	return ext && strcmp(ext, ".dat") == 0;
+}
+
+static void set_format(struct kshark_context *kshark_ctx,
+		       struct kshark_data_stream *stream,
+		       const char *filename)
+{
+	stream->format = KS_INVALIDE_DATA;
+
+	if (is_tep(filename)) {
+		stream->format = KS_TEP_DATA;
+		return;
+	}
+}
+
+/**
+ * @brief Use an existing Trace data stream to open and prepare for reading
+ *	  a trace data file specified by "file".
+ *
+ * @param stream: Input location for a Trace data stream pointer.
+ * @param file: The file to load.
+ *
+ * @returns Zero on success or a negative error code in the case of an errno.
+ */
+int kshark_stream_open(struct kshark_data_stream *stream, const char *file)
+{
+	struct kshark_context *kshark_ctx = NULL;
+
+	if (!stream || !kshark_instance(&kshark_ctx))
+		return -EFAULT;
+
+	stream->file = strdup(file);
+	set_format(kshark_ctx, stream, file);
+
+	switch (stream->format) {
+	case KS_TEP_DATA:
+		return kshark_tep_init_input(stream, file);
+
+	default:
+		return -ENODATA;
+	}
+}
+
  /**
   * @brief Get the Data stream object having given Id.
   *
@@ -313,45 +333,76 @@ int *kshark_all_streams(struct kshark_context *kshark_ctx)
  	return ids;
  }
 
+static void kshark_stream_close(struct kshark_data_stream *stream)
+{
+	struct kshark_context *kshark_ctx = NULL;
+
+	if (!stream || !kshark_instance(&kshark_ctx))
+		return;
+
+	/*
+	 * All filters are file specific. Make sure that all Process Ids and
+	 * Event Ids from this file are not going to be used with another file.
+	 */
+	kshark_hash_id_clear(stream->show_task_filter);
+	kshark_hash_id_clear(stream->hide_task_filter);
+	kshark_hash_id_clear(stream->show_event_filter);
+	kshark_hash_id_clear(stream->hide_event_filter);
+	kshark_hash_id_clear(stream->show_cpu_filter);
+	kshark_hash_id_clear(stream->hide_cpu_filter);
+
+	switch (stream->format) {
+	case KS_TEP_DATA:
+		kshark_tep_close_interface(stream);
+		break;
+
+	default:
+		break;
+	}
+
+	pthread_mutex_destroy(&stream->input_mutex);
+}
+
  /**
   * @brief Close the trace data file and free the trace data handle.
   *
   * @param kshark_ctx: Input location for the session context pointer.
+ * @param sd: Data stream identifier.
   */
-void kshark_close(struct kshark_context *kshark_ctx)
+void kshark_close(struct kshark_context *kshark_ctx, int sd)
  {
-	if (!kshark_ctx || !kshark_ctx->handle)
+	struct kshark_data_stream *stream;
+
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
  		return;
 
-	/*
-	 * All filters are file specific. Make sure that the Pids and Event Ids
-	 * from this file are not going to be used with another file.
-	 */
-	tracecmd_filter_id_clear(kshark_ctx->show_task_filter);
-	tracecmd_filter_id_clear(kshark_ctx->hide_task_filter);
-	tracecmd_filter_id_clear(kshark_ctx->show_event_filter);
-	tracecmd_filter_id_clear(kshark_ctx->hide_event_filter);
-	tracecmd_filter_id_clear(kshark_ctx->show_cpu_filter);
-	tracecmd_filter_id_clear(kshark_ctx->hide_cpu_filter);

src/libkshark.c int *kshark_all_streams(struct kshark_context *kshark_ctx) · line 383

Quoted 10 lines
src/libkshark.c +4 -6
-
-	if (kshark_ctx->advanced_event_filter) {
-		tep_filter_reset(kshark_ctx->advanced_event_filter);
-		tep_filter_free(kshark_ctx->advanced_event_filter);
-		kshark_ctx->advanced_event_filter = NULL;
-	}
+	kshark_stream_close(stream);
+	kshark_stream_free(stream);
+	kshark_ctx->stream[sd] = NULL;
+	kshark_ctx->n_streams--;

So, if you have multiple streams, and you close one that's not the last, and then add a new one, this will cause the new one to be overwritten.

Quoted 4 lines

As add_stream has:

	kshark_ctx->stream[kshark_ctx->n_streams++] = stream;

I see the problem. This is definitely wrong.

What if in addition to "n_streams" I add another counter called "last_stream_added" and initialize this counter to -1?

Then I can add streams like this:

	kshark_ctx->stream[++kshark_ctx->last_stream_added] = stream;
	++kshark_ctx->n_streams;
Quoted 18 lines

You may want to do instead:

	kshark_ctx->stream[sd] = NULL;
	while (!kshark->streams[kshark_ctx->n_streams - 1])
		kshark_ctx->n_streams--;

You can also add a free store, where you store an index in the stream[sd] field of the next free item. Then for adding you have:

	if (kshark_ctx->free >= 0) {
		sd = kshark_ctx->free;
		kshark_ctx->free = (int)kshark_ctx->stream[kshark_ctx->free];
	} else {
		sd = kshark_ctx->n_streams++;
	}

and on freeing:

Quoted 8 lines
	kshark_ctx->streams[sd] = (void *)kshark_ctx->free;
	kshark_ctx->free = sd;

Just need to initialize kshark_ctx->free to -1.

And never decrement n_streams. If you do any loops over the stream, you could verify that it is a real stream by:

I really need n_streams to show the true number of active streams because the widgets are using this a lot.

The way I loop over the active streams is the following:

	int *stream_ids = kshark_all_streams(kshark_ctx);
	for (i = 0; i < kshark_ctx->n_streams; ++i) {
		stream = kshark_ctx->stream[stream_ids[i]];

		....
	}

	free(stream_ids);

and with the addition of "last_stream_added" kshark_all_streams() will look like this:

int *kshark_all_streams(struct kshark_context *kshark_ctx)

{
	int *ids, i, count = 0;

	ids = calloc(kshark_ctx->n_streams, (sizeof(*ids)));
	if (!ids)
		return NULL;

	for (i = 0; i <= kshark_ctx->last_stream_added; ++i)
		if (kshark_ctx->stream[i])
			ids[count++] = i;

	return ids;
}

What do you think?

Thanks a lot!
Yordan

Quoted 44 lines
	if ((unsigned long)kshark_ctx->streams[sd] > kshark_ctx->n_streams)
		/* pointer to a stream */
	else
		/* a free item. */

Places like kshark_close() would need the above (if doing a free store).

-- Steve

Quoted 34 lines
src/libkshark.c +18 -9
+}
+
+/**
+ * @brief Close all currently open trace data file and free the trace data handle.
+ *
+ * @param kshark_ctx: Input location for the session context pointer.
+ */
+void kshark_close_all(struct kshark_context *kshark_ctx)
+{
+	int i, *stream_ids, n_streams;
+
+	stream_ids = kshark_all_streams(kshark_ctx);
 
  	/*
-	 * All data collections are file specific. Make sure that collections
-	 * from this file are not going to be used with another file.
+	 * Get a copy of shark_ctx->n_streams befor you start closing. Be aware
+	 * that kshark_close() will decrement shark_ctx->n_streams.
  	 */
-	kshark_free_collection_list(kshark_ctx->collections);
-	kshark_ctx->collections = NULL;
-
-	tracecmd_close(kshark_ctx->handle);
-	kshark_ctx->handle = NULL;
-	kshark_ctx->pevent = NULL;
+	n_streams = kshark_ctx->n_streams;
+	for (i = 0; i < n_streams; ++i)
+		kshark_close(kshark_ctx, stream_ids[i]);
 
-	pthread_mutex_destroy(&kshark_ctx->input_mutex);
+	free(stream_ids);
  }
 
  /**
#28 lore raw

Re: [PATCH v2 10/20] kernel-shark: Start using data streams

Quoted 11 lines · On Thu, 5 Nov 2020 16:58:51 +0200 "Yordan Karadzhov (VMware)"…

I see the problem. This is definitely wrong.

What if in addition to "n_streams" I add another counter called "last_stream_added" and initialize this counter to -1?

Then I can add streams like this:

	kshark_ctx->stream[++kshark_ctx->last_stream_added] = stream;
	++kshark_ctx->n_streams;
Quoted 1 line

You may want to do instead:

I'm thinking of doing something like this:

struct kshark_ctx {
	[..]
	unsigned int free_stream;
	[..]
};
	if (kshark_ctx->free_stream >= kshark_ctx->n_streams) {
		kshark_ctx->stream[++kshark_ctx->n_streams] = stream;
		kshark_ctx->free_stream = kshark_ctx->n_streams;

/* BTW, the stream array should be allocated to the n_streams, and
reallocated when it grows. I don't think we want a huge stream array to handle all the bits when not used. */

	} else {
		int new_stream = kshark_ctx->free_stream;

		kshark_ctx->free_stream = kshark_index(kshark_ctx->stream[new_stream]);
		kshark_ctx->stream[new_stream] = stream;
	}

For freeing (index i):

	kshark_ctx->stream[i] = kshark_ptr(kshark_ctx->free_stream);
	kshark_ctx->free_stream = i;

We could define the following (note, I just used these names for the functions, they could be named something else):

#define KSHARK_INDEX_MASK ((1 << NR_OF_BITS_FOR_STREAM) - 1)

#define KSHARK_INVALID_STREAM (~((1UL << NR_OF_BITS_FOR_STREAM) - 1))

static inline int kshark_index(void *ptr)

{
	unsigned long index = (unsigned long)ptr;

	return (int)(index & KSHARK_INDEX_MASK);
}

static inline void *kshark_ptr(unsigned int index)

{
	unsigned long p;

	p = KSHARK_INVALID_STREAM | index;

	return (void *)p;
}

The KSHARK_INVALID_STREAM and KSHARK_INDEX_MASK, would allow us to do something like this if we wanted to loop through all streams:

static inline bool kshark_is_valid_stream(void *ptr)

{
	unsigned long p = (unsigned long)ptr;

	return (p & KSHARK_INVALID_STREAM) == KSHARK_INVALID_STREAM;
}

The above works because the address of setting all those bits, would put the address into the kernel space (illegal user space address).

	for (i = 0; i < kshark_ctx->n_streams; i++) {
		if (!kshark_is_valid_stream(kshark_ctx->stream[i]))
			continue;
		/* process valid stream */
	}

-- Steve

#29 lore raw

Re: [PATCH v2 10/20] kernel-shark: Start using data streams

Quoted 84 lines · Steven Rostedt
Quoted 11 lines · On Thu, 5 Nov 2020 16:58:51 +0200 "Yordan Karadzhov (VMware)"…

I see the problem. This is definitely wrong.

What if in addition to "n_streams" I add another counter called "last_stream_added" and initialize this counter to -1?

Then I can add streams like this:

	kshark_ctx->stream[++kshark_ctx->last_stream_added] = stream;
	++kshark_ctx->n_streams;
Quoted 1 line

You may want to do instead:

I'm thinking of doing something like this:

struct kshark_ctx {
	[..]
	unsigned int free_stream;
	[..]
};
	if (kshark_ctx->free_stream >= kshark_ctx->n_streams) {
		kshark_ctx->stream[++kshark_ctx->n_streams] = stream;
		kshark_ctx->free_stream = kshark_ctx->n_streams;

/* BTW, the stream array should be allocated to the n_streams, and

    reallocated when it grows.  I don't think we want a huge stream array to
    handle all the bits when not used. */

	} else {
		int new_stream = kshark_ctx->free_stream;

		kshark_ctx->free_stream = kshark_index(kshark_ctx->stream[new_stream]);
		kshark_ctx->stream[new_stream] = stream;
	}

For freeing (index i):

	kshark_ctx->stream[i] = kshark_ptr(kshark_ctx->free_stream);
	kshark_ctx->free_stream = i;

We could define the following (note, I just used these names for the functions, they could be named something else):

#define KSHARK_INDEX_MASK ((1 << NR_OF_BITS_FOR_STREAM) - 1)

#define KSHARK_INVALID_STREAM (~((1UL << NR_OF_BITS_FOR_STREAM) - 1))

static inline int kshark_index(void *ptr)

{
	unsigned long index = (unsigned long)ptr;

	return (int)(index & KSHARK_INDEX_MASK);
}

static inline void *kshark_ptr(unsigned int index)

{
	unsigned long p;

	p = KSHARK_INVALID_STREAM | index;

	return (void *)p;
}

The KSHARK_INVALID_STREAM and KSHARK_INDEX_MASK, would allow us to do something like this if we wanted to loop through all streams:

static inline bool kshark_is_valid_stream(void *ptr)

{
	unsigned long p = (unsigned long)ptr;

	return (p & KSHARK_INVALID_STREAM) == KSHARK_INVALID_STREAM;
}

The above works because the address of setting all those bits, would put the address into the kernel space (illegal user space address).

Quoted 6 lines
	for (i = 0; i < kshark_ctx->n_streams; i++) {
		if (!kshark_is_valid_stream(kshark_ctx->stream[i]))
			continue;
		/* process valid stream */
	}

Hi Steven,

I am not sure I understand correctly your pseudo-code, so please correct me if my interpretation is wrong.

In the normal case when a new stream is added the corresponding object will be allocated and added to the array of pointers. Later if a stream is removed, instead of freeing the memory we will just manipulate it pointer so that it point to nowhere and this manipulation can be detected by the kshark_is_valid_stream(). Now if we want to add stream again, we will take the broken pointer, will restore its original value and will reuse the object without a new allocations.

And at the very end we will have to free all pointers (original or manipulated).

Is this what you are suggesting?

Thanks!
Y.

Quoted 2 lines

-- Steve

#30 lore raw

Re: [PATCH v2 10/20] kernel-shark: Start using data streams

Quoted 4 lines · On Fri, 6 Nov 2020 16:31:58 +0200 "Yordan Karadzhov (VMware)"…

Hi Steven,

I am not sure I understand correctly your pseudo-code, so please correct me if my interpretation is wrong.

Note, it's not really pseudo code. I didn't compile it, but it should work mostly unmodified.

Quoted 9 lines

In the normal case when a new stream is added the corresponding object will be allocated and added to the array of pointers. Later if a stream is removed, instead of freeing the memory we will just manipulate it pointer so that it point to nowhere and this manipulation can be detected by the kshark_is_valid_stream(). Now if we want to add stream again, we will take the broken pointer, will restore its original value and will reuse the object without a new allocations.

Quoted 4 lines

And at the very end we will have to free all pointers (original or manipulated).

Is this what you are suggesting?

Hmm, let me explain it slightly different, as nothing is "restored".

We change the value of the array from pointer to an index. For ease of explanation, let's consider this a 32bit machine and we allow 256 streams (one byte). That is:

sizeof(long) == sizeof(void *) == 4

#define NR_OF_BITS_FOR_STREAM 8

#define KSHARK_INDEX_MASK 0x000000FF
#define KSHARK_INVALID_STREAM 0xFFFFFF00

Lets say we add 4 stream in a row. Each one will detect that free_stream is equal to n_streams, and just append them to the end of the array (reallocating the array if necessary). We end up with:

free_stream = 4
n_streams = 4

streams:

  0: 0x07123010   -> stream 1
  1: 0x07123020   -> stream 2
  2: 0x07123030   -> stream 3
  3: 0x07123040   -> stream 4

Now we free stream 3 (at location 2):

free_streams = 2
n_streams = 4

streams:
0: 0x07123010 -> stream 1
1: 0x07123020 -> stream 2
2: 0xFFFFFF04 == (KSHARK_INVALID_MASK | orig:free_streams)
3: 0x07123040 -> stream 4

Now we free stream 1 (at location 0):

free_streams = 0
n_streams = 4

streams:
0: 0xFFFFFF02 == (KSHARK_INVALID_MASK | orig:free_streams)
1: 0x07123020 -> stream 2
2: 0xFFFFFF04 == (KSHARK_INVALID_MASK | 4)
3: 0x07123040 -> stream 4

Now lets add stream 5:

free_streams = 2 == (streams[orig:free_streams] & KSHARK_INDEX_MASK) n_streams = 4

streams:
0: 0x07123050 -> stream 5
1: 0x07123020 -> stream 2
2: 0xFFFFFF04
3: 0x07123040 -> stream 4

We are just making a link list of free pointers within the array of streams. This is basically exactly how memory management systems work. The free memory list is stored inside the memory itself that is being allocated.

The 0xFFFFFF is so that if we want to loop over streams, we can skip the free slots, by checking: streams[i] & 0xFFFFFF00 != 0xFFFFFF00

Note, the free slots do not point any memory location. Think of the items in the stream array as:

union {
	struct kshark_stream 	*stream;
	unsigned long		index;
};

You can differentiate without using typecasts with:

	if (streams[i].index & 0xFFFFFF00 == 0xFFFFFF00)
		index = streams[i].index & 0xFF;
	else
		stream = streams[i].stream;

Does that make more sense?

-- Steve

#31 lore raw

Re: [PATCH v2 10/20] kernel-shark: Start using data streams

Quoted 105 lines · Steven Rostedt
Quoted 4 lines · On Fri, 6 Nov 2020 16:31:58 +0200 "Yordan Karadzhov (VMware)"…

Hi Steven,

I am not sure I understand correctly your pseudo-code, so please correct me if my interpretation is wrong.

Note, it's not really pseudo code. I didn't compile it, but it should work mostly unmodified.

Quoted 9 lines

In the normal case when a new stream is added the corresponding object will be allocated and added to the array of pointers. Later if a stream is removed, instead of freeing the memory we will just manipulate it pointer so that it point to nowhere and this manipulation can be detected by the kshark_is_valid_stream(). Now if we want to add stream again, we will take the broken pointer, will restore its original value and will reuse the object without a new allocations.

Quoted 4 lines

And at the very end we will have to free all pointers (original or manipulated).

Is this what you are suggesting?

Hmm, let me explain it slightly different, as nothing is "restored".

We change the value of the array from pointer to an index. For ease of explanation, let's consider this a 32bit machine and we allow 256 streams (one byte). That is:

sizeof(long) == sizeof(void *) == 4

#define NR_OF_BITS_FOR_STREAM 8

#define KSHARK_INDEX_MASK 0x000000FF
#define KSHARK_INVALID_STREAM 0xFFFFFF00

Lets say we add 4 stream in a row. Each one will detect that free_stream is equal to n_streams, and just append them to the end of the array (reallocating the array if necessary). We end up with:

free_stream = 4
n_streams = 4

streams:

   0: 0x07123010   -> stream 1
   1: 0x07123020   -> stream 2
   2: 0x07123030   -> stream 3
   3: 0x07123040   -> stream 4

Now we free stream 3 (at location 2):

free_streams = 2
n_streams = 4

streams:

   0: 0x07123010   -> stream 1
   1: 0x07123020   -> stream 2
   2: 0xFFFFFF04	== (KSHARK_INVALID_MASK | orig:free_streams)
   3: 0x07123040   -> stream 4

Now we free stream 1 (at location 0):

free_streams = 0
n_streams = 4

streams:

   0: 0xFFFFFF02	== (KSHARK_INVALID_MASK | orig:free_streams)
   1: 0x07123020   -> stream 2
   2: 0xFFFFFF04	== (KSHARK_INVALID_MASK | 4)
   3: 0x07123040   -> stream 4

Now lets add stream 5:

free_streams = 2 == (streams[orig:free_streams] & KSHARK_INDEX_MASK) n_streams = 4

streams:

   0: 0x07123050   -> stream 5
   1: 0x07123020   -> stream 2
   2: 0xFFFFFF04
   3: 0x07123040   -> stream 4

We are just making a link list of free pointers within the array of streams. This is basically exactly how memory management systems work. The free memory list is stored inside the memory itself that is being allocated.

The 0xFFFFFF is so that if we want to loop over streams, we can skip the free slots, by checking: streams[i] & 0xFFFFFF00 != 0xFFFFFF00

Note, the free slots do not point any memory location. Think of the items in the stream array as:

union {
	struct kshark_stream 	*stream;
	unsigned long		index;
};
Quoted 9 lines

You can differentiate without using typecasts with:

	if (streams[i].index & 0xFFFFFF00 == 0xFFFFFF00)
		index = streams[i].index & 0xFF;
	else
		stream = streams[i].stream;

Does that make more sense?

Yes, it makes a lot of sense. Thanks a lot!
I am implementing it. It requires to do some rebasing in the successive patches (the GUI code). I am almost ready to send v3.

thanks!
Yordan

Quoted 2 lines

-- Steve

#32 lore raw

PATCH v2 11/20 kernel-shark: Remove dead code

In reply to Yordan Karadzhov (VMware)

Clean up the old readout code that is no longer used after switching to the implementation provided by the Data stream interface.

---
 src/libkshark.c | 807 +-----------------------------------------------
 src/libkshark.h |  77 +----
 2 files changed, 2 insertions(+), 882 deletions(-)
src/libkshark.c +1 -806
index 6eff53e5..e363395a 100644
@@ -6,7 +6,7 @@
 
  /**
  *  @file    libkshark.c
- *  @brief   API for processing of FTRACE (trace-cmd) data.
+ *  @brief   API for processing of tracing data.
  */
 
 /** Use GNU C Library. */
@@ -21,8 +21,6 @@
 #include "libkshark.h"
 #include "libkshark-tepdata.h"
 
-static __thread struct trace_seq seq;
-
 static struct kshark_context *kshark_context_handler = NULL;
 
 static bool kshark_default_context(struct kshark_context **context)
@@ -62,14 +60,6 @@ static bool kshark_default_context(struct kshark_context **context)
 	return true;
 }
 
-static bool init_thread_seq(void)
-{
-	if (!seq.buffer)
-		trace_seq_init(&seq);
-
-	return seq.buffer != NULL;
-}
-
 /**
  * @brief Initialize a kshark session. This function must be called before
  *	  calling any other kshark function. If the session has been
@@ -106,9 +96,6 @@ bool kshark_instance(struct kshark_context **kshark_ctx)
 		}
 	}
 
-	if (!init_thread_seq())
-		return false;
-
 	return true;
 }
 
@@ -487,42 +474,6 @@ char *kshark_event_from_id(int sd, int event_id)
 	return stream->interface.get_event_name(stream, &e);
 }
 
-static struct kshark_task_list *
-kshark_find_task(struct kshark_context *kshark_ctx, uint32_t key, int pid)
-{
-	struct kshark_task_list *list;
-
-	for (list = kshark_ctx->tasks[key]; list; list = list->next) {
-		if (list->pid == pid)
-			return list;
-	}
-
-	return NULL;
-}
-
-static struct kshark_task_list *
-kshark_add_task(struct kshark_context *kshark_ctx, int pid)
-{
-	struct kshark_task_list *list;
-	uint32_t key;
-
-	key = tracecmd_quick_hash(pid, KS_TASK_HASH_SHIFT);
-
-	list = kshark_find_task(kshark_ctx, key, pid);
-	if (list)
-		return list;
-
-	list = malloc(sizeof(*list));
-	if (!list)
-		return NULL;
-
-	list->pid = pid;
-	list->next = kshark_ctx->tasks[key];
-	kshark_ctx->tasks[key] = list;
-
-	return list;
-}
-
 /**
  * @brief Get an array containing the Process Ids of all tasks presented in
  *	  the loaded trace data file.
@@ -751,400 +702,6 @@ void kshark_clear_all_filters(struct kshark_context *kshark_ctx,
 		set_all_visible(&data[i]->visible);
 }
 
-static void kshark_set_entry_values(struct kshark_context *kshark_ctx,
-				    struct tep_record *record,
-				    struct kshark_entry *entry)
-{
-	/* Offset of the record */
-	entry->offset = record->offset;
-
-	/* CPU Id of the record */
-	entry->cpu = record->cpu;
-
-	/* Time stamp of the record */
-	entry->ts = record->ts;
-
-	/* Event Id of the record */
-	entry->event_id = tep_data_type(kshark_ctx->pevent, record);
-
-	/*
-	 * Is visible mask. This default value means that the entry
-	 * is visible everywhere.
-	 */
-	entry->visible = 0xFF;
-
-	/* Process Id of the record */
-	entry->pid = tep_data_pid(kshark_ctx->pevent, record);
-}
-
-/** Prior time offset of the "missed_events" entry. */
-#define ME_ENTRY_TIME_SHIFT	10
-
-static void missed_events_action(struct kshark_context *kshark_ctx,
-				 struct tep_record *record,
-				 struct kshark_entry *entry)
-{
-	/*
-	 * Use the offset field of the entry to store the number of missed
-	 * events.
-	 */
-	entry->offset = record->missed_events;
-
-	entry->cpu = record->cpu;
-
-	/*
-	 * Position the "missed_events" entry a bit before (in time)
-	 * the original record.
-	 */
-	entry->ts = record->ts - ME_ENTRY_TIME_SHIFT;
-
-	/* All custom entries must have negative event Identifiers. */
-	entry->event_id = KS_EVENT_OVERFLOW;
-
-	entry->visible = 0xFF;
-
-	entry->pid = tep_data_pid(kshark_ctx->pevent, record);
-}
-
-static const char* missed_events_dump(struct kshark_context *kshark_ctx,
-				      const struct kshark_entry *entry,
-				      bool get_info)
-{
-	int size = 0;
-	static char *buffer;
-
-	if (get_info)
-		size = asprintf(&buffer, "missed_events=%i", (int) entry->offset);
-	else
-		size = asprintf(&buffer, "missed_events");
-	if (size > 0)
-		return buffer;
-
-	return NULL;
-}
-
-/**
- * rec_list is used to pass the data to the load functions.
- * The rec_list will contain the list of entries from the source,
- * and will be a link list of per CPU entries.
- */
-struct rec_list {
-	union {
-		/* Used by kshark_load_data_records */
-		struct {
-			/** next pointer, matches entry->next */
-			struct rec_list		*next;
-			/** pointer to the raw record data */
-			struct tep_record	*rec;
-		};
-		/** entry - Used for kshark_load_data_entries() */
-		struct kshark_entry		entry;
-	};
-};
-
-/**
- * rec_type defines what type of rec_list is being used.
- */
-enum rec_type {
-	REC_RECORD,
-	REC_ENTRY,
-};
-
-static void free_rec_list(struct rec_list **rec_list, int n_cpus,
-			  enum rec_type type)
-{
-	struct rec_list *temp_rec;
-	int cpu;
-
-	for (cpu = 0; cpu < n_cpus; ++cpu) {
-		while (rec_list[cpu]) {
-			temp_rec = rec_list[cpu];
-			rec_list[cpu] = temp_rec->next;
-			if (type == REC_RECORD)
-				free_record(temp_rec->rec);
-			free(temp_rec);
-		}
-	}
-	free(rec_list);
-}
-
-static ssize_t get_records(struct kshark_context *kshark_ctx,
-			   struct rec_list ***rec_list, enum rec_type type)
-{
-	struct kshark_event_handler *evt_handler;
-	struct tep_event_filter *adv_filter;
-	struct kshark_task_list *task;
-	struct tep_record *rec;
-	struct rec_list **temp_next;
-	struct rec_list **cpu_list;
-	struct rec_list *temp_rec;
-	size_t count, total = 0;
-	int n_cpus;
-	int pid;
-	int cpu;
-
-	n_cpus = tracecmd_cpus(kshark_ctx->handle);
-	cpu_list = calloc(n_cpus, sizeof(*cpu_list));
-	if (!cpu_list)
-		return -ENOMEM;
-
-	/* Just to shorten the name */
-	if (type == REC_ENTRY)
-		adv_filter = kshark_ctx->advanced_event_filter;
-
-	for (cpu = 0; cpu < n_cpus; ++cpu) {
-		count = 0;
-		cpu_list[cpu] = NULL;
-		temp_next = &cpu_list[cpu];
-
-		rec = tracecmd_read_cpu_first(kshark_ctx->handle, cpu);
-		while (rec) {
-			*temp_next = temp_rec = calloc(1, sizeof(*temp_rec));
-			if (!temp_rec)
-				goto fail;
-
-			temp_rec->next = NULL;
-
-			switch (type) {
-			case REC_RECORD:
-				temp_rec->rec = rec;
-				pid = tep_data_pid(kshark_ctx->pevent, rec);
-				break;
-			case REC_ENTRY: {
-				struct kshark_entry *entry;
-				int ret;
-
-				if (rec->missed_events) {
-					/*
-					 * Insert a custom "missed_events" entry just
-					 * befor this record.
-					 */
-					entry = &temp_rec->entry;
-					missed_events_action(kshark_ctx, rec, entry);
-
-					temp_next = &temp_rec->next;
-					++count;
-
-					/* Now allocate a new rec_list node and comtinue. */
-					*temp_next = temp_rec = calloc(1, sizeof(*temp_rec));
-				}
-
-				entry = &temp_rec->entry;
-				kshark_set_entry_values(kshark_ctx, rec, entry);
-
-				/* Execute all plugin-provided actions (if any). */
-				evt_handler = kshark_ctx->event_handlers;
-				while ((evt_handler = kshark_find_event_handler(evt_handler,
-										entry->event_id))) {
-					evt_handler->event_func(kshark_ctx, rec, entry);
-					evt_handler = evt_handler->next;
-					entry->visible &= ~KS_PLUGIN_UNTOUCHED_MASK;
-				}
-
-				pid = entry->pid;
-				/* Apply event filtering. */
-				ret = FILTER_MATCH;
-				if (adv_filter->filters)
-					ret = tep_filter_match(adv_filter, rec);
-
-				if (!kshark_show_event(kshark_ctx, entry->event_id) ||
-				    ret != FILTER_MATCH) {
-					unset_event_filter_flag(kshark_ctx, entry);
-				}
-
-				/* Apply CPU filtering. */
-				if (!kshark_show_cpu(kshark_ctx, entry->pid)) {
-					entry->visible &= ~kshark_ctx->filter_mask;
-				}
-
-				/* Apply task filtering. */
-				if (!kshark_show_task(kshark_ctx, entry->pid)) {
-					entry->visible &= ~kshark_ctx->filter_mask;
-				}
-				free_record(rec);
-				break;
-			} /* REC_ENTRY */
-			}
-
-			task = kshark_add_task(kshark_ctx, pid);
-			if (!task) {
-				free_record(rec);
-				goto fail;
-			}
-
-			temp_next = &temp_rec->next;
-
-			++count;
-			rec = tracecmd_read_data(kshark_ctx->handle, cpu);
-		}
-
-		total += count;
-	}
-
-	*rec_list = cpu_list;
-	return total;
-
- fail:
-	free_rec_list(cpu_list, n_cpus, type);
-	return -ENOMEM;
-}
-
-static int pick_next_cpu(struct rec_list **rec_list, int n_cpus,
-			 enum rec_type type)
-{
-	uint64_t ts = 0;
-	uint64_t rec_ts;
-	int next_cpu = -1;
-	int cpu;
-
-	for (cpu = 0; cpu < n_cpus; ++cpu) {
-		if (!rec_list[cpu])
-			continue;
-
-		switch (type) {
-		case REC_RECORD:
-			rec_ts = rec_list[cpu]->rec->ts;
-			break;
-		case REC_ENTRY:
-			rec_ts = rec_list[cpu]->entry.ts;
-			break;
-		}
-		if (!ts || rec_ts < ts) {
-			ts = rec_ts;
-			next_cpu = cpu;
-		}
-	}
-
-	return next_cpu;
-}
-
-/**
- * @brief Load the content of the trace data file into an array of
- *	  kshark_entries. This function provides an abstraction of the
- *	  entries from the raw data that is read, however the "latency"
- *	  and the "info" fields can be accessed only via the offset
- *	  into the file. This makes the access to these two fields much
- *	  slower.
- *	  If one or more filters are set, the "visible" fields of each entry
- *	  is updated according to the criteria provided by the filters. The
- *	  field "filter_mask" of the session's context is used to control the
- *	  level of visibility/invisibility of the filtered entries.
- *
- * @param kshark_ctx: Input location for context pointer.
- * @param data_rows: Output location for the trace data. The user is
- *		     responsible for freeing the elements of the outputted
- *		     array.
- *
- * @returns The size of the outputted data in the case of success, or a
- *	    negative error code on failure.
- */
-ssize_t kshark_load_data_entries(struct kshark_context *kshark_ctx,
-				 struct kshark_entry ***data_rows)
-{
-	struct kshark_entry **rows;
-	struct rec_list **rec_list;
-	enum rec_type type = REC_ENTRY;
-	ssize_t count, total = 0;
-	int n_cpus;
-
-	if (*data_rows)
-		free(*data_rows);
-
-	total = get_records(kshark_ctx, &rec_list, type);
-	if (total < 0)
-		goto fail;
-
-	n_cpus = tracecmd_cpus(kshark_ctx->handle);
-
-	rows = calloc(total, sizeof(struct kshark_entry *));
-	if (!rows)
-		goto fail_free;
-
-	for (count = 0; count < total; count++) {
-		int next_cpu;
-
-		next_cpu = pick_next_cpu(rec_list, n_cpus, type);
-
-		if (next_cpu >= 0) {
-			rows[count] = &rec_list[next_cpu]->entry;
-			rec_list[next_cpu] = rec_list[next_cpu]->next;
-		}
-	}
-
-	free_rec_list(rec_list, n_cpus, type);
-	*data_rows = rows;
-	return total;
-
- fail_free:
-	free_rec_list(rec_list, n_cpus, type);
-
- fail:
-	fprintf(stderr, "Failed to allocate memory during data loading.\n");
-	return -ENOMEM;
-}
-
-/**
- * @brief Load the content of the trace data file into an array of
- *	  tep_records. Use this function only if you need fast access
- *	  to all fields of the record.
- *
- * @param kshark_ctx: Input location for the session context pointer.
- * @param data_rows: Output location for the trace data. Use free_record()
- *	 	     to free the elements of the outputted array.
- *
- * @returns The size of the outputted data in the case of success, or a
- *	    negative error code on failure.
- */
-ssize_t kshark_load_data_records(struct kshark_context *kshark_ctx,
-				 struct tep_record ***data_rows)
-{
-	struct tep_record **rows;
-	struct tep_record *rec;
-	struct rec_list **rec_list;
-	struct rec_list *temp_rec;
-	enum rec_type type = REC_RECORD;
-	ssize_t count, total = 0;
-	int n_cpus;
-
-	total = get_records(kshark_ctx, &rec_list, type);
-	if (total < 0)
-		goto fail;
-
-	n_cpus = tracecmd_cpus(kshark_ctx->handle);
-
-	rows = calloc(total, sizeof(struct tep_record *));
-	if (!rows)
-		goto fail_free;
-
-	for (count = 0; count < total; count++) {
-		int next_cpu;
-
-		next_cpu = pick_next_cpu(rec_list, n_cpus, type);
-
-		if (next_cpu >= 0) {
-			rec = rec_list[next_cpu]->rec;
-			rows[count] = rec;
-
-			temp_rec = rec_list[next_cpu];
-			rec_list[next_cpu] = rec_list[next_cpu]->next;
-			free(temp_rec);
-			/* The record is still referenced in rows */
-		}
-	}
-
-	/* There should be no records left in rec_list */
-	free_rec_list(rec_list, n_cpus, type);
-	*data_rows = rows;
-	return total;
-
- fail_free:
-	free_rec_list(rec_list, n_cpus, type);
-
- fail:
-	fprintf(stderr, "Failed to allocate memory during data loading.\n");
-	return -ENOMEM;
-}
-
 static inline void free_ptr(void *ptr)
 {
 	if (ptr)
@@ -1216,294 +773,6 @@ bool kshark_data_matrix_alloc(size_t n_rows, int16_t **cpu_array,
 	return false;
 }
 
-static const char *get_latency(struct tep_handle *pe,
-			       struct tep_record *record)
-{
-	if (!record)
-		return NULL;
-
-	trace_seq_reset(&seq);
-	tep_print_event(pe, &seq, record, "%s", TEP_PRINT_LATENCY);
-	return seq.buffer;
-}
-
-static const char *get_info(struct tep_handle *pe,
-				   struct tep_record *record,
-				   struct tep_event *event)
-{
-	char *pos;
-
-	if (!record || !event)
-		return NULL;
-
-	trace_seq_reset(&seq);
-	tep_print_event(pe, &seq, record, "%s", TEP_PRINT_INFO);
-
-	/*
-	 * The event info string contains a trailing newline.
-	 * Remove this newline.
-	 */
-	if ((pos = strchr(seq.buffer, '\n')) != NULL)
-		*pos = '\0';
-
-	return seq.buffer;
-}
-
-/**
- * @brief This function allows for an easy access to the original value of the
- *	  Process Id as recorded in the tep_record object. The record is read
- *	  from the file only in the case of an entry being touched by a plugin.
- *	  Be aware that using the kshark_get_X_easy functions can be
- *	  inefficient if you need an access to more than one of the data fields
- *	  of the record.
- *
- * @param entry: Input location for the KernelShark entry.
- *
- * @returns The original value of the Process Id as recorded in the
- *	    tep_record object on success, otherwise negative error code.
- */
-int kshark_get_pid_easy(struct kshark_entry *entry)
-{
-	struct kshark_context *kshark_ctx = NULL;
-	struct tep_record *data;
-	int pid;
-
-	if (!kshark_instance(&kshark_ctx))
-		return -ENODEV;
-
-	if (entry->visible & KS_PLUGIN_UNTOUCHED_MASK) {
-		pid = entry->pid;
-	} else {
-		/*
-		 * The entry has been touched by a plugin callback function.
-		 * Because of this we do not trust the value of "entry->pid".
-		 *
-		 * Currently the data reading operations are not thread-safe.
-		 * Use a mutex to protect the access.
-		 */
-		pthread_mutex_lock(&kshark_ctx->input_mutex);
-
-		data = tracecmd_read_at(kshark_ctx->handle, entry->offset,
-					NULL);
-		pid = tep_data_pid(kshark_ctx->pevent, data);
-		free_record(data);
-
-		pthread_mutex_unlock(&kshark_ctx->input_mutex);
-	}
-
-	return pid;
-}
-
-/**
- * @brief This function allows for an easy access to the original value of the
- *	  Task name as recorded in the tep_record object. The record is read
- *	  from the file only in the case of an entry being touched by a plugin.
- *	  Be aware that using the kshark_get_X_easy functions can be
- *	  inefficient if you need an access to more than one of the data fields
- *	  of the record.
- *
- * @param entry: Input location for the KernelShark entry.
- *
- * @returns The original name of the task, retrieved from the Process Id
- *	    recorded in the tep_record object on success, otherwise NULL.
- */
-const char *kshark_get_task_easy(struct kshark_entry *entry)
-{
-	struct kshark_context *kshark_ctx = NULL;
-	int pid = kshark_get_pid_easy(entry);
-
-	if (pid < 0)
-		return NULL;
-
-	kshark_instance(&kshark_ctx);
-	return tep_data_comm_from_pid(kshark_ctx->pevent, pid);
-}
-
-/**
- * @brief This function allows for an easy access to the latency information
- *	  recorded in the tep_record object. The record is read from the file
- *	  using the offset field of kshark_entry. Be aware that using the
- *	  kshark_get_X_easy functions can be inefficient if you need an access
- *	  to more than one of the data fields of the record.
- *
- * @param entry: Input location for the KernelShark entry.
- *
- * @returns On success the function returns a string showing the latency
- *	    information, coded into 5 fields:
- *	    interrupts disabled, need rescheduling, hard/soft interrupt,
- *	    preempt count and lock depth. On failure it returns NULL.
- */
-const char *kshark_get_latency_easy(struct kshark_entry *entry)
-{
-	struct kshark_context *kshark_ctx = NULL;
-	struct tep_record *data;
-	const char *lat;
-
-	if (!kshark_instance(&kshark_ctx))
-		return NULL;
-
-	if (entry->event_id < 0)
-		return NULL;
-
-	/*
-	 * Currently the data reading operations are not thread-safe.
-	 * Use a mutex to protect the access.
-	 */
-	pthread_mutex_lock(&kshark_ctx->input_mutex);
-
-	data = tracecmd_read_at(kshark_ctx->handle, entry->offset, NULL);
-	lat = get_latency(kshark_ctx->pevent, data);
-	free_record(data);
-
-	pthread_mutex_unlock(&kshark_ctx->input_mutex);
-
-	return lat;
-}
-
-/**
- * @brief This function allows for an easy access to the original value of the
- *	  Event Id as recorded in the tep_record object. The record is read
- *	  from the file only in the case of an entry being touched by a plugin.
- *	  Be aware that using the kshark_get_X_easy functions can be
- *	  inefficient if you need an access to more than one of the data fields
- *	  of the record.
- *
- * @param entry: Input location for the KernelShark entry.
- *
- * @returns The original value of the Event Id as recorded in the
- *	    tep_record object on success, otherwise negative error code.
- */
-int kshark_get_event_id_easy(struct kshark_entry *entry)
-{
-	struct kshark_context *kshark_ctx = NULL;
-	struct tep_record *data;
-	int event_id;
-
-	if (!kshark_instance(&kshark_ctx))
-		return -ENODEV;
-
-	if (entry->visible & KS_PLUGIN_UNTOUCHED_MASK) {
-		event_id = entry->event_id;
-	} else {
-		/*
-		 * The entry has been touched by a plugin callback function.
-		 * Because of this we do not trust the value of
-		 * "entry->event_id".
-		 *
-		 * Currently the data reading operations are not thread-safe.
-		 * Use a mutex to protect the access.
-		 */
-		pthread_mutex_lock(&kshark_ctx->input_mutex);
-
-		data = tracecmd_read_at(kshark_ctx->handle, entry->offset,
-					NULL);
-		event_id = tep_data_type(kshark_ctx->pevent, data);
-		free_record(data);
-
-		pthread_mutex_unlock(&kshark_ctx->input_mutex);
-	}
-
-	return (event_id == -1)? -EFAULT : event_id;
-}
-
-/**
- * @brief This function allows for an easy access to the original name of the
- *	  trace event as recorded in the tep_record object. The record is read
- *	  from the file only in the case of an entry being touched by a plugin.
- *	  Be aware that using the kshark_get_X_easy functions can be
- *	  inefficient if you need an access to more than one of the data fields
- *	  of the record.
- *
- * @param entry: Input location for the KernelShark entry.
- *
- * @returns The mane of the trace event recorded in the tep_record object on
- *	    success, otherwise "[UNKNOWN EVENT]" or NULL.
- */
-const char *kshark_get_event_name_easy(struct kshark_entry *entry)
-{
-	struct kshark_context *kshark_ctx = NULL;
-	struct tep_event *event;
-
-	int event_id = kshark_get_event_id_easy(entry);
-	if (event_id == -EFAULT)
-		return NULL;
-
-	kshark_instance(&kshark_ctx);
-
-	if (event_id < 0) {
-		switch (event_id) {
-		case KS_EVENT_OVERFLOW:
-			return missed_events_dump(kshark_ctx, entry, false);
-		default:
-			return NULL;
-		}
-	}
-
-	/*
-	 * Currently the data reading operations are not thread-safe.
-	 * Use a mutex to protect the access.
-	 */
-	pthread_mutex_lock(&kshark_ctx->input_mutex);
-	event = tep_find_event(kshark_ctx->pevent, event_id);
-	pthread_mutex_unlock(&kshark_ctx->input_mutex);
-
-	if (event)
-		return event->name;
-
-	return "[UNKNOWN EVENT]";
-}
-
-/**
- * @brief This function allows for an easy access to the tep_record's info
- *	  streang. The record is read from the file using the offset field of
- *	  kshark_entry. Be aware that using the kshark_get_X_easy functions can
- *	  be inefficient if you need an access to more than one of the data
- *	  fields of the record.
- *
- * @param entry: Input location for the KernelShark entry.
- *
- * @returns A string showing the data output of the trace event on success,
- *	    otherwise NULL.
- */
-const char *kshark_get_info_easy(struct kshark_entry *entry)
-{
-	struct kshark_context *kshark_ctx = NULL;
-	struct tep_event *event;
-	struct tep_record *data;
-	const char *info = NULL;
-	int event_id;
-
-	if (!kshark_instance(&kshark_ctx))
-		return NULL;
-
-	if (entry->event_id < 0) {
-		switch (entry->event_id) {
-		case KS_EVENT_OVERFLOW:
-			return missed_events_dump(kshark_ctx, entry, true);
-		default:
-			return NULL;
-		}
-	}
-
-	/*
-	 * Currently the data reading operations are not thread-safe.
-	 * Use a mutex to protect the access.
-	 */
-	pthread_mutex_lock(&kshark_ctx->input_mutex);
-
-	data = tracecmd_read_at(kshark_ctx->handle, entry->offset, NULL);
-	event_id = tep_data_type(kshark_ctx->pevent, data);
-	event = tep_find_event(kshark_ctx->pevent, event_id);
-	if (event)
-		info = get_info(kshark_ctx->pevent, data, event);
-
-	free_record(data);
-
-	pthread_mutex_unlock(&kshark_ctx->input_mutex);
-
-	return info;
-}
-
 /**
  * @brief Convert the timestamp of the trace record (nanosecond precision) into
  *	  seconds and microseconds.
@@ -1520,44 +789,6 @@ void kshark_convert_nano(uint64_t time, uint64_t *sec, uint64_t *usec)
 	*usec = (time - s * 1000000000ULL) / 1000;
 }
 
-/**
- * @brief Dump into a string the content a custom entry. The function allocates
- *	  a null terminated string and returns a pointer to this string.
- *
- * @param kshark_ctx: Input location for the session context pointer.
- * @param entry: A Kernel Shark entry to be printed.
- * @param info_func:
- *
- * @returns The returned string contains a semicolon-separated list of data
- *	    fields. The user has to free the returned string.
- */
-char* kshark_dump_custom_entry(struct kshark_context *kshark_ctx,
-			       const struct kshark_entry *entry,
-			       kshark_custom_info_func info_func)
-{
-	const char *event_name, *task, *info;
-	char *entry_str;
-	int size = 0;
-
-	task = tep_data_comm_from_pid(kshark_ctx->pevent, entry->pid);
-	event_name = info_func(kshark_ctx, entry, false);
-	info = info_func(kshark_ctx, entry, true);
-
-	size = asprintf(&entry_str, "%" PRIu64 "; %s-%i; CPU %i; ; %s; %s",
-			entry->ts,
-			task,
-			entry->pid,
-			entry->cpu,
-			event_name,
-			info);
-
-	if (size > 0)
-		return entry_str;
-
-	return NULL;
-}
-
-
 /**
  * @brief Binary search inside a time-sorted array of kshark_entries.
  *
@@ -1594,42 +825,6 @@ ssize_t kshark_find_entry_by_time(uint64_t time,
 	return h;
 }
 
-/**
- * @brief Binary search inside a time-sorted array of tep_records.
- *
- * @param time: The value of time to search for.
- * @param data: Input location for the trace data.
- * @param l: Array index specifying the lower edge of the range to search in.
- * @param h: Array index specifying the upper edge of the range to search in.
- *
- * @returns On success, the first tep_record inside the range, having a
-	    timestamp equal or bigger than "time".
-	    If all entries inside the range have timestamps greater than "time"
-	    the function returns BSEARCH_ALL_GREATER (negative value).
-	    If all entries inside the range have timestamps smaller than "time"
-	    the function returns BSEARCH_ALL_SMALLER (negative value).
- */
-ssize_t kshark_find_record_by_time(uint64_t time,
-				   struct tep_record **data,
-				   size_t l, size_t h)
-{
-	size_t mid;
-
-	if (data[l]->ts > time)
-		return BSEARCH_ALL_GREATER;
-
-	if (data[h]->ts < time)
-		return BSEARCH_ALL_SMALLER;
-
-	/*
-	 * After executing the BSEARCH macro, "l" will be the index of the last
-	 * record having timestamp < time and "h" will be the index of the
-	 * first record having timestamp >= time.
-	 */
-	BSEARCH(h, l, data[mid]->ts < time);
-	return h;
-}
-
 /**
  * @brief Simple Pid matching function to be user for data requests.
  *
src/libkshark.h +1 -76

131 lines not on this page · raw message

Signature · 1 line
2.25.1
#33 lore raw

PATCH v2 12/20 kernel-shark: Redesign the plugin interface

In reply to Yordan Karadzhov (VMware)

This patch can be treated as an almost complete rewrite of the way the C API of KernelShark deals with plugins. First of all, the two pluggable user actions (one executed during data loading and another executed during plotting) have separate handlers. This way the user can register only one of the two callback functions, if this is what is needed.

The second substantial change is that instead of having a single interface for loading the plugin, we now have 3 different interfaces. The one that exists in version 1 of KernelShark is now renamed to Data Processing Interface (dpi).

The first new interface for loading can be used to register user provided implementation of the Data stream readout and is called Data Readout Interface (dri). Via this plugin loading interface the user can open trace data having an arbitrary format. In order to make this possible the user has to provide a plugin that contains an implementation of the data readout methods defined by the kshark_data_stream_interface and to register all those methods.

The second new plugin loading interface is called Control interface and can be used to provide the plugin with an access to the GUI's Main window object. Via this interface the plugin can became capable to modify the GUI. Such a modification for example can be to add new dialog to the menus or to change the state of the GUI (to select entry 1with the marker, to zoom, ...).

---
 src/libkshark-plugin.c | 597 ++++++++++++++++++++++++++++++++++-------
 src/libkshark-plugin.h | 279 ++++++++++++++-----
 src/libkshark.c        |  56 +++-
 src/libkshark.h        |  25 +-
 4 files changed, 791 insertions(+), 166 deletions(-)
src/libkshark-plugin.c +504 -93
index 4b21392a..8e5299cf 100644
@@ -1,7 +1,7 @@
 // SPDX-License-Identifier: LGPL-2.1
 
 /*
- * Copyright (C) 2017 VMware Inc, Yordan Karadzhov <[email protected]>
+ * Copyright (C) 2017 VMware Inc, Yordan Karadzhov (VMware) <[email protected]>
  */
 
  /**
@@ -13,8 +13,7 @@
 #ifndef _GNU_SOURCE
 /** Use GNU C Library. */
 #define _GNU_SOURCE
-
-#endif
+#endif // _GNU_SOURCE
 
 #include <stdio.h>
 #include <stdlib.h>
@@ -27,23 +26,36 @@
 #include "libkshark-plugin.h"
 #include "libkshark.h"
 
-static struct kshark_event_handler *
-gui_event_handler_alloc(int event_id,
-			kshark_plugin_event_handler_func evt_func,
-			kshark_plugin_draw_handler_func dw_func)
+static struct kshark_event_proc_handler *
+data_event_handler_alloc(int event_id,
+			 kshark_plugin_event_handler_func evt_func)
 {
-	struct kshark_event_handler *handler = malloc(sizeof(*handler));
+	struct kshark_event_proc_handler *handler = malloc(sizeof(*handler));
 
 	if (!handler) {
-		fprintf(stderr,
-			"failed to allocate memory for gui eventhandler");
+		fputs("failed to allocate memory for event handler\n", stderr);
 		return NULL;
 	}
 
 	handler->next = NULL;
 	handler->id = event_id;
 	handler->event_func = evt_func;
-	handler->draw_func = dw_func;
+
+	return handler;
+}
+
+static struct kshark_draw_handler *
+data_draw_handler_alloc(kshark_plugin_draw_handler_func draw_func)
+{
+	struct kshark_draw_handler *handler = malloc(sizeof(*handler));
+
+	if (!handler) {
+		fputs("failed to allocate memory for draw handler\n", stderr);
+		return NULL;
+	}
+
+	handler->next = NULL;
+	handler->draw_func = draw_func;
 
 	return handler;
 }
@@ -55,8 +67,8 @@ gui_event_handler_alloc(int event_id,
  * @param handlers: Input location for the Event handler list.
  * @param event_id: Event Id to search for.
  */
-struct kshark_event_handler *
-kshark_find_event_handler(struct kshark_event_handler *handlers, int event_id)
+struct kshark_event_proc_handler *
+kshark_find_event_handler(struct kshark_event_proc_handler *handlers, int event_id)
 {
 	for (; handlers; handlers = handlers->next)
 		if (handlers->id == event_id)
@@ -68,26 +80,25 @@ kshark_find_event_handler(struct kshark_event_handler *handlers, int event_id)
 /**
  * @brief Add new event handler to an existing list of handlers.
  *
- * @param handlers: Input location for the Event handler list.
+ * @param stream: Input location for a Trace data stream pointer.
  * @param event_id: Event Id.
  * @param evt_func: Input location for an Event action provided by the plugin.
- * @param dw_func: Input location for a Draw action provided by the plugin.
  *
  * @returns Zero on success, or a negative error code on failure.
  */
-int kshark_register_event_handler(struct kshark_event_handler **handlers,
+int kshark_register_event_handler(struct kshark_data_stream *stream,
 				  int event_id,
-				  kshark_plugin_event_handler_func evt_func,
-				  kshark_plugin_draw_handler_func dw_func)
+				  kshark_plugin_event_handler_func evt_func)
 {
-	struct kshark_event_handler *handler =
-		gui_event_handler_alloc(event_id, evt_func, dw_func);
+	struct kshark_event_proc_handler *handler =
+		data_event_handler_alloc(event_id, evt_func);
 
 	if(!handler)
 		return -ENOMEM;
 
-	handler->next = *handlers;
-	*handlers = handler;
+	handler->next = stream->event_handlers;
+	stream->event_handlers = handler;
+
 	return 0;
 }
 
@@ -95,23 +106,20 @@ int kshark_register_event_handler(struct kshark_event_handler **handlers,
  * @brief Search the list for a specific plugin handle. If such a plugin handle
  *	  exists, unregister (remove and free) this handle from the list.
  *
- * @param handlers: Input location for the Event handler list.
+ * @param stream: Input location for a Trace data stream pointer.
  * @param event_id: Event Id of the plugin handler to be unregistered.
- * @param evt_func: Event action function of the handler to be unregistered.
- * @param dw_func: Draw action function of the handler to be unregistered.
+ * @param evt_func: Event action function to be unregistered.
  */
-void kshark_unregister_event_handler(struct kshark_event_handler **handlers,
+void kshark_unregister_event_handler(struct kshark_data_stream *stream,
 				     int event_id,
-				     kshark_plugin_event_handler_func evt_func,
-				     kshark_plugin_draw_handler_func dw_func)
+				     kshark_plugin_event_handler_func evt_func)
 {
-	struct kshark_event_handler **last;
+	struct kshark_event_proc_handler **last;
 
-	for (last = handlers; *last; last = &(*last)->next) {
+	for (last = &stream->event_handlers; *last; last = &(*last)->next) {
 		if ((*last)->id == event_id &&
-		    (*last)->event_func == evt_func &&
-		    (*last)->draw_func == dw_func) {
-			struct kshark_event_handler *this_handler;
+		    (*last)->event_func == evt_func) {
+			struct kshark_event_proc_handler *this_handler;
 			this_handler = *last;
 			*last = this_handler->next;
 			free(this_handler);
@@ -126,9 +134,71 @@ void kshark_unregister_event_handler(struct kshark_event_handler **handlers,
  *
  * @param handlers: Input location for the Event handler list.
  */
-void kshark_free_event_handler_list(struct kshark_event_handler *handlers)
+void kshark_free_event_handler_list(struct kshark_event_proc_handler *handlers)
+{
+	struct kshark_event_proc_handler *last;
+
+	while (handlers) {
+		last = handlers;
+		handlers = handlers->next;
+		free(last);
+	}
+}
+
+/**
+ * @brief Add new event handler to an existing list of handlers.
+ *
+ * @param stream: Input location for a Trace data stream pointer.
+ * @param draw_func: Input location for a Draw action provided by the plugin.
+ *
+ * @returns Zero on success, or a negative error code on failure.
+ */
+int kshark_register_draw_handler(struct kshark_data_stream *stream,
+				 kshark_plugin_draw_handler_func draw_func)
+{
+	struct kshark_draw_handler *handler = data_draw_handler_alloc(draw_func);
+
+	if(!handler)
+		return -ENOMEM;
+
+	handler->next = stream->draw_handlers;
+	stream->draw_handlers = handler;
+
+	return 0;
+}
+
+/**
+ * @brief Search the list for a specific plugin handle. If such a plugin handle
+ *	  exists, unregister (remove and free) this handle from the list.
+ *
+ * @param stream: Input location for a Trace data stream pointer.
+ * @param draw_func: Draw action function to be unregistered.
+ */
+void kshark_unregister_draw_handler(struct kshark_data_stream *stream,
+				    kshark_plugin_draw_handler_func draw_func)
+{
+	struct kshark_draw_handler **last;
+
+	for (last = &stream->draw_handlers; *last; last = &(*last)->next) {
+		if ((*last)->draw_func == draw_func) {
+			struct kshark_draw_handler *this_handler;
+			this_handler = *last;
+			*last = this_handler->next;
+			free(this_handler);
+
+			return;
+		}
+	}
+}
+
+/**
+ * @brief Free all DRaw handlers in a given list.
+ *
+ * @param handlers: Input location for the Draw handler list.
+ */
+void kshark_free_draw_handler_list(struct kshark_draw_handler *handlers)
 {
-	struct kshark_event_handler *last;
+	struct kshark_draw_handler *last;
 
 	while (handlers) {
 		last = handlers;
@@ -139,96 +209,182 @@ void kshark_free_event_handler_list(struct kshark_event_handler *handlers)
 
 /**
  * @brief Allocate memory for a new plugin. Add this plugin to the list of
- *	  plugins used by the session.
+ *	  plugins.
  *
  * @param kshark_ctx: Input location for the session context pointer.
+ * @param name: The name of the plugin to register.
  * @param file: The plugin object file to load.
  *
- * @returns Zero on success, or a negative error code on failure.
+ * @returns The plugin object on success, or NULL on failure.
  */
-int kshark_register_plugin(struct kshark_context *kshark_ctx,
-			   const char *file)
+struct kshark_plugin_list *
+kshark_register_plugin(struct kshark_context *kshark_ctx,
+		       const char *name,
+		       const char *file)
 {
-	struct kshark_plugin_list *plugin = kshark_ctx->plugins;
+	kshark_plugin_load_func init_func, close_func;
+	kshark_check_data_func check_func;
+	struct kshark_plugin_list *plugin;
 	struct stat st;
 	int ret;
 
-	while (plugin) {
-		if (strcmp(plugin->file, file) == 0)
-			return -EEXIST;
+	printf("loading plugin \"%s\" from %s\n", name, file);
 
-		plugin = plugin->next;
+	plugin = kshark_find_plugin(kshark_ctx->plugins, file);
+	if(plugin) {
+		fputs("the plugin is already loaded.\n", stderr);
+		return NULL;
 	}
 
 	ret = stat(file, &st);
 	if (ret < 0) {
-		fprintf(stderr, "plugin %s not found\n", file);
-		return -ENODEV;
+		fprintf(stderr, "plugin %s not found.\n", file);
+		return NULL;
 	}
 
-	plugin = calloc(sizeof(struct kshark_plugin_list), 1);
+	plugin = calloc(1, sizeof(struct kshark_plugin_list));
 	if (!plugin) {
-		fprintf(stderr, "failed to allocate memory for plugin\n");
-		return -ENOMEM;
+		fputs("failed to allocate memory for plugin.\n", stderr);
+		return NULL;
 	}
 
-	if (asprintf(&plugin->file, "%s", file) <= 0) {
+	plugin->handle = dlopen(file, RTLD_NOW | RTLD_GLOBAL);
+	if (!plugin->handle) {
 		fprintf(stderr,
-			"failed to allocate memory for plugin file name");
-		return -ENOMEM;
+			"failed to open plugin file.\n%s\n",
+			dlerror());
+		goto fail;
 	}
 
-	plugin->handle = dlopen(plugin->file, RTLD_NOW | RTLD_GLOBAL);
-	if (!plugin->handle)
+	plugin->file = strdup(file);
+	plugin->name = strdup(name);
+	if (!plugin->file|| !plugin->name)
 		goto fail;
 
-	plugin->init = dlsym(plugin->handle,
-			     KSHARK_PLUGIN_INITIALIZER_NAME);
+	plugin->ctrl_interface =
+		dlsym(plugin->handle, KSHARK_MENU_PLUGIN_INITIALIZER_NAME);
+
+	init_func = dlsym(plugin->handle,
+			  KSHARK_PLOT_PLUGIN_INITIALIZER_NAME);
 
-	plugin->close = dlsym(plugin->handle,
-			      KSHARK_PLUGIN_DEINITIALIZER_NAME);
+	close_func = dlsym(plugin->handle,
+			   KSHARK_PLOT_PLUGIN_DEINITIALIZER_NAME);
 
-	if (!plugin->init || !plugin->close)
+	if (!!(long)init_func && !!(long)close_func) {
+		plugin->process_interface =
+			calloc(1, sizeof(*plugin->process_interface));
+
+		if (!plugin->process_interface)
+			goto fail;
+
+		plugin->process_interface->name = strdup(plugin->name);
+		plugin->process_interface->init = init_func;
+		plugin->process_interface->close = close_func;
+	} else if (!!(long)init_func || !!(long)close_func) {
+		fprintf(stderr,
+			"incomplete draw interface found (will be ignored).\n%s\n",
+			dlerror());
+	}
+
+	init_func = dlsym(plugin->handle,
+			  KSHARK_INPUT_INITIALIZER_NAME);
+
+	close_func = dlsym(plugin->handle,
+			   KSHARK_INPUT_DEINITIALIZER_NAME);
+
+	check_func = dlsym(plugin->handle,
+			   KSHARK_INPUT_CHECK_NAME);
+
+	if (!!(long)init_func &&
+	    !!(long)close_func &&
+	    !!(long)check_func) {
+		plugin->readout_interface =
+			calloc(1, sizeof(*plugin->readout_interface));
+
+		if (!plugin->readout_interface)
+			goto fail;
+
+		plugin->readout_interface->name = strdup(plugin->name);
+		plugin->readout_interface->init = init_func;
+		plugin->readout_interface->close = close_func;
+		plugin->readout_interface->check_data = check_func;
+
+		kshark_register_input(kshark_ctx, plugin->readout_interface);
+	} else if (!!(long)init_func ||
+		   !!(long)close_func ||
+		   !!(long)check_func) {
+		fprintf(stderr,
+			"incomplete input interface found (will be ignored).\n%s\n",
+			dlerror());
+	}
+
+	if (!plugin->process_interface &&
+	    !plugin->readout_interface &&
+	    !plugin->ctrl_interface) {
+		fputs("no interfaces found in this plugin.\n", stderr);
 		goto fail;
+	}
 
 	plugin->next = kshark_ctx->plugins;
 	kshark_ctx->plugins = plugin;
+	kshark_ctx->n_plugins++;
 
-	return 0;
+	return plugin;
 
  fail:
-	fprintf(stderr, "cannot load plugin '%s'\n%s\n",
-		plugin->file, dlerror());
+	fprintf(stderr, "cannot load plugin '%s'\n", file);
 
-	if (plugin->handle) {
+	if (plugin->handle)
 		dlclose(plugin->handle);
-		plugin->handle = NULL;
-	}
 
 	free(plugin);
 
-	return EFAULT;
+	return NULL;
+}
+
+/** Close and free this plugin. */
+static void free_plugin(struct kshark_plugin_list *plugin)
+{
+	dlclose(plugin->handle);
+
+	if (plugin->process_interface){
+		free(plugin->process_interface->name);
+		free(plugin->process_interface);
+	}
+
+	if (plugin->readout_interface) {
+		free(plugin->readout_interface->name);
+		free(plugin->readout_interface);
+	}
+
+	free(plugin->name);
+	free(plugin->file);
+	free(plugin);
 }
 
 /**
  * @brief Unrgister a plugin.
  *
- * @param kshark_ctx: Input location for context pointer.
+ * @param kshark_ctx: Input location for the session context pointer.
+ * @param name: The name of the plugin to unregister.
  * @param file: The plugin object file to unregister.
  */
 void kshark_unregister_plugin(struct kshark_context *kshark_ctx,
+			      const char *name,
 			      const char *file)
 {
 	struct kshark_plugin_list **last;
 
 	for (last = &kshark_ctx->plugins; *last; last = &(*last)->next) {
-		if (strcmp((*last)->file, file) == 0) {
+		if (strcmp((*last)->process_interface->name, name) == 0 &&
+		    strcmp((*last)->file, file) == 0) {
 			struct kshark_plugin_list *this_plugin;
+
 			this_plugin = *last;
 			*last = this_plugin->next;
+			free_plugin(this_plugin);
 
-			dlclose(this_plugin->handle);
-			free(this_plugin);
+			kshark_ctx->n_plugins--;
 
 			return;
 		}
@@ -248,48 +404,303 @@ void kshark_free_plugin_list(struct kshark_plugin_list *plugins)
 		last = plugins;
 		plugins = plugins->next;
 
-		free(last->file);
-		dlclose(last->handle);
+		free_plugin(last);
+	}
+}
+
+/**
+ * @brief Register a data readout interface (input).
+ *
+ * @param kshark_ctx: Input location for the context pointer.
+ * @param plugin: Input location for the data readout interface (input).
+ */
+struct kshark_dri_list *
+kshark_register_input(struct kshark_context *kshark_ctx,
+		      struct kshark_dri *plugin)
+{
+	struct kshark_dri_list *input;
+
+	input = calloc(1, sizeof(*input));
+	if (!input) {
+		fputs("failed to allocate memory for readout plugin.\n", stderr);
+		return NULL;
+	}
 
+	input->interface = plugin;
+	input->next = kshark_ctx->inputs;
+	kshark_ctx->inputs = input;
+
+	return input;
+}
+
+/**
+ * @brief Unrgister a data readout interface (input).
+ *
+ * @param kshark_ctx: Input location for the context pointer.
+ * @param name: The data readout's name.
+ */
+void kshark_unregister_input(struct kshark_context *kshark_ctx,
+			     const char *name)
+{
+	struct kshark_dri_list **last;
+
+	for (last = &kshark_ctx->inputs; *last; last = &(*last)->next) {
+		if (strcmp((*last)->interface->name, name) == 0) {
+			struct kshark_dri_list *this_input;
+			this_input = *last;
+			*last = this_input->next;
+
+			free(this_input);
+
+			return;
+		}
+	}
+}
+
+/**
+ * @brief Free a list of plugin interfaces.
+ *
+ * @param plugins: Input location for the plugins list.
+ */
+void
+kshark_free_dpi_list(struct kshark_dpi_list *plugins)
+{
+	struct kshark_dpi_list *last;
+
+	while (plugins) {
+		last = plugins;
+		plugins = plugins->next;
 		free(last);
 	}
 }
 
 /**
- * @brief Use this function to initialize/update/deinitialize all registered
- *	  plugins.
+ * @brief Find a plugin by its library file.
+ *
+ * @param plugins: A list of plugins to search in.
+ * @param lib: The plugin object file to load.
  *
- * @param kshark_ctx: Input location for context pointer.
+ * @returns The plugin object on success, or NULL on failure.
+ */
+struct kshark_plugin_list *
+kshark_find_plugin(struct kshark_plugin_list *plugins, const char *lib)
+{
+	for (; plugins; plugins = plugins->next)
+		if (strcmp(plugins->file, lib) == 0)
+			return plugins;
+
+	return NULL;
+}
+
+/**
+ * @brief Find a plugin by its name.
+ *
+ * @param plugins: A list of plugins to search in.
+ * @param name: The plugin object file to load.
+ *
+ * @returns The plugin object on success, or NULL on failure.
+ */
+struct kshark_plugin_list *
+kshark_find_plugin_by_name(struct kshark_plugin_list *plugins,
+			   const char *name)
+{
+	for (; plugins; plugins = plugins->next)
+		if (strcmp(plugins->name, name) == 0)
+			return plugins;
+
+	return NULL;
+}
+
+/**
+ * @brief Register plugin to a given data stream.
+ *
+ * @param stream: Input location for a Trace data stream pointer.
+ * @param plugin: Input location for the data processing interface.
+ * @param active: If false, the plugin will be registered but disables.
+ *		  Otherwise the plugin will be active.
+ *
+ * @returns The plugin object on success, or NULL on failure.
+ */
+struct kshark_dpi_list *
+kshark_register_plugin_to_stream(struct kshark_data_stream *stream,
+				 struct kshark_dpi *plugin,
+				 bool active)
+{
+	struct kshark_dpi_list *plugin_list = stream->plugins;
+
+	/* Check if the plugin is already registered to this stream. */
+	while (plugin_list) {
+		if (strcmp(plugin_list->interface->name, plugin->name) == 0 &&
+		    plugin_list->interface->init == plugin->init &&
+		    plugin_list->interface->close == plugin->close) {
+			kshark_handle_dpi(stream, plugin_list,
+					  KSHARK_PLUGIN_CLOSE);
+
+			plugin_list->status =
+				(active)? KSHARK_PLUGIN_ENABLED : 0;
+
+			return plugin_list;
+		}
+
+		plugin_list = plugin_list->next;
+	}
+
+	plugin_list = calloc(1, sizeof(*plugin_list));
+	plugin_list->interface = plugin;
+
+	if (active)
+		plugin_list->status = KSHARK_PLUGIN_ENABLED;
+
+	plugin_list->next = stream->plugins;
+	stream->plugins = plugin_list;
+	stream->n_plugins++;
+
+	return plugin_list;
+}
+
+/**
+ * @brief Unregister plugin to a given data stream.
+ *
+ * @param stream: Input location for a Trace data stream pointer.
+ * @param plugin: Input location for the data processing interface.
+ */
+void kshark_unregister_plugin_from_stream(struct kshark_data_stream *stream,
+				          struct kshark_dpi *plugin)
+{
+	struct kshark_dpi_list **last;
+
+	for (last = &stream->plugins; *last; last = &(*last)->next) {
+		if ((*last)->interface->init == plugin->init &&
+		    (*last)->interface->close == plugin->close &&
+		    strcmp((*last)->interface->name, plugin->name) == 0) {
+			struct kshark_dpi_list *this_plugin;
+
+			this_plugin = *last;
+			*last = this_plugin->next;
+			this_plugin->interface->close(stream);
+			free(this_plugin);
+
+			stream->n_plugins--;
+
+			return;
+		}
+	}
+}
+
+static void plugin_init(struct kshark_data_stream *stream,
+			struct kshark_dpi_list *plugin)
+{
+	int handler_count = plugin->interface->init(stream);
+
+	if (handler_count > 0) {
+		plugin->status &= ~KSHARK_PLUGIN_FAILED;
+		plugin->status |= KSHARK_PLUGIN_LOADED;
+	} else {
+		fprintf(stderr,
+			"plugin \"%s\" failed to initialize on stream %s:%s\n",
+			plugin->interface->name,
+			stream->file,
+			stream->name);
+
+		plugin->status |= KSHARK_PLUGIN_FAILED;
+		plugin->status &= ~KSHARK_PLUGIN_LOADED;
+	}
+}
+
+static void plugin_close(struct kshark_data_stream *stream,
+			 struct kshark_dpi_list *plugin)
+{
+	plugin->interface->close(stream);
+	plugin->status &= ~KSHARK_PLUGIN_LOADED;
+}
+
+/**
+ * @brief Use this function to initialize/update/deinitialize a plugin for
+ *	  a given Data stream.
+ *
+ * @param stream: Input location for a Trace data stream pointer.
+ * @param plugin: The plugin to be handled.
  * @param task_id: Action identifier specifying the action to be executed.
  *
  * @returns The number of successful added/removed plugin handlers on success,
  *	    or a negative error code on failure.
  */
-int kshark_handle_plugins(struct kshark_context *kshark_ctx,
-			  enum kshark_plugin_actions task_id)
+int kshark_handle_dpi(struct kshark_data_stream *stream,
+		      struct kshark_dpi_list *plugin,
+		      enum kshark_plugin_actions task_id)
 {
-	struct kshark_plugin_list *plugin;
 	int handler_count = 0;
 
-	for (plugin = kshark_ctx->plugins; plugin; plugin = plugin->next) {
-		switch (task_id) {
-		case KSHARK_PLUGIN_INIT:
-			handler_count += plugin->init(kshark_ctx);
-			break;
+	switch (task_id) {
+	case KSHARK_PLUGIN_INIT:
+		if (plugin->status & KSHARK_PLUGIN_ENABLED)
+			plugin_init(stream, plugin);
 
-		case KSHARK_PLUGIN_UPDATE:
-			plugin->close(kshark_ctx);
-			handler_count += plugin->init(kshark_ctx);
-			break;
+		break;
 
-		case KSHARK_PLUGIN_CLOSE:
-			handler_count += plugin->close(kshark_ctx);
-			break;
+	case KSHARK_PLUGIN_UPDATE:
+		if (plugin->status & KSHARK_PLUGIN_LOADED)
+			plugin_close(stream, plugin);
 
-		default:
-			return -EINVAL;
-		}
+		plugin->status &= ~KSHARK_PLUGIN_FAILED;
+
+		if (plugin->status & KSHARK_PLUGIN_ENABLED)
+			plugin_init(stream, plugin);
+
+		break;
+
+	case KSHARK_PLUGIN_CLOSE:
+		if (plugin->status & KSHARK_PLUGIN_LOADED)
+			plugin_close(stream, plugin);
+
+		plugin->status &= ~KSHARK_PLUGIN_FAILED;
+		break;
+
+	default:
+		return -EINVAL;
 	}
 
 	return handler_count;
 }
+
+/**
+ * @brief Use this function to initialize/update/deinitialize all registered
+ *	  data processing plugins for a given Data stream.
+ *
+ * @param stream: Input location for a Trace data stream pointer.
+ * @param task_id: Action identifier specifying the action to be executed. Can
+ *		   be KSHARK_PLUGIN_INIT, KSHARK_PLUGIN_UPDATE or
+ *		   KSHARK_PLUGIN_CLOSE.
+ *
+ * @returns The number of successful added/removed plugin handlers on success,
+ *	    or a negative error code on failure.
+ */
+int kshark_handle_all_dpis(struct kshark_data_stream *stream,
+			   enum kshark_plugin_actions task_id)
+{
+	struct kshark_dpi_list *plugin;
+	int handler_count = 0;
+
+	for (plugin = stream->plugins; plugin; plugin = plugin->next)
+		handler_count +=
+			kshark_handle_dpi(stream, plugin, task_id);
+
+	return handler_count;
+}
+
+/**
+ * @brief Free all readout interfaces in a given list.
+ *
+ * @param inputs: Input location for the inputs list.
+ */
+void kshark_free_dri_list(struct kshark_dri_list *inputs)
+{
+	struct kshark_dri_list *last;
+
+	while (inputs) {
+		last = inputs;
+		inputs = inputs->next;
+
+		free(last);
+	}
+}
src/libkshark-plugin.h +209 -70

355 lines not on this page · raw message

src/libkshark.c +55 -1

122 lines not on this page · raw message

src/libkshark.h +23 -2
index 54c71a77..a73539f4 100644
@@ -313,6 +313,18 @@ struct kshark_data_stream {
 	/** The type of the data. */
 	enum kshark_data_format		format;
 
+	/** List of Plugin interfaces. */
+	struct kshark_dpi_list	*plugins;
+
+	/** The number of plugins registered for this stream.*/
+	int			n_plugins;
+
+	/** List of Plugin's Event handlers. */
+	struct kshark_event_proc_handler	*event_handlers;
+
+	/** List of Plugin's Draw handlers. */
+	struct kshark_draw_handler		*draw_handlers;
+
 	/**
 	 * The interface of methods used to operate over the data from a given
 	 * stream.
@@ -368,11 +380,17 @@ struct kshark_context {
 	/** List of Data collections. */
 	struct kshark_entry_collection *collections;
 
+	/** List of data readout interfaces. */
+	struct kshark_dri_list		*inputs;
+
+	/** The number of readout interfaces. */
+	int				n_inputs;
+
 	/** List of Plugins. */
 	struct kshark_plugin_list	*plugins;
 
-	/** List of Plugin Event handlers. */
-	struct kshark_event_handler	*event_handlers;
+	/** The number of plugins. */
+	int				n_plugins;
 };
 
 bool kshark_instance(struct kshark_context **kshark_ctx);
@@ -665,6 +683,9 @@ void kshark_clear_all_filters(struct kshark_context *kshark_ctx,
 			      struct kshark_entry **data,
 			      size_t n_entries);
 
+void kshark_plugin_actions(struct kshark_data_stream *stream,
+			   void *record, struct kshark_entry *entry);
+
 /** Search failed identifiers. */
 enum kshark_search_failed {
 	/** All entries have timestamps greater timestamps. */
Signature · 1 line
2.25.1
#34 lore raw

Re: [PATCH v2 12/20] kernel-shark: Redesign the plugin interface

Quoted 11 lines · On Mon, 12 Oct 2020 16:35:15 +0300 "Yordan Karadzhov (VMware)"…

This patch can be treated as an almost complete rewrite of the way the C API of KernelShark deals with plugins. First of all, the two pluggable user actions (one executed during data loading and another executed during plotting) have separate handlers. This way the user can register only one of the two callback functions, if this is what is needed.

The second substantial change is that instead of having a single interface for loading the plugin, we now have 3 different interfaces. The one that exists in version 1 of KernelShark is now renamed to Data Processing Interface (dpi).

Gotta love TLAs and their collisions. Note, DPI also means "Dots per inch" which is used for visualization. https://en.wikipedia.org/wiki/Dots_per_inch

Not saying that we need to change it. But just need to be aware when mentioning "dpi" to clarify what you mean, to not confuse an audience.

Quoted 4 lines

The first new interface for loading can be used to register user provided implementation of the Data stream readout and is called Data Readout Interface (dri). Via this plugin loading interface the user

DRI can also mean "Dietary Reference Intakes", but I guess we can safely assume that nobody will confuse our DRI with that one ;-)

Quoted 141 lines

can open trace data having an arbitrary format. In order to make this possible the user has to provide a plugin that contains an implementation of the data readout methods defined by the kshark_data_stream_interface and to register all those methods.

The second new plugin loading interface is called Control interface and can be used to provide the plugin with an access to the GUI's Main window object. Via this interface the plugin can became capable to modify the GUI. Such a modification for example can be to add new dialog to the menus or to change the state of the GUI (to select entry 1with the marker, to zoom, ...).

---
 src/libkshark-plugin.c | 597 ++++++++++++++++++++++++++++++++++-------
 src/libkshark-plugin.h | 279 ++++++++++++++-----
 src/libkshark.c        |  56 +++-
 src/libkshark.h        |  25 +-
 4 files changed, 791 insertions(+), 166 deletions(-)
src/libkshark-plugin.c +34 -23
index 4b21392a..8e5299cf 100644
@@ -1,7 +1,7 @@
 // SPDX-License-Identifier: LGPL-2.1
 
 /*
- * Copyright (C) 2017 VMware Inc, Yordan Karadzhov <[email protected]>
+ * Copyright (C) 2017 VMware Inc, Yordan Karadzhov (VMware) <[email protected]>
  */
 
  /**
@@ -13,8 +13,7 @@
 #ifndef _GNU_SOURCE
 /** Use GNU C Library. */
 #define _GNU_SOURCE
-
-#endif
+#endif // _GNU_SOURCE
 
 #include <stdio.h>
 #include <stdlib.h>
@@ -27,23 +26,36 @@
 #include "libkshark-plugin.h"
 #include "libkshark.h"
 
-static struct kshark_event_handler *
-gui_event_handler_alloc(int event_id,
-			kshark_plugin_event_handler_func evt_func,
-			kshark_plugin_draw_handler_func dw_func)
+static struct kshark_event_proc_handler *
+data_event_handler_alloc(int event_id,
+			 kshark_plugin_event_handler_func evt_func)
 {
-	struct kshark_event_handler *handler = malloc(sizeof(*handler));
+	struct kshark_event_proc_handler *handler = malloc(sizeof(*handler));
 
 	if (!handler) {
-		fprintf(stderr,
-			"failed to allocate memory for gui eventhandler");
+		fputs("failed to allocate memory for event handler\n", stderr);
 		return NULL;
 	}
 
 	handler->next = NULL;
 	handler->id = event_id;
 	handler->event_func = evt_func;
-	handler->draw_func = dw_func;
+
+	return handler;
+}
+
+static struct kshark_draw_handler *
+data_draw_handler_alloc(kshark_plugin_draw_handler_func draw_func)
+{
+	struct kshark_draw_handler *handler = malloc(sizeof(*handler));
+
+	if (!handler) {
+		fputs("failed to allocate memory for draw handler\n", stderr);
+		return NULL;
+	}
+
+	handler->next = NULL;
+	handler->draw_func = draw_func;
 
 	return handler;
 }
@@ -55,8 +67,8 @@ gui_event_handler_alloc(int event_id,
  * @param handlers: Input location for the Event handler list.
  * @param event_id: Event Id to search for.
  */
-struct kshark_event_handler *
-kshark_find_event_handler(struct kshark_event_handler *handlers, int event_id)
+struct kshark_event_proc_handler *
+kshark_find_event_handler(struct kshark_event_proc_handler *handlers, int event_id)
 {
 	for (; handlers; handlers = handlers->next)
 		if (handlers->id == event_id)
@@ -68,26 +80,25 @@ kshark_find_event_handler(struct kshark_event_handler *handlers, int event_id)
 /**
  * @brief Add new event handler to an existing list of handlers.
  *
- * @param handlers: Input location for the Event handler list.
+ * @param stream: Input location for a Trace data stream pointer.
  * @param event_id: Event Id.
  * @param evt_func: Input location for an Event action provided by the plugin.
- * @param dw_func: Input location for a Draw action provided by the plugin.
  *
  * @returns Zero on success, or a negative error code on failure.
  */
-int kshark_register_event_handler(struct kshark_event_handler **handlers,
+int kshark_register_event_handler(struct kshark_data_stream *stream,
 				  int event_id,
-				  kshark_plugin_event_handler_func evt_func,
-				  kshark_plugin_draw_handler_func dw_func)
+				  kshark_plugin_event_handler_func evt_func)
 {
-	struct kshark_event_handler *handler =
-		gui_event_handler_alloc(event_id, evt_func, dw_func);
+	struct kshark_event_proc_handler *handler =
+		data_event_handler_alloc(event_id, evt_func);
 
 	if(!handler)
 		return -ENOMEM;
 
-	handler->next = *handlers;
-	*handlers = handler;
+	handler->next = stream->event_handlers;
+	stream->event_handlers = handler;
+
 	return 0;
 }
 
@@ -95,23 +106,20 @@ int kshark_register_event_handler(struct kshark_event_handler **handlers,
  * @brief Search the list for a specific plugin handle. If such a plugin handle
  *	  exists, unregister (remove and free) this handle from the list.
  *
- * @param handlers: Input location for the Event handler list.
+ * @param stream: Input location for a Trace data stream pointer.
  * @param event_id: Event Id of the plugin handler to be unregistered.

src/libkshark-plugin.c int kshark_register_event_handler(struct kshark_event_handler **handlers, · line 115

Quoted 10 lines
src/libkshark-plugin.c +3 -5
- * @param evt_func: Event action function of the handler to be unregistered.
- * @param dw_func: Draw action function of the handler to be unregistered.
+ * @param evt_func: Event action function to be unregistered.
  */
-void kshark_unregister_event_handler(struct kshark_event_handler **handlers,
+void kshark_unregister_event_handler(struct kshark_data_stream *stream,
 				     int event_id,
-				     kshark_plugin_event_handler_func evt_func,
-				     kshark_plugin_draw_handler_func dw_func)
+				     kshark_plugin_event_handler_func evt_func)

May be useful to have a return code just to see if one was found that needed to be unregistered. Not a hard requirement, but perhaps something that may be nice to have.

Quoted 20 lines
src/libkshark-plugin.c +4 -6
 {
-	struct kshark_event_handler **last;
+	struct kshark_event_proc_handler **last;
 
-	for (last = handlers; *last; last = &(*last)->next) {
+	for (last = &stream->event_handlers; *last; last = &(*last)->next) {
 		if ((*last)->id == event_id &&
-		    (*last)->event_func == evt_func &&
-		    (*last)->draw_func == dw_func) {
-			struct kshark_event_handler *this_handler;
+		    (*last)->event_func == evt_func) {
+			struct kshark_event_proc_handler *this_handler;
 			this_handler = *last;
 			*last = this_handler->next;
 			free(this_handler);
@@ -126,9 +134,71 @@ void kshark_unregister_event_handler(struct kshark_event_handler **handlers,
  *
  * @param handlers: Input location for the Event handler list.
  */
-void kshark_free_event_handler_list(struct kshark_event_handler *handlers)

src/libkshark-plugin.c void kshark_unregister_event_handler(struct kshark_event_handler **handlers, · line 146

Quoted 10 lines
src/libkshark-plugin.c +10 -0
+void kshark_free_event_handler_list(struct kshark_event_proc_handler *handlers)
+{
+	struct kshark_event_proc_handler *last;
+
+	while (handlers) {
+		last = handlers;
+		handlers = handlers->next;
+		free(last);
+	}
+}

I'm looking at the KsUtils.cpp that calls this, and I think it would be safer to pass in the kshark_ctx instead. Make the above function static, and then make two helper functions (for later).

void kshark_ctx_free_event_handlers(struct kshark_ctx *ctx)

{
	free_event_handler_list(ctx->event_handlers);
	ctx->event_handlers = NULL;
}

and

kshark_stream_free_event_handlers(struct kshark_data_stream *stream)

{
	free_event_handler_list(stream->event_handlers);
	stream->event_handers = NULL;
}

That way, we do not need to worry that the event_handlers will be left pointing to freed data if the code changes in the future.

Quoted 139 lines
src/libkshark-plugin.c +85 -22
+
+/**
+ * @brief Add new event handler to an existing list of handlers.
+ *
+ * @param stream: Input location for a Trace data stream pointer.
+ * @param draw_func: Input location for a Draw action provided by the plugin.
+ *
+ * @returns Zero on success, or a negative error code on failure.
+ */
+int kshark_register_draw_handler(struct kshark_data_stream *stream,
+				 kshark_plugin_draw_handler_func draw_func)
+{
+	struct kshark_draw_handler *handler = data_draw_handler_alloc(draw_func);
+
+	if(!handler)
+		return -ENOMEM;
+
+	handler->next = stream->draw_handlers;
+	stream->draw_handlers = handler;
+
+	return 0;
+}
+
+/**
+ * @brief Search the list for a specific plugin handle. If such a plugin handle
+ *	  exists, unregister (remove and free) this handle from the list.
+ *
+ * @param stream: Input location for a Trace data stream pointer.
+ * @param draw_func: Draw action function to be unregistered.
+ */
+void kshark_unregister_draw_handler(struct kshark_data_stream *stream,
+				    kshark_plugin_draw_handler_func draw_func)
+{
+	struct kshark_draw_handler **last;
+
+	for (last = &stream->draw_handlers; *last; last = &(*last)->next) {
+		if ((*last)->draw_func == draw_func) {
+			struct kshark_draw_handler *this_handler;
+			this_handler = *last;
+			*last = this_handler->next;
+			free(this_handler);
+
+			return;
+		}
+	}
+}
+
+/**
+ * @brief Free all DRaw handlers in a given list.
+ *
+ * @param handlers: Input location for the Draw handler list.
+ */
+void kshark_free_draw_handler_list(struct kshark_draw_handler *handlers)
 {
-	struct kshark_event_handler *last;
+	struct kshark_draw_handler *last;
 
 	while (handlers) {
 		last = handlers;
@@ -139,96 +209,182 @@ void kshark_free_event_handler_list(struct kshark_event_handler *handlers)
 
 /**
  * @brief Allocate memory for a new plugin. Add this plugin to the list of
- *	  plugins used by the session.
+ *	  plugins.
  *
  * @param kshark_ctx: Input location for the session context pointer.
+ * @param name: The name of the plugin to register.
  * @param file: The plugin object file to load.
  *
- * @returns Zero on success, or a negative error code on failure.
+ * @returns The plugin object on success, or NULL on failure.
  */
-int kshark_register_plugin(struct kshark_context *kshark_ctx,
-			   const char *file)
+struct kshark_plugin_list *
+kshark_register_plugin(struct kshark_context *kshark_ctx,
+		       const char *name,
+		       const char *file)
 {
-	struct kshark_plugin_list *plugin = kshark_ctx->plugins;
+	kshark_plugin_load_func init_func, close_func;
+	kshark_check_data_func check_func;
+	struct kshark_plugin_list *plugin;
 	struct stat st;
 	int ret;
 
-	while (plugin) {
-		if (strcmp(plugin->file, file) == 0)
-			return -EEXIST;
+	printf("loading plugin \"%s\" from %s\n", name, file);
 
-		plugin = plugin->next;
+	plugin = kshark_find_plugin(kshark_ctx->plugins, file);
+	if(plugin) {
+		fputs("the plugin is already loaded.\n", stderr);
+		return NULL;
 	}
 
 	ret = stat(file, &st);
 	if (ret < 0) {
-		fprintf(stderr, "plugin %s not found\n", file);
-		return -ENODEV;
+		fprintf(stderr, "plugin %s not found.\n", file);
+		return NULL;
 	}
 
-	plugin = calloc(sizeof(struct kshark_plugin_list), 1);
+	plugin = calloc(1, sizeof(struct kshark_plugin_list));
 	if (!plugin) {
-		fprintf(stderr, "failed to allocate memory for plugin\n");
-		return -ENOMEM;
+		fputs("failed to allocate memory for plugin.\n", stderr);
+		return NULL;
 	}
 
-	if (asprintf(&plugin->file, "%s", file) <= 0) {
+	plugin->handle = dlopen(file, RTLD_NOW | RTLD_GLOBAL);
+	if (!plugin->handle) {
 		fprintf(stderr,
-			"failed to allocate memory for plugin file name");
-		return -ENOMEM;
+			"failed to open plugin file.\n%s\n",
+			dlerror());
+		goto fail;
 	}
 
-	plugin->handle = dlopen(plugin->file, RTLD_NOW | RTLD_GLOBAL);
-	if (!plugin->handle)
+	plugin->file = strdup(file);
+	plugin->name = strdup(name);
+	if (!plugin->file|| !plugin->name)
 		goto fail;
 
-	plugin->init = dlsym(plugin->handle,
-			     KSHARK_PLUGIN_INITIALIZER_NAME);
+	plugin->ctrl_interface =
+		dlsym(plugin->handle, KSHARK_MENU_PLUGIN_INITIALIZER_NAME);
+

src/libkshark-plugin.c void kshark_free_event_handler_list(struct kshark_event_handler *handlers) · line 273

Quoted 10 lines
src/libkshark-plugin.c +5 -3
+	init_func = dlsym(plugin->handle,
+			  KSHARK_PLOT_PLUGIN_INITIALIZER_NAME);
 
-	plugin->close = dlsym(plugin->handle,
-			      KSHARK_PLUGIN_DEINITIALIZER_NAME);
+	close_func = dlsym(plugin->handle,
+			   KSHARK_PLOT_PLUGIN_DEINITIALIZER_NAME);
 
-	if (!plugin->init || !plugin->close)
+	if (!!(long)init_func && !!(long)close_func) {

You shouldn't need the !! here. It should just work with:

	if (init_func && close_func)

src/libkshark-plugin.c void kshark_free_event_handler_list(struct kshark_event_handler *handlers) · line 280

Quoted 7 lines
src/libkshark-plugin.c +7 -0
+		plugin->process_interface =
+			calloc(1, sizeof(*plugin->process_interface));
+
+		if (!plugin->process_interface)
+			goto fail;
+
+		plugin->process_interface->name = strdup(plugin->name);

You should check the return value of name.

Quoted 251 lines
src/libkshark-plugin.c +192 -16
+		plugin->process_interface->init = init_func;
+		plugin->process_interface->close = close_func;
+	} else if (!!(long)init_func || !!(long)close_func) {
+		fprintf(stderr,
+			"incomplete draw interface found (will be ignored).\n%s\n",
+			dlerror());
+	}
+
+	init_func = dlsym(plugin->handle,
+			  KSHARK_INPUT_INITIALIZER_NAME);
+
+	close_func = dlsym(plugin->handle,
+			   KSHARK_INPUT_DEINITIALIZER_NAME);
+
+	check_func = dlsym(plugin->handle,
+			   KSHARK_INPUT_CHECK_NAME);
+
+	if (!!(long)init_func &&
+	    !!(long)close_func &&
+	    !!(long)check_func) {
+		plugin->readout_interface =
+			calloc(1, sizeof(*plugin->readout_interface));
+
+		if (!plugin->readout_interface)
+			goto fail;
+
+		plugin->readout_interface->name = strdup(plugin->name);
+		plugin->readout_interface->init = init_func;
+		plugin->readout_interface->close = close_func;
+		plugin->readout_interface->check_data = check_func;
+
+		kshark_register_input(kshark_ctx, plugin->readout_interface);
+	} else if (!!(long)init_func ||
+		   !!(long)close_func ||
+		   !!(long)check_func) {
+		fprintf(stderr,
+			"incomplete input interface found (will be ignored).\n%s\n",
+			dlerror());
+	}
+
+	if (!plugin->process_interface &&
+	    !plugin->readout_interface &&
+	    !plugin->ctrl_interface) {
+		fputs("no interfaces found in this plugin.\n", stderr);
 		goto fail;
+	}
 
 	plugin->next = kshark_ctx->plugins;
 	kshark_ctx->plugins = plugin;
+	kshark_ctx->n_plugins++;
 
-	return 0;
+	return plugin;
 
  fail:
-	fprintf(stderr, "cannot load plugin '%s'\n%s\n",
-		plugin->file, dlerror());
+	fprintf(stderr, "cannot load plugin '%s'\n", file);
 
-	if (plugin->handle) {
+	if (plugin->handle)
 		dlclose(plugin->handle);
-		plugin->handle = NULL;
-	}
 
 	free(plugin);
 
-	return EFAULT;
+	return NULL;
+}
+
+/** Close and free this plugin. */
+static void free_plugin(struct kshark_plugin_list *plugin)
+{
+	dlclose(plugin->handle);
+
+	if (plugin->process_interface){
+		free(plugin->process_interface->name);
+		free(plugin->process_interface);
+	}
+
+	if (plugin->readout_interface) {
+		free(plugin->readout_interface->name);
+		free(plugin->readout_interface);
+	}
+
+	free(plugin->name);
+	free(plugin->file);
+	free(plugin);
 }
 
 /**
  * @brief Unrgister a plugin.
  *
- * @param kshark_ctx: Input location for context pointer.
+ * @param kshark_ctx: Input location for the session context pointer.
+ * @param name: The name of the plugin to unregister.
  * @param file: The plugin object file to unregister.
  */
 void kshark_unregister_plugin(struct kshark_context *kshark_ctx,
+			      const char *name,
 			      const char *file)
 {
 	struct kshark_plugin_list **last;
 
 	for (last = &kshark_ctx->plugins; *last; last = &(*last)->next) {
-		if (strcmp((*last)->file, file) == 0) {
+		if (strcmp((*last)->process_interface->name, name) == 0 &&
+		    strcmp((*last)->file, file) == 0) {
 			struct kshark_plugin_list *this_plugin;
+
 			this_plugin = *last;
 			*last = this_plugin->next;
+			free_plugin(this_plugin);
 
-			dlclose(this_plugin->handle);
-			free(this_plugin);
+			kshark_ctx->n_plugins--;
 
 			return;
 		}
@@ -248,48 +404,303 @@ void kshark_free_plugin_list(struct kshark_plugin_list *plugins)
 		last = plugins;
 		plugins = plugins->next;
 
-		free(last->file);
-		dlclose(last->handle);
+		free_plugin(last);
+	}
+}
+
+/**
+ * @brief Register a data readout interface (input).
+ *
+ * @param kshark_ctx: Input location for the context pointer.
+ * @param plugin: Input location for the data readout interface (input).
+ */
+struct kshark_dri_list *
+kshark_register_input(struct kshark_context *kshark_ctx,
+		      struct kshark_dri *plugin)
+{
+	struct kshark_dri_list *input;
+
+	input = calloc(1, sizeof(*input));
+	if (!input) {
+		fputs("failed to allocate memory for readout plugin.\n", stderr);
+		return NULL;
+	}
 
+	input->interface = plugin;
+	input->next = kshark_ctx->inputs;
+	kshark_ctx->inputs = input;
+
+	return input;
+}
+
+/**
+ * @brief Unrgister a data readout interface (input).
+ *
+ * @param kshark_ctx: Input location for the context pointer.
+ * @param name: The data readout's name.
+ */
+void kshark_unregister_input(struct kshark_context *kshark_ctx,
+			     const char *name)
+{
+	struct kshark_dri_list **last;
+
+	for (last = &kshark_ctx->inputs; *last; last = &(*last)->next) {
+		if (strcmp((*last)->interface->name, name) == 0) {
+			struct kshark_dri_list *this_input;
+			this_input = *last;
+			*last = this_input->next;
+
+			free(this_input);
+
+			return;
+		}
+	}
+}
+
+/**
+ * @brief Free a list of plugin interfaces.
+ *
+ * @param plugins: Input location for the plugins list.
+ */
+void
+kshark_free_dpi_list(struct kshark_dpi_list *plugins)
+{
+	struct kshark_dpi_list *last;
+
+	while (plugins) {
+		last = plugins;
+		plugins = plugins->next;
 		free(last);
 	}
 }
 
 /**
- * @brief Use this function to initialize/update/deinitialize all registered
- *	  plugins.
+ * @brief Find a plugin by its library file.
+ *
+ * @param plugins: A list of plugins to search in.
+ * @param lib: The plugin object file to load.
  *
- * @param kshark_ctx: Input location for context pointer.
+ * @returns The plugin object on success, or NULL on failure.
+ */
+struct kshark_plugin_list *
+kshark_find_plugin(struct kshark_plugin_list *plugins, const char *lib)
+{
+	for (; plugins; plugins = plugins->next)
+		if (strcmp(plugins->file, lib) == 0)
+			return plugins;
+
+	return NULL;
+}
+
+/**
+ * @brief Find a plugin by its name.
+ *
+ * @param plugins: A list of plugins to search in.
+ * @param name: The plugin object file to load.
+ *
+ * @returns The plugin object on success, or NULL on failure.
+ */
+struct kshark_plugin_list *
+kshark_find_plugin_by_name(struct kshark_plugin_list *plugins,
+			   const char *name)
+{
+	for (; plugins; plugins = plugins->next)
+		if (strcmp(plugins->name, name) == 0)
+			return plugins;
+
+	return NULL;
+}
+
+/**
+ * @brief Register plugin to a given data stream.
+ *
+ * @param stream: Input location for a Trace data stream pointer.
+ * @param plugin: Input location for the data processing interface.
+ * @param active: If false, the plugin will be registered but disables.
+ *		  Otherwise the plugin will be active.
+ *
+ * @returns The plugin object on success, or NULL on failure.
+ */
+struct kshark_dpi_list *
+kshark_register_plugin_to_stream(struct kshark_data_stream *stream,
+				 struct kshark_dpi *plugin,
+				 bool active)

src/libkshark-plugin.c void kshark_free_plugin_list(struct kshark_plugin_list *plugins) · line 537

Quoted 10 lines
src/libkshark-plugin.c +10 -0
+{
+	struct kshark_dpi_list *plugin_list = stream->plugins;
+
+	/* Check if the plugin is already registered to this stream. */
+	while (plugin_list) {
+		if (strcmp(plugin_list->interface->name, plugin->name) == 0 &&
+		    plugin_list->interface->init == plugin->init &&
+		    plugin_list->interface->close == plugin->close) {
+			kshark_handle_dpi(stream, plugin_list,
+					  KSHARK_PLUGIN_CLOSE);

Why are we closing the plugin here?

src/libkshark-plugin.c void kshark_free_plugin_list(struct kshark_plugin_list *plugins) · line 540

Quoted 3 lines
src/libkshark-plugin.c +3 -0
+
+			plugin_list->status =
+				(active)? KSHARK_PLUGIN_ENABLED : 0;

No need for the parenthesis around "active".

		active ? KSHARK_PLUGIN_ENABLED : 0;

-- Steve

Quoted 21 lines
src/libkshark-plugin.c +21 -0
+
+			return plugin_list;
+		}
+
+		plugin_list = plugin_list->next;
+	}
+
+	plugin_list = calloc(1, sizeof(*plugin_list));
+	plugin_list->interface = plugin;
+
+	if (active)
+		plugin_list->status = KSHARK_PLUGIN_ENABLED;
+
+	plugin_list->next = stream->plugins;
+	stream->plugins = plugin_list;
+	stream->n_plugins++;
+
+	return plugin_list;
+}
+
+/**
#35 lore raw

PATCH v2 13/20 kernel-shark: Complete the stream integration

In reply to Yordan Karadzhov (VMware)

The patch contains a number various relatively small modifications needed in order to finalize the integration. Unfortunately those changes are hard to disentangle so we will do everything in a single patch.

---
 examples/CMakeLists.txt    |   8 +-
 examples/datafilter.c      |  67 +++---
 src/CMakeLists.txt         |   3 +-
 src/libkshark-collection.c | 121 +++++++---
 src/libkshark-tepdata.c    | 352 +++++++++++++++++++++++++++++
 src/libkshark-tepdata.h    |  49 ++++
 src/libkshark.c            | 447 +++++++++++++++++++++++++------------
 src/libkshark.h            | 133 ++++++-----
 8 files changed, 920 insertions(+), 260 deletions(-)
examples/CMakeLists.txt +4 -4
index f6ed8978..8d40e42c 100644
@@ -4,10 +4,10 @@ message(STATUS "dataload")
 add_executable(dload          dataload.c)
 target_link_libraries(dload   kshark)
 
-# message(STATUS "datafilter")
-# add_executable(dfilter          datafilter.c)
-# target_link_libraries(dfilter   kshark)
-#
+message(STATUS "datafilter")
+add_executable(dfilter          datafilter.c)
+target_link_libraries(dfilter   kshark)
+
 # message(STATUS "datahisto")
 # add_executable(dhisto          datahisto.c)
 # target_link_libraries(dhisto   kshark)
examples/datafilter.c +34 -33

136 lines not on this page · raw message

src/CMakeLists.txt +2 -1
index 2bdab1a5..74e12a28 100644
@@ -7,7 +7,8 @@ add_library(kshark SHARED libkshark.c
                           libkshark-plugin.c
                           libkshark-tepdata.c
 #                           libkshark-configio.c
-                          libkshark-collection.c)
+                          libkshark-collection.c
+                          libkshark-tepdata.c)
 
 target_link_libraries(kshark ${TRACECMD_LIBRARY}
                              ${TRACEFS_LIBRARY}
src/libkshark-collection.c +96 -25

268 lines not on this page · raw message

src/libkshark-tepdata.c +352 -0

373 lines not on this page · raw message

src/libkshark-tepdata.h +49 -0

58 lines not on this page · raw message

src/libkshark.c +308 -139

613 lines not on this page · raw message

src/libkshark.h +75 -58

265 lines not on this page · raw message

Signature · 1 line
2.25.1
#36 lore raw

Re: [PATCH v2 13/20] kernel-shark: Complete the stream integration

Quoted 22 lines
src/libkshark-tepdata.c +6 -0
index 30c383c8..d9d57843 100644
@@ -318,6 +318,9 @@ static ssize_t get_records(struct kshark_context *kshark_ctx,
 
 				pid = entry->pid;
 
+				/* Apply Id filtering. */
+				kshark_apply_filters(kshark_ctx, stream, entry);
+
 				/* Apply advanced event filtering. */
 				if (adv_filter && adv_filter->filters &&
 				    tep_filter_match(adv_filter, rec) != FILTER_MATCH)
@@ -1160,6 +1163,43 @@ out:
 	return peer_handle;
 }
 
+/** A list of built in default plugins for FTRACE (trace-cmd) data. */
+const char *tep_plugin_names[] = {
+	"sched_events",

src/libkshark-tepdata.c out: · line 1178

Quoted 10 lines
src/libkshark-tepdata.c +10 -0
+	"missed_events",
+	"kvm_combo",
+};
+
+/**
+ * Register to the data stream all default plugins for FTRACE (trace-cmd) data.
+ */
+int kshark_tep_handle_plugins(struct kshark_context *kshark_ctx, int sd)
+{
+	int i, n_tep_plugins = sizeof(tep_plugin_names) / sizeof (const char *);

Note, a safer way is:

	n_tep_plugins = sizeof(tep_plugins_names) / sizeof(tep_plugin_names[0]);

Or use the ARRAY_SIZE() macro from trace-cmd.git/include/trace-cmd/trace-cmd.h:

#define ARRAY_SIZE(_a) (sizeof(_a) / sizeof((_a)[0]))

	n_tep_plugins = ARRAY_SIZE(tep_plugins_names);

;-)

Quoted 61 lines
src/libkshark-tepdata.c +54 -0
+	struct kshark_plugin_list *plugin;
+	struct kshark_data_stream *stream;
+
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
+		return -EEXIST;
+
+	for (i = 0; i < n_tep_plugins; ++i) {
+		plugin = kshark_find_plugin_by_name(kshark_ctx->plugins,
+						    tep_plugin_names[i]);
+
+		if (plugin && plugin->process_interface) {
+			kshark_register_plugin_to_stream(stream,
+							 plugin->process_interface,
+							 true);
+		} else {
+			fprintf(stderr, "Plugin \"%s\" not found.\n",
+				tep_plugin_names[i]);
+		}
+	}
+
+	return kshark_handle_all_dpis(stream, KSHARK_PLUGIN_INIT);
+}
+
 /** The Process Id of the Idle tasks is zero. */
 #define LINUX_IDLE_TASK_PID	0
 
@@ -1210,6 +1250,172 @@ static int kshark_tep_stream_init(struct kshark_data_stream *stream,
 	return -EFAULT;
 }
 
+static struct tracecmd_input *get_top_input(struct kshark_context *kshark_ctx,
+					    int sd)
+{
+	struct kshark_data_stream *top_stream;
+
+	top_stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!top_stream)
+		return NULL;
+
+	return kshark_get_tep_input(top_stream);
+}
+
+/**
+ * @brief Get an array containing the names of all buffers in FTRACE data
+ *	  file.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ * @param sd: Data stream identifier of the top buffers in the FTRACE data
+ *	  file.
+ * @param n_buffers: Output location for the size of the outputted array,
+ *	    or a negative error code on failure.
+ *
+ * @returns Array of strings on success, or NULL on failure. The user is
+ *	    responsible for freeing the elements of the outputted array.
+ */
+char **kshark_tep_get_buffer_names(struct kshark_context *kshark_ctx, int sd,
+				   int *n_buffers)
+{
+	struct tracecmd_input *top_input;
+	char **buffer_names;

src/libkshark-tepdata.c static int kshark_tep_stream_init(struct kshark_data_stream *stream, · line 1292

Quoted 10 lines
src/libkshark-tepdata.c +10 -0
+	int i, n;
+
+	top_input = get_top_input(kshark_ctx, sd);
+	if (!top_input) {
+		*n_buffers = -EFAULT;
+		return NULL;
+	}
+
+	n = tracecmd_buffer_instances(top_input);
+	buffer_names = malloc(n * sizeof(char *));

Need to check buffer_names.

src/libkshark-tepdata.c static int kshark_tep_stream_init(struct kshark_data_stream *stream, · line 1297

Quoted 5 lines
src/libkshark-tepdata.c +5 -0
+
+	for (i = 0; i < n; ++i)
+		buffer_names[i] =
+			strdup(tracecmd_buffer_instance_name(top_input, i));
+

And each buffer_names[i].

Quoted 4 lines
src/libkshark-tepdata.c +4 -0
+	*n_buffers = n;
+	return buffer_names;
+}
+

src/libkshark-tepdata.c static int kshark_tep_stream_init(struct kshark_data_stream *stream, · line 1311

Quoted 10 lines
src/libkshark-tepdata.c +10 -0
+static void set_stream_fields(struct tracecmd_input *top_input, int i,
+			      const char *file,
+			      const char *name,
+			      struct kshark_data_stream *buffer_stream,
+			      struct tracecmd_input **buffer_input)
+{
+	*buffer_input = tracecmd_buffer_instance_handle(top_input, i);
+
+	buffer_stream->name = strdup(name);
+	buffer_stream->file = strdup(file);

And the ->name and ->file.

Quoted 240 lines
src/libkshark-tepdata.c +233 -0
+	buffer_stream->format = KS_TEP_DATA;
+}
+
+/**
+ * @brief Open a given buffers in FTRACE (trace-cmd) data file.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ * @param sd: Data stream identifier of the top buffers in the FTRACE data
+ *	  file.
+ * @param buffer_name: The name of the buffer to open.
+ *
+ * @returns Data stream identifier of the buffer on success. Otherwise a
+ *	    negative error code.
+ */
+int kshark_tep_open_buffer(struct kshark_context *kshark_ctx, int sd,
+			   const char *buffer_name)
+{
+	struct kshark_data_stream *top_stream, *buffer_stream;
+	struct tracecmd_input *top_input, *buffer_input;
+	int i, sd_buffer, n_buffers, ret = -ENODATA;
+	char **names;
+
+	top_stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!top_stream)
+		return -EFAULT;
+
+	top_input = kshark_get_tep_input(top_stream);
+	if (!top_input)
+		return -EFAULT;
+
+	names = kshark_tep_get_buffer_names(kshark_ctx, sd, &n_buffers);
+
+	sd_buffer = kshark_add_stream(kshark_ctx);
+	buffer_stream = kshark_get_data_stream(kshark_ctx, sd_buffer);
+	if (!buffer_stream)
+		return -EFAULT;
+
+	for (i = 0; i < n_buffers; ++i) {
+		if (strcmp(buffer_name, names[i]) == 0) {
+			set_stream_fields(top_input, i,
+					  top_stream->file,
+					  buffer_name,
+					  buffer_stream,
+					  &buffer_input);
+
+			ret = kshark_tep_stream_init(buffer_stream,
+						     buffer_input);
+			break;
+		}
+	}
+
+	for (i = 0; i < n_buffers; ++i)
+		free(names[i]);
+	free(names);
+
+	return (ret < 0)? ret : buffer_stream->stream_id;
+}
+
+/**
+ * @brief Initialize data streams for all buffers in a FTRACE (trace-cmd) data
+ *	  file.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ * @param sd: Data stream identifier of the top buffers in the FTRACE data
+ *	  file.
+ *
+ * @returns The total number of data streams initialized on success. Otherwise
+ *	    a negative error code.
+ */
+int kshark_tep_init_all_buffers(struct kshark_context *kshark_ctx,
+				int sd)
+{
+	struct kshark_data_stream *top_stream, *buffer_stream;
+	struct tracecmd_input *buffer_input;
+	struct tracecmd_input *top_input;
+	int i, n_buffers, sd_buffer, ret;
+
+	top_stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!top_stream)
+		return -EFAULT;
+
+	top_input = kshark_get_tep_input(top_stream);
+	if (!top_input)
+		return -EFAULT;
+
+	n_buffers = tracecmd_buffer_instances(top_input);
+	for (i = 0; i < n_buffers; ++i) {
+		sd_buffer = kshark_add_stream(kshark_ctx);
+		if (sd_buffer < 0)
+			return -EFAULT;
+
+		buffer_stream = kshark_ctx->stream[sd_buffer];
+
+		set_stream_fields(top_input, i,
+				  top_stream->file,
+				  tracecmd_buffer_instance_name(top_input, i),
+				  buffer_stream,
+				  &buffer_input);
+
+		ret = kshark_tep_stream_init(buffer_stream, buffer_input);
+		if (ret != 0)
+			return -EFAULT;
+	}
+
+	return n_buffers;
+}
+
 /** Initialize the FTRACE data input (from file). */
 int kshark_tep_init_input(struct kshark_data_stream *stream,
 			  const char *file)
@@ -1389,3 +1595,149 @@ char **kshark_tracecmd_local_plugins()
 {
 	return tracefs_tracers(tracefs_get_tracing_dir());
 }
+
+/**
+ * @brief Free an array, allocated by kshark_tracecmd_get_hostguest_mapping() API
+ *
+ *
+ * @param map: Array, allocated by kshark_tracecmd_get_hostguest_mapping() API
+ * @param count: Number of entries in the array
+ *
+ */
+void kshark_tracecmd_free_hostguest_map(struct kshark_host_guest_map *map, int count)
+{
+	int i;
+
+	if (!map)
+		return;
+	for (i = 0; i < count; i++) {
+		free(map[i].guest_name);
+		free(map[i].cpu_pid);
+		memset(&map[i], 0, sizeof(*map));
+	}
+	free(map);
+}
+
+/**
+ * @brief Get mapping of guest VCPU to host task, running that VCPU.
+ *	  Array of mappings for each guest is allocated and returned
+ *	  in map input parameter.
+ *
+ *
+ * @param map: Returns allocated array of kshark_host_guest_map structures, each
+ *	       one describing VCPUs mapping of one guest.
+ *
+ * @return The number of entries in the *map array, or a negative error code on
+ *	   failure.
+ */
+int kshark_tracecmd_get_hostguest_mapping(struct kshark_host_guest_map **map)
+{
+	struct kshark_host_guest_map *gmap = NULL;
+	struct tracecmd_input *peer_handle = NULL;
+	struct kshark_data_stream *peer_stream;
+	struct tracecmd_input *guest_handle = NULL;
+	struct kshark_data_stream *guest_stream;
+	struct kshark_context *kshark_ctx = NULL;
+	unsigned long long trace_id;
+	const char *name;
+	int vcpu_count;
+	const int *cpu_pid;
+	int *stream_ids;
+	int i, j, k;
+	int count = 0;
+	int ret;
+
+	if (!map || !kshark_instance(&kshark_ctx))
+		return -EFAULT;
+	if (*map)
+		return -EEXIST;
+
+	stream_ids = kshark_all_streams(kshark_ctx);
+	for (i = 0; i < kshark_ctx->n_streams; i++) {
+		guest_stream = kshark_get_data_stream(kshark_ctx, stream_ids[i]);
+		if (!guest_stream || guest_stream->format != KS_TEP_DATA)
+			continue;
+		guest_handle = kshark_get_tep_input(guest_stream);
+		if (!guest_handle)
+			continue;
+		trace_id = tracecmd_get_traceid(guest_handle);
+		if (!trace_id)
+			continue;
+		for (j = 0; j < kshark_ctx->n_streams; j++) {
+			if (stream_ids[i] == stream_ids[j])
+				continue;
+			peer_stream = kshark_get_data_stream(kshark_ctx, stream_ids[j]);
+			if (!peer_stream || peer_stream->format != KS_TEP_DATA)
+				continue;
+			peer_handle = kshark_get_tep_input(peer_stream);
+			if (!peer_handle)
+				continue;
+			ret = tracecmd_get_guest_cpumap(peer_handle, trace_id,
+							&name, &vcpu_count, &cpu_pid);
+			if (!ret && vcpu_count) {
+				gmap = realloc(*map,
+					       (count + 1) * sizeof(struct kshark_host_guest_map));
+				if (!gmap)
+					goto mem_error;
+				*map = gmap;
+				memset(&gmap[count], 0, sizeof(struct kshark_host_guest_map));
+				count++;
+				gmap[count - 1].guest_id = stream_ids[i];
+				gmap[count - 1].host_id = stream_ids[j];
+				gmap[count - 1].guest_name = strdup(name);
+				if (!gmap[count - 1].guest_name)
+					goto mem_error;
+				gmap[count - 1].vcpu_count = vcpu_count;
+				gmap[count - 1].cpu_pid = malloc(sizeof(int) * vcpu_count);
+				if (!gmap[count - 1].cpu_pid)
+					goto mem_error;
+				for (k = 0; k < vcpu_count; k++)
+					gmap[count - 1].cpu_pid[k] = cpu_pid[k];
+				break;
+			}
+		}
+	}
+
+	free(stream_ids);
+	return count;
+
+mem_error:
+	free(stream_ids);
+	if (*map) {
+		kshark_tracecmd_free_hostguest_map(*map, count);
+		*map = NULL;
+	}
+
+	return -ENOMEM;
+}
+
+/**
+ * @brief Find the data stream corresponding the top buffer of a FTRACE
+ *	  (trace-cmd) data file.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ * @param file: The name of the file.
+ *
+ * @returns Data stream identifier of the top buffers in the FTRACE data
+ *	    fileon success. Otherwise a negative error code.
+ */

src/libkshark-tepdata.c char **kshark_tracecmd_local_plugins() · line 1733

Quoted 10 lines
src/libkshark-tepdata.c +10 -0
+int kshark_tep_find_top_stream(struct kshark_context *kshark_ctx,
+			       const char *file)
+{
+	struct kshark_data_stream *top_stream = NULL, *stream;
+	int i, *stream_ids = kshark_all_streams(kshark_ctx);
+
+	for (i = 0; i < kshark_ctx->n_streams; ++i) {
+		stream = kshark_ctx->stream[stream_ids[i]];
+		if (strcmp(stream->file, file) == 0 &&
+		    strcmp(stream->name, "top") == 0)

I noticed that you hardcode the top_stream name as "top". A couple of comments. One, good software practice is not to have any open constants. That is, always use a macro, as it makes it easier to go global changes later on. Also, don't call it "top". What happens if I make an instance called "top". Will it confuse this?

Maybe make it a non printable character:

const char top_name[] = { 0x1b, 0x00 }; // Non printable character
#define TOP_NAME (char *)&top_name

Or something like this.

Quoted 427 lines
src/libkshark-tepdata.c +10 -0
+			top_stream = stream;
+	}
+
+	free(stream_ids);
+
+	if (!top_stream)
+		return -EEXIST;
+
+	return top_stream->stream_id;
+}
src/libkshark-tepdata.h +49 -0
index a2bd211e..b6c9439e 100644
@@ -41,9 +41,58 @@ void kshark_tep_filter_reset(struct kshark_data_stream *stream);
 
 char **kshark_tracecmd_local_plugins();
 
+struct tep_handle;
+
+struct tep_handle *kshark_get_tep(struct kshark_data_stream *stream);
+
+struct tracecmd_input;
+
+struct tracecmd_input *kshark_get_tep_input(struct kshark_data_stream *stream);
+
+struct tep_record;
+
 ssize_t kshark_load_tep_records(struct kshark_context *kshark_ctx, int sd,
 				struct tep_record ***data_rows);
 
+/**
+ * Structure representing the mapping between the virtual CPUs and their
+ * corresponding processes in the host.
+ */
+struct kshark_host_guest_map {
+	/** ID of guest stream */
+	int guest_id;
+
+	/** ID of host stream */
+	int host_id;
+
+	/** Guest name */
+	char *guest_name;
+
+	/** Number of guest's CPUs in *cpu_pid array */
+	int vcpu_count;
+
+	/** Array of host task PIDs, index is the VCPU id */
+	int *cpu_pid;
+};
+
+void kshark_tracecmd_free_hostguest_map(struct kshark_host_guest_map *map,
+					int count);
+
+int kshark_tracecmd_get_hostguest_mapping(struct kshark_host_guest_map **map);
+
+char **kshark_tep_get_buffer_names(struct kshark_context *kshark_ctx, int sd,
+				   int *n_buffers);
+
+int kshark_tep_open_buffer(struct kshark_context *kshark_ctx, int sd,
+			   const char *buffer_name);
+
+int kshark_tep_init_all_buffers(struct kshark_context *kshark_ctx, int sd);
+
+int kshark_tep_handle_plugins(struct kshark_context *kshark_ctx, int sd);
+
+int kshark_tep_find_top_stream(struct kshark_context *kshark_ctx,
+			       const char *file);
+
 #ifdef __cplusplus
 }
 #endif
src/libkshark.c +175 -84
index bd2e4cc0..6ce7b6ba 100644
@@ -1,7 +1,7 @@
 // SPDX-License-Identifier: LGPL-2.1
 
 /*
- * Copyright (C) 2017 VMware Inc, Yordan Karadzhov <[email protected]>
+ * Copyright (C) 2017 VMware Inc, Yordan Karadzhov (VMware) <[email protected]>
  */
 
  /**
@@ -9,8 +9,10 @@
  *  @brief   API for processing of tracing data.
  */
 
+#ifndef _GNU_SOURCE
 /** Use GNU C Library. */
-#define _GNU_SOURCE 1
+#define _GNU_SOURCE
+#endif // _GNU_SOURCE
 
 // C
 #include <stdlib.h>
@@ -36,16 +38,6 @@ static bool kshark_default_context(struct kshark_context **context)
 				    sizeof(*kshark_ctx->stream));
 
 	kshark_ctx->collections = NULL;
-	kshark_ctx->plugins = NULL;
-
-	kshark_ctx->show_task_filter = tracecmd_filter_id_hash_alloc();
-	kshark_ctx->hide_task_filter = tracecmd_filter_id_hash_alloc();
-
-	kshark_ctx->show_event_filter = tracecmd_filter_id_hash_alloc();
-	kshark_ctx->hide_event_filter = tracecmd_filter_id_hash_alloc();
-
-	kshark_ctx->show_cpu_filter = tracecmd_filter_id_hash_alloc();
-	kshark_ctx->hide_cpu_filter = tracecmd_filter_id_hash_alloc();
 
 	kshark_ctx->filter_mask = 0x0;
 
@@ -386,6 +378,12 @@ void kshark_close(struct kshark_context *kshark_ctx, int sd)
 	if (!stream)
 		return;
 
+	/*
+	 * All data collections are file specific. Make sure that collections
+	 * from this file are not going to be used with another file.
+	 */
+	kshark_unregister_stream_collections(&kshark_ctx->collections, sd);
+
 	/* Close all active plugins for this stream. */
 	if (stream->plugins) {
 		kshark_handle_all_dpis(stream, KSHARK_PLUGIN_CLOSE);
@@ -530,103 +528,134 @@ ssize_t kshark_get_task_pids(struct kshark_context *kshark_ctx, int sd,
 	return stream->tasks->count;
 }
 
-static bool filter_find(struct tracecmd_filter_id *filter, int pid,
+static bool filter_find(struct kshark_hash_id *filter, int pid,
 			bool test)
 {
 	return !filter || !filter->count ||
-		!!(unsigned long)tracecmd_filter_id_find(filter, pid) == test;
+	       kshark_hash_id_find(filter, pid) == test;
 }
 
-static bool kshark_show_task(struct kshark_context *kshark_ctx, int pid)
+static bool kshark_show_task(struct kshark_data_stream *stream, int pid)
 {
-	return filter_find(kshark_ctx->show_task_filter, pid, true) &&
-	       filter_find(kshark_ctx->hide_task_filter, pid, false);
+	return filter_find(stream->show_task_filter, pid, true) &&
+	       filter_find(stream->hide_task_filter, pid, false);
 }
 
-static bool kshark_show_event(struct kshark_context *kshark_ctx, int pid)
+static bool kshark_show_event(struct kshark_data_stream *stream, int pid)
 {
-	return filter_find(kshark_ctx->show_event_filter, pid, true) &&
-	       filter_find(kshark_ctx->hide_event_filter, pid, false);
+	return filter_find(stream->show_event_filter, pid, true) &&
+	       filter_find(stream->hide_event_filter, pid, false);
 }
 
-static bool kshark_show_cpu(struct kshark_context *kshark_ctx, int cpu)
+static bool kshark_show_cpu(struct kshark_data_stream *stream, int cpu)
 {
-	return filter_find(kshark_ctx->show_cpu_filter, cpu, true) &&
-	       filter_find(kshark_ctx->hide_cpu_filter, cpu, false);
+	return filter_find(stream->show_cpu_filter, cpu, true) &&
+	       filter_find(stream->hide_cpu_filter, cpu, false);
+}
+
+static struct kshark_hash_id *get_filter(struct kshark_context *kshark_ctx,
+					 int sd,
+					 enum kshark_filter_type filter_id)
+{
+	struct kshark_data_stream *stream;
+
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
+		return NULL;
+
+	return kshark_get_filter(stream, filter_id);
 }
 
 /**
- * @brief Add an Id value to the filster specified by "filter_id".
+ * @brief Get an Id Filter.
+ *
+ * @param stream: Input location for a Trace data stream pointer.
+ * @param filter_id: Identifier of the filter.
+ */
+struct kshark_hash_id *
+kshark_get_filter(struct kshark_data_stream *stream,
+		  enum kshark_filter_type filter_id)
+{
+	switch (filter_id) {
+	case KS_SHOW_CPU_FILTER:
+		return stream->show_cpu_filter;
+	case KS_HIDE_CPU_FILTER:
+		return stream->hide_cpu_filter;
+	case KS_SHOW_EVENT_FILTER:
+		return stream->show_event_filter;
+	case KS_HIDE_EVENT_FILTER:
+		return stream->hide_event_filter;
+	case KS_SHOW_TASK_FILTER:
+		return stream->show_task_filter;
+	case KS_HIDE_TASK_FILTER:
+		return stream->hide_task_filter;
+	default:
+		return NULL;
+	}
+}
+
+/**
+ * @brief Add an Id value to the filter specified by "filter_id".
  *
  * @param kshark_ctx: Input location for the session context pointer.
+ * @param sd: Data stream identifier.
  * @param filter_id: Identifier of the filter.
  * @param id: Id value to be added to the filter.
  */
-void kshark_filter_add_id(struct kshark_context *kshark_ctx,
+void kshark_filter_add_id(struct kshark_context *kshark_ctx, int sd,
 			  int filter_id, int id)
 {
-	struct tracecmd_filter_id *filter;
+	struct kshark_hash_id *filter;
 
-	switch (filter_id) {
-		case KS_SHOW_CPU_FILTER:
-			filter = kshark_ctx->show_cpu_filter;
-			break;
-		case KS_HIDE_CPU_FILTER:
-			filter = kshark_ctx->hide_cpu_filter;
-			break;
-		case KS_SHOW_EVENT_FILTER:
-			filter = kshark_ctx->show_event_filter;
-			break;
-		case KS_HIDE_EVENT_FILTER:
-			filter = kshark_ctx->hide_event_filter;
-			break;
-		case KS_SHOW_TASK_FILTER:
-			filter = kshark_ctx->show_task_filter;
-			break;
-		case KS_HIDE_TASK_FILTER:
-			filter = kshark_ctx->hide_task_filter;
-			break;
-		default:
-			return;
+	filter = get_filter(kshark_ctx, sd, filter_id);
+	if (filter)
+		kshark_hash_id_add(filter, id);
+}
+
+/**
+ * @brief Get an array containing all Ids associated with a given Id Filter.
+ *
+ * @param kshark_ctx: Input location for context pointer.
+ * @param sd: Data stream identifier.
+ * @param filter_id: Identifier of the filter.
+ * @param n: Output location for the size of the returned array.
+ *
+ * @return The user is responsible for freeing the array.
+ */
+int *kshark_get_filter_ids(struct kshark_context *kshark_ctx, int sd,
+			   int filter_id, int *n)
+{
+	struct kshark_hash_id *filter;
+
+	filter = get_filter(kshark_ctx, sd, filter_id);
+	if (filter) {
+		if (n)
+			*n = filter->count;
+
+		return kshark_hash_ids(filter);
 	}
 
-	tracecmd_filter_id_add(filter, id);
+	if (n)
+		*n = 0;
+
+	return NULL;
 }
 
 /**
- * @brief Clear (reset) the filster specified by "filter_id".
+ * @brief Clear (reset) the filter specified by "filter_id".
  *
  * @param kshark_ctx: Input location for the session context pointer.
+ * @param sd: Data stream identifier.
  * @param filter_id: Identifier of the filter.
  */
-void kshark_filter_clear(struct kshark_context *kshark_ctx, int filter_id)
+void kshark_filter_clear(struct kshark_context *kshark_ctx, int sd,
+			 int filter_id)
 {
-	struct tracecmd_filter_id *filter;
-
-	switch (filter_id) {
-		case KS_SHOW_CPU_FILTER:
-			filter = kshark_ctx->show_cpu_filter;
-			break;
-		case KS_HIDE_CPU_FILTER:
-			filter = kshark_ctx->hide_cpu_filter;
-			break;
-		case KS_SHOW_EVENT_FILTER:
-			filter = kshark_ctx->show_event_filter;
-			break;
-		case KS_HIDE_EVENT_FILTER:
-			filter = kshark_ctx->hide_event_filter;
-			break;
-		case KS_SHOW_TASK_FILTER:
-			filter = kshark_ctx->show_task_filter;
-			break;
-		case KS_HIDE_TASK_FILTER:
-			filter = kshark_ctx->hide_task_filter;
-			break;
-		default:
-			return;
-	}
+	struct kshark_hash_id *filter;
 
-	tracecmd_filter_id_clear(filter);
+	filter = get_filter(kshark_ctx, sd, filter_id);
+	if (filter)
+		kshark_hash_id_clear(filter);
 }
 
 /**
@@ -636,7 +665,7 @@ void kshark_filter_clear(struct kshark_context *kshark_ctx, int filter_id)
  *
  * @returns True if the Id filter is set, otherwise False.
  */
-bool kshark_this_filter_is_set(struct tracecmd_filter_id *filter)
+bool kshark_this_filter_is_set(struct kshark_hash_id *filter)
 {
 	return filter && filter->count;
 }
@@ -645,17 +674,49 @@ bool kshark_this_filter_is_set(struct tracecmd_filter_id *filter)
  * @brief Check if an Id filter is set.
  *
  * @param kshark_ctx: Input location for the session context pointer.
+ * @param sd: Data stream identifier.
  *
- * @returns True if at least one Id filter is set, otherwise False.
+ * @returns True if at least one Id filter of the stream is set, otherwise
+ *	    False.
  */
-bool kshark_filter_is_set(struct kshark_context *kshark_ctx)
+bool kshark_filter_is_set(struct kshark_context *kshark_ctx, int sd)
 {
-	return kshark_this_filter_is_set(kshark_ctx->show_task_filter) ||
--              kshark_this_filter_is_set(kshark_ctx->hide_task_filter) ||
--              kshark_this_filter_is_set(kshark_ctx->show_cpu_filter) ||
--              kshark_this_filter_is_set(kshark_ctx->hide_cpu_filter) ||
--              kshark_this_filter_is_set(kshark_ctx->show_event_filter) ||
--              kshark_this_filter_is_set(kshark_ctx->hide_event_filter);
+	struct kshark_data_stream *stream;
+
+	stream = kshark_get_data_stream(kshark_ctx, sd);
+	if (!stream)
+		return false;
+
+	return kshark_this_filter_is_set(stream->show_task_filter) ||
+	       kshark_this_filter_is_set(stream->hide_task_filter)  ||
+	       kshark_this_filter_is_set(stream->show_cpu_filter)   ||
+	       kshark_this_filter_is_set(stream->hide_cpu_filter)   ||
+	       kshark_this_filter_is_set(stream->show_event_filter) ||
+	       kshark_this_filter_is_set(stream->hide_event_filter);
+}
+
+/**
+ * @brief Apply filters to a given entry.
+ *
+ * @param kshark_ctx: Input location for the session context pointer.
+ * @param stream: Input location for a Trace data stream pointer.
+ * @param entry: Input location for entry.
+ */
+void kshark_apply_filters(struct kshark_context *kshark_ctx,
+			  struct kshark_data_stream *stream,
+			  struct kshark_entry *entry)
+{
+	/* Apply event filtering. */
+	if (!kshark_show_event(stream, entry->event_id))
+		unset_event_filter_flag(kshark_ctx, entry);
+
+	/* Apply CPU filtering. */
+	if (!kshark_show_cpu(stream, entry->cpu))
+		entry->visible &= ~kshark_ctx->filter_mask;
+
+	/* Apply task filtering. */
+	if (!kshark_show_task(stream, entry->pid))
+		entry->visible &= ~kshark_ctx->filter_mask;
 }
 
 static void set_all_visible(uint8_t *v) {
@@ -663,56 +724,100 @@ static void set_all_visible(uint8_t *v) {
 	*v |= 0xFF & ~KS_PLUGIN_UNTOUCHED_MASK;
 }
 
+static void filter_entries(struct kshark_context *kshark_ctx, int sd,
+			   struct kshark_entry **data, size_t n_entries)
+{
+	struct kshark_data_stream *stream = NULL;
+	size_t i;
+
+	/* Sanity checks before starting. */
+	if (sd >= 0) {
+		/* We will filter particular Data stream. */
+		stream = kshark_get_data_stream(kshark_ctx, sd);
+		if (!stream)
+			return;
+
+		if (stream->format == KS_TEP_DATA &&
+		    kshark_tep_filter_is_set(stream)) {
+			/* The advanced filter is set. */
+			fprintf(stderr,
+				"Failed to filter (sd = %i)!\n", sd);
+			fprintf(stderr,
+				"Reset the Advanced filter or reload the data.\n");
+
+			return;
+		}
+
+		if (!kshark_filter_is_set(kshark_ctx, sd))
+			return;
+	}
+
+	/* Apply only the Id filters. */
+	for (i = 0; i < n_entries; ++i) {

src/libkshark.c static void set_all_visible(uint8_t *v) { · line 766

Quoted 10 lines
src/libkshark.c +10 -0
+		if (sd >= 0) {
+			/*
+			 * We only filter particular stream. Chack is the entry
+			 * belongs to this stream.
+			 */
+			if (data[i]->stream_id != sd)
+				continue;
+		} else {
+			/* We filter all streams. */
+			stream = kshark_ctx->stream[data[i]->stream_id];

Like we have discussed, we should be able to get a stream from the kshark_ctx and the content of data[i], if we make data[i] point to a pointer to an entry.

-- Steve

Quoted 10 lines
src/libkshark.c +10 -0
+		}
+
+		/* Start with and entry which is visible everywhere. */
+		set_all_visible(&data[i]->visible);
+
+		/* Apply Id filtering. */
+		kshark_apply_filters(kshark_ctx, stream, data[i]);
+	}
+}
+
#37 lore raw

PATCH v2 14/20 kernel-shark: Provide merging of multiple data streams

In reply to Yordan Karadzhov (VMware)

The C API provides loading of the trace data in two different forms. The firs one is an array of kshark_entries and is being used by the KernelShark GUI. The second is a matrix-like structure that has all the fields of the kshark_entry stored in separate arrays, forming the columns of the matrix. The second form of the data is used by trace-cruncher. In this patch we add methods for merging of several data streams into a single data set. Both kshark_entries and matrix forms of the data are supported. This patch includes a simple example that demonstrate how to open a file that contains multiple buffers. Each buffers is loaded into a separate Data stream and those streams are merged together.

---
 examples/CMakeLists.txt    |   4 +
 examples/multibufferload.c |  60 +++++++++
 src/libkshark.c            | 255 +++++++++++++++++++++++++++++++++++++
 src/libkshark.h            |  47 +++++++
 4 files changed, 366 insertions(+)
 create mode 100644 examples/multibufferload.c
examples/CMakeLists.txt +4 -0
index 8d40e42c..831eee24 100644
@@ -8,6 +8,10 @@ message(STATUS "datafilter")
 add_executable(dfilter          datafilter.c)
 target_link_libraries(dfilter   kshark)
 
+message(STATUS "multibufferload")
+add_executable(mbload          multibufferload.c)
+target_link_libraries(mbload   kshark)
+
 # message(STATUS "datahisto")
 # add_executable(dhisto          datahisto.c)
 # target_link_libraries(dhisto   kshark)
examples/multibufferload.c new file +60 -0

60 lines not on this page · raw message

src/libkshark.c +255 -0

258 lines not on this page · raw message

src/libkshark.h +47 -0

59 lines not on this page · raw message

Signature · 1 line
2.25.1
#38 lore raw

PATCH v2 15/20 kernel-shark: Integrate the stream definitions with data model

In reply to Yordan Karadzhov (VMware)

The visualization model is adapted in order to be able to distinguish the entries belonging to different Data streams.
We re-enable the visualization model example as well.

---
 examples/CMakeLists.txt |   8 ++--
 examples/datahisto.c    |  38 +++++++--------
 src/CMakeLists.txt      |   2 +-
 src/libkshark-model.c   | 102 ++++++++++++++++++++++++----------------
 src/libkshark-model.h   |  34 +++++++-------
 5 files changed, 102 insertions(+), 82 deletions(-)
examples/CMakeLists.txt +4 -4
index 831eee24..2f6acea2 100644
@@ -12,10 +12,10 @@ message(STATUS "multibufferload")
 add_executable(mbload          multibufferload.c)
 target_link_libraries(mbload   kshark)
 
-# message(STATUS "datahisto")
-# add_executable(dhisto          datahisto.c)
-# target_link_libraries(dhisto   kshark)
-#
+message(STATUS "datahisto")
+add_executable(dhisto          datahisto.c)
+target_link_libraries(dhisto   kshark)
+
 # message(STATUS "confogio")
 # add_executable(confio          configio.c)
 # target_link_libraries(confio   kshark)
examples/datahisto.c +19 -19
index b177b088..568072d8 100644
@@ -7,6 +7,7 @@
 // C
 #include <stdio.h>
 #include <stdlib.h>
+#include <string.h>
 
 // KernelShark
 #include "libkshark.h"
@@ -16,7 +17,7 @@
 
 const char *default_file = "trace.dat";
 
-void dump_bin(struct kshark_trace_histo *histo, int bin,
+void dump_bin(struct kshark_trace_histo *histo, int bin, int sd,
 	      const char *type, int val)
 {
 	const struct kshark_entry *e_front, *e_back;
@@ -26,22 +27,22 @@ void dump_bin(struct kshark_trace_histo *histo, int bin,
 	printf("bin %i {\n", bin);
 	if (strcmp(type, "cpu") == 0) {
 		e_front = ksmodel_get_entry_front(histo, bin, true,
-						  kshark_match_cpu, val,
+						  kshark_match_cpu, sd, &val,
 						  NULL,
 						  &i_front);
 
 		e_back = ksmodel_get_entry_back(histo, bin, true,
-						kshark_match_cpu, val,
+						kshark_match_cpu, sd, &val,
 						NULL,
 						&i_back);
 	} else if (strcmp(type, "task") == 0) {
 		e_front = ksmodel_get_entry_front(histo, bin, true,
-						  kshark_match_pid, val,
+						  kshark_match_pid, sd, &val,
 						  NULL,
 						  &i_front);
 
 		e_back = ksmodel_get_entry_back(histo, bin, true,
-						kshark_match_pid, val,
+						kshark_match_pid, sd, &val,
 						NULL,
 						&i_back);
 	} else {
@@ -67,12 +68,12 @@ void dump_bin(struct kshark_trace_histo *histo, int bin,
 	puts("}\n");
 }
 
-void dump_histo(struct kshark_trace_histo *histo, const char *type, int val)
+void dump_histo(struct kshark_trace_histo *histo, int sd, const char *type, int val)
 {
 	size_t bin;
 
 	for (bin = 0; bin < histo->n_bins; ++bin)
-		dump_bin(histo, bin, type, val);
+		dump_bin(histo, bin, sd, type, val);
 }
 
 int main(int argc, char **argv)
@@ -81,8 +82,7 @@ int main(int argc, char **argv)
 	struct kshark_entry **data = NULL;
 	struct kshark_trace_histo histo;
 	ssize_t i, n_rows, n_tasks;
-	bool status;
-	int *pids;
+	int sd, *pids;
 
 	/* Create a new kshark session. */
 	kshark_ctx = NULL;
@@ -91,24 +91,24 @@ int main(int argc, char **argv)
 
 	/* Open a trace data file produced by trace-cmd. */
 	if (argc > 1)
-		status = kshark_open(kshark_ctx, argv[1]);
+		sd = kshark_open(kshark_ctx, argv[1]);
 	else
-		status = kshark_open(kshark_ctx, default_file);
+		sd = kshark_open(kshark_ctx, default_file);
 
-	if (!status) {
+	if (sd < 0) {
 		kshark_free(kshark_ctx);
 		return 1;
 	}
 
 	/* Load the content of the file into an array of entries. */
-	n_rows = kshark_load_data_entries(kshark_ctx, &data);
+	n_rows = kshark_load_entries(kshark_ctx, sd, &data);
 	if (n_rows < 1) {
 		kshark_free(kshark_ctx);
 		return 1;
 	}
 
 	/* Get a list of all tasks. */
-	n_tasks = kshark_get_task_pids(kshark_ctx, &pids);
+	n_tasks = kshark_get_task_pids(kshark_ctx, sd, &pids);
 
 	/* Initialize the Visualization Model. */
 	ksmodel_init(&histo);
@@ -119,7 +119,7 @@ int main(int argc, char **argv)
 	ksmodel_fill(&histo, data, n_rows);
 
 	/* Dump the raw bins. */
-	dump_histo(&histo, "", 0);
+	dump_histo(&histo, sd, "", 0);
 
 	puts("\n...\n\n");
 
@@ -127,13 +127,13 @@ int main(int argc, char **argv)
 	 * Change the state of the model. Do 50% Zoom-In and dump only CPU 0.
 	 */
 	ksmodel_zoom_in(&histo, .50, -1);
-	dump_histo(&histo, "cpu", 0);
+	dump_histo(&histo, sd, "cpu", 0);
 
 	puts("\n...\n\n");
 
 	/* Shift forward by two bins and this time dump only CPU 1. */
 	ksmodel_shift_forward(&histo, 2);
-	dump_histo(&histo, "cpu", 1);
+	dump_histo(&histo, sd, "cpu", 1);
 
 	puts("\n...\n\n");
 
@@ -142,7 +142,7 @@ int main(int argc, char **argv)
 	 * Task.
 	 */
 	ksmodel_zoom_out(&histo, .10, N_BINS - 1);
-	dump_histo(&histo, "task", pids[n_tasks - 1]);
+	dump_histo(&histo, sd, "task", pids[n_tasks - 1]);
 
 	/* Reset (clear) the model. */
 	ksmodel_clear(&histo);
@@ -154,7 +154,7 @@ int main(int argc, char **argv)
 	free(data);
 
 	/* Close the file. */
-	kshark_close(kshark_ctx);
+	kshark_close(kshark_ctx, sd);
 
 	/* Close the session. */
 	kshark_free(kshark_ctx);
src/CMakeLists.txt +1 -1
index 74e12a28..72eb8fd2 100644
@@ -3,7 +3,7 @@ message("\n src ...")
 message(STATUS "libkshark")
 add_library(kshark SHARED libkshark.c
                           libkshark-hash.c
-#                           libkshark-model.c
+                          libkshark-model.c
                           libkshark-plugin.c
                           libkshark-tepdata.c
 #                           libkshark-configio.c
src/libkshark-model.c +61 -41

358 lines not on this page · raw message

src/libkshark-model.h +17 -17
index 47793b11..87e252eb 100644
@@ -1,13 +1,13 @@
 /* SPDX-License-Identifier: LGPL-2.1 */
 
 /*
- * Copyright (C) 2017 VMware Inc, Yordan Karadzhov <[email protected]>
+ * Copyright (C) 2017 VMware Inc, Yordan Karadzhov (VMware) <[email protected]>
  */
 
- /**
-  *  @file    libkshark-model.h
-  *  @brief   Visualization model for FTRACE (trace-cmd) data.
-  */
+/**
+ *  @file    libkshark-model.h
+ *  @brief   Time series visualization model for tracing data.
+ */
 
 #ifndef _LIB_KSHARK_MODEL_H
 #define _LIB_KSHARK_MODEL_H
@@ -102,64 +102,64 @@ ssize_t ksmodel_first_index_at_bin(struct kshark_trace_histo *histo, int bin);
 ssize_t ksmodel_last_index_at_bin(struct kshark_trace_histo *histo, int bin);
 
 ssize_t ksmodel_first_index_at_cpu(struct kshark_trace_histo *histo,
-				   int bin, int cpu);
+				   int bin, int sd, int cpu);
 
 ssize_t ksmodel_first_index_at_pid(struct kshark_trace_histo *histo,
-				   int bin, int pid);
+				   int bin, int sd, int pid);
 
 const struct kshark_entry *
 ksmodel_get_entry_front(struct kshark_trace_histo *histo,
 			int bin, bool vis_only,
-			matching_condition_func func, int val,
+			matching_condition_func func, int sd, int *values,
 			struct kshark_entry_collection *col,
 			ssize_t *index);
 
 const struct kshark_entry *
 ksmodel_get_entry_back(struct kshark_trace_histo *histo,
 		       int bin, bool vis_only,
-		       matching_condition_func func, int val,
+		       matching_condition_func func, int sd, int *values,
 		       struct kshark_entry_collection *col,
 		       ssize_t *index);
 
 int ksmodel_get_pid_front(struct kshark_trace_histo *histo,
-			  int bin, int cpu, bool vis_only,
+			  int bin, int sd, int cpu, bool vis_only,
 			  struct kshark_entry_collection *col,
 			  ssize_t *index);
 
 int ksmodel_get_pid_back(struct kshark_trace_histo *histo,
-			 int bin, int cpu, bool vis_only,
+			 int bin, int sd, int cpu, bool vis_only,
 			 struct kshark_entry_collection *col,
 			 ssize_t *index);
 
 int ksmodel_get_cpu_front(struct kshark_trace_histo *histo,
-			  int bin, int pid, bool vis_only,
+			  int bin, int sd, int pid, bool vis_only,
 			  struct kshark_entry_collection *col,
 			  ssize_t *index);
 
 int ksmodel_get_cpu_back(struct kshark_trace_histo *histo,
-			 int bin, int pid, bool vis_only,
+			 int bin, int sd, int pid, bool vis_only,
 			 struct kshark_entry_collection *col,
 			 ssize_t *index);
 
 bool ksmodel_cpu_visible_event_exist(struct kshark_trace_histo *histo,
-				     int bin, int cpu,
+				     int bin, int sd, int cpu,
 				     struct kshark_entry_collection *col,
 				     ssize_t *index);
 
 bool ksmodel_task_visible_event_exist(struct kshark_trace_histo *histo,
-				      int bin, int pid,
+				      int bin, int sd, int pid,
 				      struct kshark_entry_collection *col,
 				      ssize_t *index);
 
 const struct kshark_entry *
 ksmodel_get_cpu_missed_events(struct kshark_trace_histo *histo,
-			      int bin, int cpu,
+			      int bin, int sd, int cpu,
 			      struct kshark_entry_collection *col,
 			      ssize_t *index);
 
 const struct kshark_entry *
 ksmodel_get_task_missed_events(struct kshark_trace_histo *histo,
-			       int bin, int pid,
+			       int bin, int sd, int pid,
 			       struct kshark_entry_collection *col,
 			       ssize_t *index);
 
Signature · 1 line
2.25.1
#39 lore raw

PATCH v2 16/20 kernel-shark: Use only signed types for model defs

In reply to Yordan Karadzhov (VMware)

All variables that are used to describe the state of the model, like the range of the model or the size of the bin must have signed integer types. This is a continuation of the change started by switching to signed timestamps.

---
 src/libkshark-model.c | 12 ++++++------
 src/libkshark-model.h | 18 +++++++++---------
 2 files changed, 15 insertions(+), 15 deletions(-)
src/libkshark-model.c +6 -6
index 97cff314..f3864bfd 100644
@@ -91,10 +91,10 @@ static bool ksmodel_histo_alloc(struct kshark_trace_histo *histo, size_t n)
 }
 
 static void ksmodel_set_in_range_bining(struct kshark_trace_histo *histo,
-					size_t n, uint64_t min, uint64_t max,
+					size_t n, int64_t min, int64_t max,
 					bool force_in_range)
 {
-	uint64_t corrected_range, delta_range, range = max - min;
+	int64_t corrected_range, delta_range, range = max - min;
 	struct kshark_entry *last;
 
 	/* The size of the bin must be >= 1, hence the range must be >= n. */
@@ -163,7 +163,7 @@ static void ksmodel_set_in_range_bining(struct kshark_trace_histo *histo,
  * @param max: Upper edge of the time-window to be visualized.
  */
 void ksmodel_set_bining(struct kshark_trace_histo *histo,
-			size_t n, uint64_t min, uint64_t max)
+			size_t n, int64_t min, int64_t max)
 {
 	ksmodel_set_in_range_bining(histo, n, min, max, false);
 }
@@ -256,7 +256,7 @@ static size_t ksmodel_set_upper_edge(struct kshark_trace_histo *histo)
 static void ksmodel_set_next_bin_edge(struct kshark_trace_histo *histo,
 				      size_t bin, size_t last_row)
 {
-	uint64_t time_min, time_max;
+	int64_t time_min, time_max;
 	size_t next_bin = bin + 1;
 	ssize_t row;
 
@@ -598,9 +598,9 @@ void ksmodel_shift_backward(struct kshark_trace_histo *histo, size_t n)
  * @param histo: Input location for the model descriptor.
  * @param ts: position in time to be visualized.
  */
-void ksmodel_jump_to(struct kshark_trace_histo *histo, uint64_t ts)
+void ksmodel_jump_to(struct kshark_trace_histo *histo, int64_t ts)
 {
-	uint64_t min, max, range_min;
+	int64_t min, max, range_min;
 
 	if (ts > histo->min && ts < histo->max) {
 		/*
src/libkshark-model.h +9 -9
index 87e252eb..b8624809 100644
@@ -58,16 +58,16 @@ struct kshark_trace_histo {
 	 * Lower edge of the time-window to be visualized. Only entries having
 	 * timestamp >= min will be visualized.
 	 */
-	uint64_t		min;
+	int64_t			min;
 
 	/**
 	 * Upper edge of the time-window to be visualized. Only entries having
 	 * timestamp <= max will be visualized.
 	 */
-	uint64_t		max;
+	int64_t			max;
 
 	/** The size in time for each bin. */
-	uint64_t		bin_size;
+	int64_t			bin_size;
 
 	/** Number of bins. */
 	int			n_bins;
@@ -78,7 +78,7 @@ void ksmodel_init(struct kshark_trace_histo *histo);
 void ksmodel_clear(struct kshark_trace_histo *histo);
 
 void ksmodel_set_bining(struct kshark_trace_histo *histo,
-			size_t n, uint64_t min, uint64_t max);
+			size_t n, int64_t min, int64_t max);
 
 void ksmodel_fill(struct kshark_trace_histo *histo,
 		  struct kshark_entry **data, size_t n);
@@ -89,7 +89,7 @@ void ksmodel_shift_forward(struct kshark_trace_histo *histo, size_t n);
 
 void ksmodel_shift_backward(struct kshark_trace_histo *histo, size_t n);
 
-void ksmodel_jump_to(struct kshark_trace_histo *histo, uint64_t ts);
+void ksmodel_jump_to(struct kshark_trace_histo *histo, int64_t ts);
 
 void ksmodel_zoom_out(struct kshark_trace_histo *histo,
 		      double r, int mark);
@@ -163,16 +163,16 @@ ksmodel_get_task_missed_events(struct kshark_trace_histo *histo,
 			       struct kshark_entry_collection *col,
 			       ssize_t *index);
 
-static inline double ksmodel_bin_time(struct kshark_trace_histo *histo,
+static inline int64_t ksmodel_bin_ts(struct kshark_trace_histo *histo,
 				      int bin)
 {
-	return (histo->min + bin*histo->bin_size) * 1e-9;
+	return (histo->min + bin*histo->bin_size);
 }
 
-static inline uint64_t ksmodel_bin_ts(struct kshark_trace_histo *histo,
+static inline double ksmodel_bin_time(struct kshark_trace_histo *histo,
 				      int bin)
 {
-	return (histo->min + bin*histo->bin_size);
+	return ksmodel_bin_ts(histo, bin) * 1e-9;
 }
 
 #ifdef __cplusplus
Signature · 1 line
2.25.1
#40 lore raw

PATCH v2 17/20 kernel-shark: Add ksmodel_get_bin()

In reply to Yordan Karadzhov (VMware)

The method retrieves the Id of the bin of the model that contains given entry.

---
 src/libkshark-model.c | 27 +++++++++++++++++++++++++++
 src/libkshark-model.h |  3 +++
 2 files changed, 30 insertions(+)
src/libkshark-model.c +27 -0
index f3864bfd..2301b069 100644
@@ -1325,3 +1325,30 @@ ksmodel_get_task_missed_events(struct kshark_trace_histo *histo,
 				       match_pid_missed_events, sd, &pid,
 				       col, index);
 }
+
+/**
+ * @brief Find the bin Id of a give entry.
+ *
+ * @param histo: Input location for the model descriptor.
+ * @param entry: Input location for entry.
+ *
+ * @returns If the timestamp of the entry is inside the range of the model the
+ * 	    function returns the Id of the bin it belongs to.
+ * 	    If the timestamp of the entry is outside of the range of the model
+ * 	    the function returns UPPER_OVERFLOW_BIN or LOWER_OVERFLOW_BIN
+ * 	    (negative values).
+ */
+int ksmodel_get_bin(struct kshark_trace_histo *histo,
+		    const struct kshark_entry *entry)
+{
+	if (entry->ts < histo->min)
+		return UPPER_OVERFLOW_BIN;
+
+	if (entry->ts > histo->max)
+		return LOWER_OVERFLOW_BIN;
+
+	if (entry->ts == histo->max)
+		return histo->n_bins - 1;
+
+	return  (entry->ts - histo->min) / histo->bin_size;
+}
src/libkshark-model.h +3 -0
index b8624809..8989ee0e 100644
@@ -175,6 +175,9 @@ static inline double ksmodel_bin_time(struct kshark_trace_histo *histo,
 	return ksmodel_bin_ts(histo, bin) * 1e-9;
 }
 
+int ksmodel_get_bin(struct kshark_trace_histo *histo,
+		    const struct kshark_entry *entry);
+
 #ifdef __cplusplus
 }
 #endif // __cplusplus
Signature · 1 line
2.25.1
#41 lore raw

PATCH v2 18/20 kernel-shark: Protect ksmodel_set_in_range_bining()

In reply to Yordan Karadzhov (VMware)

Handle the case when the number of bins is zero or negative.

---
 src/libkshark-model.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
src/libkshark-model.c +12 -1
index 2301b069..44f829d6 100644
@@ -97,8 +97,19 @@ static void ksmodel_set_in_range_bining(struct kshark_trace_histo *histo,
 	int64_t corrected_range, delta_range, range = max - min;
 	struct kshark_entry *last;
 
+	if (n <= 0) {
+		histo->n_bins = histo->bin_size = 0;
+		histo->min = min;
+		histo->max = max;
+
+		free(histo->bin_count);
+		free(histo->map);
+
+		return;
+	}
+
 	/* The size of the bin must be >= 1, hence the range must be >= n. */
-	if (n == 0 || range < n) {
+	if (range < n) {
 		range = n;
 		max = min + n;
 	}
Signature · 1 line
2.25.1
#42 lore raw

PATCH v2 19/20 kernel-shark: Add methods for time calibration

In reply to Yordan Karadzhov (VMware)

We add an infrastructure for correcting the timestamps of the entries. This is needed in order to correlate Data streams that have been recorded using non-synchronized clocks. The infrastructure can handle an arbitrary timestamps correction formula, however for the moment we only provide calibration that adds a constant offset.

---
 src/libkshark-tepdata.c | 13 +++++-
 src/libkshark.c         | 99 +++++++++++++++++++++++++++++++++++++++++
 src/libkshark.h         | 27 +++++++++++
 3 files changed, 138 insertions(+), 1 deletion(-)
src/libkshark-tepdata.c +12 -1
index d9d57843..31cb33d5 100644
@@ -296,6 +296,9 @@ static ssize_t get_records(struct kshark_context *kshark_ctx,
 					entry = &temp_rec->entry;
 					missed_events_action(stream, rec, entry);
 
+					/* Apply time calibration. */
+					kshark_postprocess_entry(stream, rec, entry);
+
 					entry->stream_id = stream->stream_id;
 
 					temp_next = &temp_rec->next;
@@ -316,6 +319,12 @@ static ssize_t get_records(struct kshark_context *kshark_ctx,
 
 				entry->stream_id = stream->stream_id;
 
+				/*
+				 * Post-process the content of the entry. This includes
+				 * time calibration and event-specific plugin actions.
+				 */
+				kshark_postprocess_entry(stream, rec, entry);
+
 				pid = entry->pid;
 
 				/* Apply Id filtering. */
@@ -481,8 +490,10 @@ static ssize_t tepdata_load_matrix(struct kshark_data_stream *stream,
 			if (cpu_array)
 				(*cpu_array)[count] = e->cpu;
 
-			if (ts_array)
+			if (ts_array) {
+				kshark_calib_entry(stream, e);
 				(*ts_array)[count] = e->ts;
+			}
 
 			if (pid_array)
 				(*pid_array)[count] = e->pid;
src/libkshark.c +99 -0

117 lines not on this page · raw message

src/libkshark.h +27 -0
index fbe1b2ac..dc30665b 100644
@@ -115,6 +115,12 @@ void kshark_hash_id_free(struct kshark_hash_id *hash);
 
 int *kshark_hash_ids(struct kshark_hash_id *hash);
 
+/**
+ * Timestamp calibration function type. To be user for system clock
+ * calibration.
+ */
+typedef void (*time_calib_func) (int64_t *, int64_t *);
+
 struct kshark_data_stream;
 
 /** A function type to be used to initialize the interface of the data stream. */
@@ -315,6 +321,15 @@ struct kshark_data_stream {
 	/** The number of plugins registered for this stream.*/
 	int			n_plugins;
 
+	/** System clock calibration function. */
+	time_calib_func		calib;
+
+	/** An array of time calibration constants. */
+	int64_t			*calib_array;
+
+	/** The size of the array of time calibration constants. */
+	size_t			calib_array_size;
+
 	/** List of Plugin's Event handlers. */
 	struct kshark_event_proc_handler	*event_handlers;
 
@@ -669,6 +684,12 @@ void kshark_clear_all_filters(struct kshark_context *kshark_ctx,
 void kshark_plugin_actions(struct kshark_data_stream *stream,
 			   void *record, struct kshark_entry *entry);
 
+void kshark_calib_entry(struct kshark_data_stream *stream,
+			struct kshark_entry *entry);
+
+void kshark_postprocess_entry(struct kshark_data_stream *stream,
+			      void *record, struct kshark_entry *entry);
+
 /** Search failed identifiers. */
 enum kshark_search_failed {
 	/** All entries have greater timestamps. */
@@ -1059,6 +1080,12 @@ struct kshark_config_doc *kshark_open_config_file(const char *file_name,
 
 struct kshark_config_doc *kshark_json_to_conf(struct json_object *jobj);
 
+void kshark_offset_calib(int64_t *ts, int64_t *atgv);
+
+void kshark_set_clock_offset(struct kshark_context *kshark_ctx,
+			     struct kshark_entry **entries, size_t size,
+			     int sd, int64_t offset);
+
 /** Structure representing a data set made of KernelShark entries. */
 struct kshark_entry_data_set {
 	/** Array of entries pointers. */
Signature · 1 line
2.25.1
#43 lore raw

PATCH v2 20/20 kernel-shark: Integrate streams with libkshark-configio

In reply to Yordan Karadzhov (VMware)

The configuration input/output is adapted in order to be able to work with the new version of the C API. Now it can handle multiple Data streams. We are re-enabling the corresponding example as well.

---
 examples/CMakeLists.txt  |    8 +-
 examples/configio.c      |   20 +-
 src/CMakeLists.txt       |    2 +-
 src/libkshark-configio.c | 1173 ++++++++++++++++++++++++++++++++------
 src/libkshark.h          |   73 ++-
 5 files changed, 1060 insertions(+), 216 deletions(-)
examples/CMakeLists.txt +4 -4
index 2f6acea2..b8bc79a4 100644
@@ -16,10 +16,10 @@ message(STATUS "datahisto")
 add_executable(dhisto          datahisto.c)
 target_link_libraries(dhisto   kshark)
 
-# message(STATUS "confogio")
-# add_executable(confio          configio.c)
-# target_link_libraries(confio   kshark)
-#
+message(STATUS "confogio")
+add_executable(confio          configio.c)
+target_link_libraries(confio   kshark)
+
 # message(STATUS "dataplot")
 # add_executable(dplot          dataplot.cpp)
 # target_link_libraries(dplot   kshark-plot)
examples/configio.c +12 -8
index faa116a1..9710d533 100644
@@ -7,22 +7,26 @@ int main(int argc, char **argv)
 {
 	struct kshark_config_doc *conf, *filter, *hello;
 	struct kshark_context *kshark_ctx;
-	int *ids = NULL, i;
+	struct kshark_data_stream *stream;
+	int sd, *ids = NULL, i;
 
 	/* Create a new kshark session. */
 	kshark_ctx = NULL;
 	if (!kshark_instance(&kshark_ctx))
 		return 1;
 
+	sd = kshark_add_stream(kshark_ctx);
+	stream = kshark_ctx->stream[sd];
+
 	if (argc == 1) {
-		tracecmd_filter_id_add(kshark_ctx->show_task_filter, 314);
-		tracecmd_filter_id_add(kshark_ctx->show_task_filter, 42);
+		kshark_hash_id_add(stream->show_task_filter, 314);
+		kshark_hash_id_add(stream->show_task_filter, 42);
 
 		/* Create a new Confog. doc. */
 		conf = kshark_config_new("foo.bar.config", KS_CONFIG_JSON);
 
 		/* Add filter's info. */
-		filter = kshark_export_all_filters(kshark_ctx, KS_CONFIG_JSON);
+		filter = kshark_export_all_filters(kshark_ctx, sd, KS_CONFIG_JSON);
 		kshark_config_doc_add(conf, "Filters" ,filter);
 
 		/* Add "Hello Kernel" message. */
@@ -39,11 +43,11 @@ int main(int argc, char **argv)
 		/* Retrieve the filter's info. */
 		filter = kshark_config_alloc(KS_CONFIG_JSON);
 		if (kshark_config_doc_get(conf, "Filters" ,filter)) {
-			kshark_import_all_filters(kshark_ctx, filter);
+			kshark_import_all_filters(kshark_ctx, sd, filter);
 
 			/* Get the array of Ids to be fitered. */
-			ids = tracecmd_filter_ids(kshark_ctx->show_task_filter);
-			for (i = 0; i < kshark_ctx->show_task_filter->count; ++i)
+			ids = kshark_hash_ids(stream->show_task_filter);
+			for (i = 0; i < stream->show_task_filter->count; ++i)
 				printf("pid: %i\n", ids[i]);
 		}
 
@@ -58,7 +62,7 @@ int main(int argc, char **argv)
 	}
 
 	kshark_free_config_doc(conf);
-
+	kshark_close(kshark_ctx, sd);
 	kshark_free(kshark_ctx);
 
 	return 0;
src/CMakeLists.txt +1 -1
index 72eb8fd2..14a6ab69 100644
@@ -6,7 +6,7 @@ add_library(kshark SHARED libkshark.c
                           libkshark-model.c
                           libkshark-plugin.c
                           libkshark-tepdata.c
-#                           libkshark-configio.c
+                          libkshark-configio.c
                           libkshark-collection.c
                           libkshark-tepdata.c)
 
src/libkshark-configio.c +984 -189

1,724 lines not on this page · raw message

src/libkshark.h +59 -14

129 lines not on this page · raw message

Signature · 1 line
2.25.1
#44 lore raw

Re: [PATCH v2 20/20] kernel-shark: Integrate streams with libkshark-configio

Quoted 4 lines · On Mon, 12 Oct 2020 16:35:23 +0300 "Yordan Karadzhov (VMware)"…
src/libkshark-configio.c +2 -1
+int kshark_import_trace_file(struct kshark_context *kshark_ctx,
+			     struct kshark_config_doc *conf)
 {
-	const char *file = NULL;

src/libkshark-configio.c static bool kshark_trace_file_from_json(const char **file, · line 602

Quoted 10 lines
src/libkshark-configio.c +6 -2
+	const char *file = NULL, *name = NULL;
+	int sd = -1;
+
 	switch (conf->format) {
 	case KS_CONFIG_JSON:
-		if (kshark_trace_file_from_json(&file, conf->conf_doc))
-			kshark_open(kshark_ctx, file);
+		if (kshark_trace_file_from_json(&file, &name, "data",
+						conf->conf_doc)) {
+			if (strcmp(name, "top") == 0) {

Is "top" something that could be named by a user? That is, we could have a conflict if a buffer is called "top"?

If this is a special meaning, then we should probably make this something A user can not do, or even better, unlikely to have. And we shouldn't have it hardcoded as "top" anyway, it should be a macro.

#define KERNEL_SHARK_TOP_STREAM "/ / / __Top__ / / /"

Then everywhere use that.

Like in kshark_tep_init_input():

	stream->name = strdup(KERNEL_SHARK_TOP_STREAM);

And also replace "top" in this code with that macro as well.

-- Steve

Quoted 34 lines
src/libkshark-configio.c +32 -0
+				sd = kshark_open(kshark_ctx, file);
+			} else {
+				int sd_top;
+
+				sd_top = kshark_tep_find_top_stream(kshark_ctx,
+								    file);
+				if (sd_top < 0) {
+					/*
+					 * The "top" steam (buffer) has to be
+					 * initialized first.
+					 */
+					sd_top = kshark_open(kshark_ctx, file);
+				}
+
+				if (sd_top >= 0)
+					sd = kshark_tep_open_buffer(kshark_ctx,
+								    sd_top,
+								    name);
+
+				if (sd >= 0)
+				kshark_tep_handle_plugins(kshark_ctx, sd);
+			}
+		}
+
+		break;
 
+	default:
+		fprintf(stderr, "Document format %d not supported\n",
+			conf->format);
 		break;
+	}
+
+	return sd;
+}
#45 lore raw

Re: [PATCH v2 20/20] kernel-shark: Integrate streams with libkshark-configio

Quoted 28 lines · Steven Rostedt
Quoted 4 lines · On Mon, 12 Oct 2020 16:35:23 +0300 "Yordan Karadzhov (VMware)"…
src/libkshark-configio.c +2 -1
+int kshark_import_trace_file(struct kshark_context *kshark_ctx,
+			     struct kshark_config_doc *conf)
  {
-	const char *file = NULL;

src/libkshark-configio.c static bool kshark_trace_file_from_json(const char **file, · line 602

Quoted 10 lines
src/libkshark-configio.c +6 -2
+	const char *file = NULL, *name = NULL;
+	int sd = -1;
+
  	switch (conf->format) {
  	case KS_CONFIG_JSON:
-		if (kshark_trace_file_from_json(&file, conf->conf_doc))
-			kshark_open(kshark_ctx, file);
+		if (kshark_trace_file_from_json(&file, &name, "data",
+						conf->conf_doc)) {
+			if (strcmp(name, "top") == 0) {

Is "top" something that could be named by a user? That is, we could have a conflict if a buffer is called "top"?

If this is a special meaning, then we should probably make this something A user can not do, or even better, unlikely to have. And we shouldn't have it hardcoded as "top" anyway, it should be a macro.

#define KERNEL_SHARK_TOP_STREAM "/ / / __Top__ / / /"

Quoted 7 lines

Then everywhere use that.

Like in kshark_tep_init_input():

	stream->name = strdup(KERNEL_SHARK_TOP_STREAM);

And also replace "top" in this code with that macro as well.

Hi Steven,

In the review of patch 13/20 you already suggested to replace the name "top" with a non-printable character:

const char top_name[] = { 0x1b, 0x00 }; // Non printable character
#define TOP_NAME (char *)&top_name

This change will be included in v3.

Thanks!
Yordan

Quoted 38 lines

-- Steve

Quoted 34 lines
src/libkshark-configio.c +32 -0
+				sd = kshark_open(kshark_ctx, file);
+			} else {
+				int sd_top;
+
+				sd_top = kshark_tep_find_top_stream(kshark_ctx,
+								    file);
+				if (sd_top < 0) {
+					/*
+					 * The "top" steam (buffer) has to be
+					 * initialized first.
+					 */
+					sd_top = kshark_open(kshark_ctx, file);
+				}
+
+				if (sd_top >= 0)
+					sd = kshark_tep_open_buffer(kshark_ctx,
+								    sd_top,
+								    name);
+
+				if (sd >= 0)
+				kshark_tep_handle_plugins(kshark_ctx, sd);
+			}
+		}
+
+		break;
 
+	default:
+		fprintf(stderr, "Document format %d not supported\n",
+			conf->format);
  		break;
+	}
+
+	return sd;
+}
#46 lore raw

Re: [PATCH v2 20/20] kernel-shark: Integrate streams with libkshark-configio

Quoted 7 lines · On Mon, 9 Nov 2020 16:55:58 +0200 "Yordan Karadzhov (VMware)"…

Hi Steven,

In the review of patch 13/20 you already suggested to replace the name "top" with a non-printable character:

const char top_name[] = { 0x1b, 0x00 }; // Non printable character
#define TOP_NAME (char *)&top_name

Ah, I forgot ;-)

Quoted 2 lines

This change will be included in v3.

OK, looking forward to it. (Yes, I'm still on PTO today :-)

-- Steve