Skip to contentSkip to conversation

histograms: bug fixes and convention compliance

PATCH Superseded Stevie Alvarez 3 participants 17 messages Aug 3, 2023 – Aug 7, 2023 Kbuild

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover histograms: bug fixes and convention compliance

From: "Stevie Alvarez (Google)" <[email protected]>

Changes in v2:
* Renamed interface from histograms.h to traceeval-hist.h.
* Internal traceeval_type arrays terminated by size rather than type.
* type_alloc() guarantees traceeval_type name field is a valid pointer.
* traceeval_release() sets released pointers to NULL.
* Conditional simplifications and fixes across the board.
* Removed function stubs and their interface definitions.
* Added a timestamp flag.
* Added typedefs for traceeval_dynamic helper function signatures.
* Documentation follows established conventions (kernel docs, line comments).
* Documentation moved from interface to source code.
* Internal testing interface.
* General documentation spell checking.

Since the addition of the traceeval_dynamic function signature typedefs, warnings pertaining to these function signatures and traceeval_type have popped up. Steven, Ross, and anyone else, any ideas on why these warnings are showing up?

Warning from function signatures in include/traceeval-hist.h, line 65 and 69:
warning: ‘struct traceeval_type’ declared inside parameter list will not be visible outside of this definition or declaration

struct traceeval_type *);

Warning from dyn_release call in src/histograms.c line 258:
warning: passing argument 2 of
‘(defs + (sizetype)(i * 48))->dyn_release’ from incompatible pointer type

defs[i].dyn_release(data[i].dyn_data, &defs[i]);

note: expected ‘struct traceeval_type *’ but argument is of type ‘struct traceeval_type *’

Warning from dyn_cmp call in src/histograms.c line 409:

	warning: passing argument 3 of ‘type->dyn_cmp’ from incompatible
	pointer type

		return type->dyn_cmp(orig->dyn_data, copy->dyn_data, type);

	note: expected ‘struct traceeval_type *’ but argument is of type
	‘struct traceeval_type *’

---
v1 discussion: https://lore.kernel.org/linux-trace-devel/[email protected]/T/

Stevie Alvarez (Google) (5):
histograms: Initial histograms interface
histograms: Add traceeval initialize
histograms: Add traceeval release
histograms: Add traceeval compare
histograms: Initial unit tests

 Makefile                 |   2 +-
 include/traceeval-hist.h | 135 ++++++++++
 include/traceeval-test.h |  15 ++
 src/Makefile             |   1 +
 src/histograms.c         | 522 +++++++++++++++++++++++++++++++++++++++
 utest/.gitignore         |   1 +
 utest/Makefile           |  35 +++
 utest/eval-test.h        |  13 +
 utest/eval-utest.c       |  28 +++
 utest/traceeval-utest.c  | 217 ++++++++++++++++
 10 files changed, 968 insertions(+), 1 deletion(-)
 create mode 100644 include/traceeval-hist.h
 create mode 100644 include/traceeval-test.h
 create mode 100644 src/histograms.c
 create mode 100644 utest/.gitignore
 create mode 100644 utest/Makefile
 create mode 100644 utest/eval-test.h
 create mode 100644 utest/eval-utest.c
 create mode 100644 utest/traceeval-utest.c
Signature · 1 line
2.41.0
#1 lore raw

PATCH v2 1/5 histograms: Initial histograms interface

From: "Stevie Alvarez (Google)" <[email protected]>

Initial header file for libtraceeval's histogram API. The interface provides a simple way of aggregating trace data and reading through said data.

---
 include/traceeval-hist.h | 128 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 128 insertions(+)
 create mode 100644 include/traceeval-hist.h
include/traceeval-hist.h new file +128 -0
new file mode 100644
index 0000000..4664974
@@ -0,0 +1,128 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * libtraceeval histogram interface.
+ *
+ * Copyright (C) 2023 Google Inc, Steven Rostedt <[email protected]>
+ * Copyright (C) 2023 Google Inc, Stevie Alvarez <[email protected]>
+ */
+#ifndef __LIBTRACEEVAL_HIST_H__
+#define __LIBTRACEEVAL_HIST_H__
+
+#include <stdlib.h>
+#include <stddef.h>
+#include <stdbool.h>
+
+/* Data definition interfaces */
+
+/* Field name/descriptor for number of hits */
+#define TRACEEVAL_VAL_HITS ((const char *)(-1UL))
+
+/* Data type distinguishers */
+enum traceeval_data_type {
+	TRACEEVAL_TYPE_NONE,
+	TRACEEVAL_TYPE_STRING,
+	TRACEEVAL_TYPE_NUMBER,
+	TRACEEVAL_TYPE_NUMBER_64,
+	TRACEEVAL_TYPE_NUMBER_32,
+	TRACEEVAL_TYPE_NUMBER_16,
+	TRACEEVAL_TYPE_NUMBER_8,
+	TRACEEVAL_TYPE_DYNAMIC
+};
+
+/* Statistics specification flags */
+enum traceeval_flags {
+	TRACEEVAL_FL_SIGNED		= (1 << 0),
+	TRACEEVAL_FL_TIMESTAMP		= (1 << 1),
+	TRACEEVAL_FL_STATS		= (1 << 2)
+};
+
+/*
+ * Trace data entry for a traceeval histogram
+ * Constitutes keys and values.
+ */
+union traceeval_data {
+	char				*string;
+	struct traceeval_dynamic	*dyn_data;
+	unsigned long long		number_64;
+	unsigned long			number;
+	unsigned int			number_32;
+	unsigned short			number_16;
+	unsigned char			number_8;
+};
+
+/*
+ * struct traceeval_dynamic - Storage for dynamic traceeval_types
+ * @size: The size of the dynamic type
+ * @data: The pointer to the data of the dynamic type
+ */
+struct traceeval_dynamic {
+	void		*data;
+	size_t		size;
+};
+
+/* struct traceeval_dynamic release function signature */
+typedef void (*traceeval_dyn_release_fn)(struct traceeval_dynamic *,
+		struct traceeval_type *);
+
+/* struct traceeval_dynamic compare function signature */
+typedef int (*traceeval_dyn_cmp_fn)(struct traceeval_dynamic *,
+		struct traceeval_dynamic *, struct traceeval_type *);
+
+/*
+ * struct traceeval_type - Describes the type of a traceevent_data instance
+ * @type: The enum type that describes the traceeval_data
+ * @name: The string name of the traceeval_data
+ * @flags: flags to describe the traceeval_data
+ * @id: User specified identifier
+ * @dyn_release: For dynamic types called on release (ignored for other types)
+ * @dyn_cmp: A way to compare dynamic types (ignored for other types)
+ *
+ * The traceeval_type structure defines expectations for a corresponding
+ * traceeval_data instance for a traceeval histogram instance. Used to
+ * describe both keys and values.
+ *
+ * The @id field is an optional value in case the user has multiple struct
+ * traceeval_type instances with @type fields set to TRACEEVAL_TYPE_DYNAMIC,
+ * which each relate to distinct user defined struct traceeval_dynamic
+ * 'sub-types'.
+ *
+ * For flexibility, @dyn_cmp() and @dyn_release() take a struct
+ * traceeval_type instance. This allows the user to distinguish between
+ * different sub-types of struct traceeval_dynamic within a single
+ * callback function by examining the @id field. This is not a required
+ * approach, merely one that is accommodated.
+ *
+ * @dyn_cmp() is used to compare two struct traceeval_dynamic instances when a
+ * corresponding struct traceeval_type is reached with its type field set to
+ * TRACEEVAL_TYPE_DYNAMIC. It should return 0 on equality, 1 if the first
+ * argument is greater than the second, -1 for the other way around, and -2 on
+ * error.
+ *
+ * dyn_release() is used during traceeval_release() to release a union
+ * traceeval_data's struct traceeval_dynamic field when the corresponding
+ * traceeval_type type is set to TRACEEVAL_TYPE_DYNAMIC.
+ */
+struct traceeval_type {
+	char				*name;
+	traceeval_dyn_release_fn	dyn_release;
+	traceeval_dyn_cmp_fn		dyn_cmp;
+	enum traceeval_data_type	type;
+	size_t				flags;
+	size_t				id;
+};
+
+/* Statistics about a given entry element */
+struct traceeval_stat {
+	unsigned long long	max;
+	unsigned long long	min;
+	unsigned long long	total;
+	unsigned long long	avg;
+	unsigned long long	std;
+};
+
+/* Iterator over aggregated data */
+struct traceeval_iterator;
+
+struct traceeval;
+
+#endif /* __LIBTRACEEVAL_HIST_H__ */
Signature · 1 line
2.41.0
#2 lore raw

Re: [PATCH v2 1/5] histograms: Initial histograms interface

Quoted 36 lines · On Thu, 3 Aug 2023 18:53:59 -0400 Stevie Alvarez <[email protected]>…

From: "Stevie Alvarez (Google)" <[email protected]>

Initial header file for libtraceeval's histogram API. The interface provides a simple way of aggregating trace data and reading through said data.

---
 include/traceeval-hist.h | 128 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 128 insertions(+)
 create mode 100644 include/traceeval-hist.h
include/traceeval-hist.h new file +18 -0
new file mode 100644
index 0000000..4664974
@@ -0,0 +1,128 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * libtraceeval histogram interface.
+ *
+ * Copyright (C) 2023 Google Inc, Steven Rostedt <[email protected]>
+ * Copyright (C) 2023 Google Inc, Stevie Alvarez <[email protected]>
+ */
+#ifndef __LIBTRACEEVAL_HIST_H__
+#define __LIBTRACEEVAL_HIST_H__
+
+#include <stdlib.h>
+#include <stddef.h>
+#include <stdbool.h>
+
+/* Data definition interfaces */
+
+/* Field name/descriptor for number of hits */
+#define TRACEEVAL_VAL_HITS ((const char *)(-1UL))

include/traceeval-hist.h line 28

Quoted 10 lines
include/traceeval-hist.h new file +10 -0
+
+/* Data type distinguishers */
+enum traceeval_data_type {
+	TRACEEVAL_TYPE_NONE,
+	TRACEEVAL_TYPE_STRING,
+	TRACEEVAL_TYPE_NUMBER,
+	TRACEEVAL_TYPE_NUMBER_64,
+	TRACEEVAL_TYPE_NUMBER_32,
+	TRACEEVAL_TYPE_NUMBER_16,
+	TRACEEVAL_TYPE_NUMBER_8,

Hmm, I'm thinking it would be nice to rearrange this a bit to:

enum traceeval_data_type {
TRACEEVAL_TYPE_NONE,
TRACEEVAL_TYPE_NUMBER_8,
TRACEEVAL_TYPE_NUMBER_16,
TRACEEVAL_TYPE_NUMBER_32,
TRACEEVAL_TYPE_NUMBER_64,
TRACEEVAL_TYPE_NUMBER,
TRACEEVAL_TYPE_STRING,

That way NUMBER_8 will be 1, NUMBER_16 is 2, NUMBER_32 is 3 and NUMBER_64 is 4. If we ever wanted to do a trick, we can get the byte size from:

2^(1 * (enum - 1))

2^(1 * (NUMBER_8 - 1)) = 2^(1 * 0) = 1
2^(1 * (NUMBER_16 - 1)) = 2^(1 * 1) = 2
2^(1 * (NUMBER_32 - 1)) = 2^(1 * 2) = 4
2^(1 * (NUMBER_64 - 1)) = 2^(1 * 3) = 8

I love hacks!

include/traceeval-hist.h line 36

Quoted 8 lines
include/traceeval-hist.h new file +8 -0
+	TRACEEVAL_TYPE_DYNAMIC
+};
+
+/* Statistics specification flags */
+enum traceeval_flags {
+	TRACEEVAL_FL_SIGNED		= (1 << 0),
+	TRACEEVAL_FL_TIMESTAMP		= (1 << 1),
+	TRACEEVAL_FL_STATS		= (1 << 2)

We may as well remove STATS until we decided to use it.

Maybe even get rid of this enum until we decide to use it.

Remember, once it's exposed, it is API.

Quoted 16 lines
include/traceeval-hist.h new file +16 -0
+};
+
+/*
+ * Trace data entry for a traceeval histogram
+ * Constitutes keys and values.
+ */
+union traceeval_data {
+	char				*string;
+	struct traceeval_dynamic	*dyn_data;
+	unsigned long long		number_64;
+	unsigned long			number;
+	unsigned int			number_32;
+	unsigned short			number_16;
+	unsigned char			number_8;
+};
+

include/traceeval-hist.h line 62

Quoted 10 lines
include/traceeval-hist.h new file +10 -0
+/*
+ * struct traceeval_dynamic - Storage for dynamic traceeval_types
+ * @size: The size of the dynamic type
+ * @data: The pointer to the data of the dynamic type
+ */
+struct traceeval_dynamic {
+	void		*data;
+	size_t		size;
+};
+

As I replied to the cover letter, you need to add:

struct traceeval_type;

to get rid of the warning.

Quoted 39 lines
include/traceeval-hist.h new file +39 -0
+/* struct traceeval_dynamic release function signature */
+typedef void (*traceeval_dyn_release_fn)(struct traceeval_dynamic *,
+		struct traceeval_type *);
+
+/* struct traceeval_dynamic compare function signature */
+typedef int (*traceeval_dyn_cmp_fn)(struct traceeval_dynamic *,
+		struct traceeval_dynamic *, struct traceeval_type *);
+
+/*
+ * struct traceeval_type - Describes the type of a traceevent_data instance
+ * @type: The enum type that describes the traceeval_data
+ * @name: The string name of the traceeval_data
+ * @flags: flags to describe the traceeval_data
+ * @id: User specified identifier
+ * @dyn_release: For dynamic types called on release (ignored for other types)
+ * @dyn_cmp: A way to compare dynamic types (ignored for other types)
+ *
+ * The traceeval_type structure defines expectations for a corresponding
+ * traceeval_data instance for a traceeval histogram instance. Used to
+ * describe both keys and values.
+ *
+ * The @id field is an optional value in case the user has multiple struct
+ * traceeval_type instances with @type fields set to TRACEEVAL_TYPE_DYNAMIC,
+ * which each relate to distinct user defined struct traceeval_dynamic
+ * 'sub-types'.
+ *
+ * For flexibility, @dyn_cmp() and @dyn_release() take a struct
+ * traceeval_type instance. This allows the user to distinguish between
+ * different sub-types of struct traceeval_dynamic within a single
+ * callback function by examining the @id field. This is not a required
+ * approach, merely one that is accommodated.
+ *
+ * @dyn_cmp() is used to compare two struct traceeval_dynamic instances when a
+ * corresponding struct traceeval_type is reached with its type field set to
+ * TRACEEVAL_TYPE_DYNAMIC. It should return 0 on equality, 1 if the first
+ * argument is greater than the second, -1 for the other way around, and -2 on
+ * error.
+ *
+ * dyn_release() is used during traceeval_release() to release a union

include/traceeval-hist.h line 111

Quoted 10 lines
include/traceeval-hist.h new file +10 -0
+ * traceeval_data's struct traceeval_dynamic field when the corresponding
+ * traceeval_type type is set to TRACEEVAL_TYPE_DYNAMIC.
+ */
+struct traceeval_type {
+	char				*name;
+	traceeval_dyn_release_fn	dyn_release;
+	traceeval_dyn_cmp_fn		dyn_cmp;
+	enum traceeval_data_type	type;
+	size_t				flags;
+	size_t				id;

Let's reorder this a little. Normally function pointers come at the end of a structure. That's more of a guideline than a rule, but let's have it here.

struct traceeval_type {
	char				*name;
	enum traceeval_data_type	type;
	size_t				flags;
	size_t				id;
	traceeval_dyn_release_fn	dyn_release;
	traceeval_dyn_cmp_fn		dyn_cmp;
};

Especially since dynamic types are going to be rare, we don't want it in the hot cache.

-- Steve

Quoted 17 lines
include/traceeval-hist.h new file +17 -0
+};
+
+/* Statistics about a given entry element */
+struct traceeval_stat {
+	unsigned long long	max;
+	unsigned long long	min;
+	unsigned long long	total;
+	unsigned long long	avg;
+	unsigned long long	std;
+};
+
+/* Iterator over aggregated data */
+struct traceeval_iterator;
+
+struct traceeval;
+
+#endif /* __LIBTRACEEVAL_HIST_H__ */
#3 lore raw

Re: [PATCH v2 1/5] histograms: Initial histograms interface

Quoted 184 lines · Steven Rostedt
Quoted 36 lines · On Thu, 3 Aug 2023 18:53:59 -0400 Stevie Alvarez <[email protected]>…

From: "Stevie Alvarez (Google)" <[email protected]>

Initial header file for libtraceeval's histogram API. The interface provides a simple way of aggregating trace data and reading through said data.

---
 include/traceeval-hist.h | 128 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 128 insertions(+)
 create mode 100644 include/traceeval-hist.h
include/traceeval-hist.h new file +18 -0
new file mode 100644
index 0000000..4664974
@@ -0,0 +1,128 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * libtraceeval histogram interface.
+ *
+ * Copyright (C) 2023 Google Inc, Steven Rostedt <[email protected]>
+ * Copyright (C) 2023 Google Inc, Stevie Alvarez <[email protected]>
+ */
+#ifndef __LIBTRACEEVAL_HIST_H__
+#define __LIBTRACEEVAL_HIST_H__
+
+#include <stdlib.h>
+#include <stddef.h>
+#include <stdbool.h>
+
+/* Data definition interfaces */
+
+/* Field name/descriptor for number of hits */
+#define TRACEEVAL_VAL_HITS ((const char *)(-1UL))

include/traceeval-hist.h line 28

Quoted 10 lines
include/traceeval-hist.h new file +10 -0
+
+/* Data type distinguishers */
+enum traceeval_data_type {
+	TRACEEVAL_TYPE_NONE,
+	TRACEEVAL_TYPE_STRING,
+	TRACEEVAL_TYPE_NUMBER,
+	TRACEEVAL_TYPE_NUMBER_64,
+	TRACEEVAL_TYPE_NUMBER_32,
+	TRACEEVAL_TYPE_NUMBER_16,
+	TRACEEVAL_TYPE_NUMBER_8,

Hmm, I'm thinking it would be nice to rearrange this a bit to:

enum traceeval_data_type {
TRACEEVAL_TYPE_NONE,
TRACEEVAL_TYPE_NUMBER_8,
TRACEEVAL_TYPE_NUMBER_16,
TRACEEVAL_TYPE_NUMBER_32,
TRACEEVAL_TYPE_NUMBER_64,
TRACEEVAL_TYPE_NUMBER,
TRACEEVAL_TYPE_STRING,

That way NUMBER_8 will be 1, NUMBER_16 is 2, NUMBER_32 is 3 and NUMBER_64 is 4. If we ever wanted to do a trick, we can get the byte size from:

2^(1 * (enum - 1))

2^(1 * (NUMBER_8 - 1)) = 2^(1 * 0) = 1
2^(1 * (NUMBER_16 - 1)) = 2^(1 * 1) = 2
2^(1 * (NUMBER_32 - 1)) = 2^(1 * 2) = 4
2^(1 * (NUMBER_64 - 1)) = 2^(1 * 3) = 8

I love hacks!

include/traceeval-hist.h line 36

Quoted 8 lines
include/traceeval-hist.h new file +8 -0
+	TRACEEVAL_TYPE_DYNAMIC
+};
+
+/* Statistics specification flags */
+enum traceeval_flags {
+	TRACEEVAL_FL_SIGNED		= (1 << 0),
+	TRACEEVAL_FL_TIMESTAMP		= (1 << 1),
+	TRACEEVAL_FL_STATS		= (1 << 2)

We may as well remove STATS until we decided to use it.

Maybe even get rid of this enum until we decide to use it.

Remember, once it's exposed, it is API.

Quoted 16 lines
include/traceeval-hist.h new file +16 -0
+};
+
+/*
+ * Trace data entry for a traceeval histogram
+ * Constitutes keys and values.
+ */
+union traceeval_data {
+	char				*string;
+	struct traceeval_dynamic	*dyn_data;
+	unsigned long long		number_64;
+	unsigned long			number;
+	unsigned int			number_32;
+	unsigned short			number_16;
+	unsigned char			number_8;
+};
+

include/traceeval-hist.h line 62

Quoted 10 lines
include/traceeval-hist.h new file +10 -0
+/*
+ * struct traceeval_dynamic - Storage for dynamic traceeval_types
+ * @size: The size of the dynamic type
+ * @data: The pointer to the data of the dynamic type
+ */
+struct traceeval_dynamic {
+	void		*data;
+	size_t		size;
+};
+

As I replied to the cover letter, you need to add:

struct traceeval_type;

to get rid of the warning.

Quoted 39 lines
include/traceeval-hist.h new file +39 -0
+/* struct traceeval_dynamic release function signature */
+typedef void (*traceeval_dyn_release_fn)(struct traceeval_dynamic *,
+		struct traceeval_type *);
+
+/* struct traceeval_dynamic compare function signature */
+typedef int (*traceeval_dyn_cmp_fn)(struct traceeval_dynamic *,
+		struct traceeval_dynamic *, struct traceeval_type *);
+
+/*
+ * struct traceeval_type - Describes the type of a traceevent_data instance
+ * @type: The enum type that describes the traceeval_data
+ * @name: The string name of the traceeval_data
+ * @flags: flags to describe the traceeval_data
+ * @id: User specified identifier
+ * @dyn_release: For dynamic types called on release (ignored for other types)
+ * @dyn_cmp: A way to compare dynamic types (ignored for other types)
+ *
+ * The traceeval_type structure defines expectations for a corresponding
+ * traceeval_data instance for a traceeval histogram instance. Used to
+ * describe both keys and values.
+ *
+ * The @id field is an optional value in case the user has multiple struct
+ * traceeval_type instances with @type fields set to TRACEEVAL_TYPE_DYNAMIC,
+ * which each relate to distinct user defined struct traceeval_dynamic
+ * 'sub-types'.
+ *
+ * For flexibility, @dyn_cmp() and @dyn_release() take a struct
+ * traceeval_type instance. This allows the user to distinguish between
+ * different sub-types of struct traceeval_dynamic within a single
+ * callback function by examining the @id field. This is not a required
+ * approach, merely one that is accommodated.
+ *
+ * @dyn_cmp() is used to compare two struct traceeval_dynamic instances when a
+ * corresponding struct traceeval_type is reached with its type field set to
+ * TRACEEVAL_TYPE_DYNAMIC. It should return 0 on equality, 1 if the first
+ * argument is greater than the second, -1 for the other way around, and -2 on
+ * error.
+ *
+ * dyn_release() is used during traceeval_release() to release a union

include/traceeval-hist.h line 111

Quoted 10 lines
include/traceeval-hist.h new file +10 -0
+ * traceeval_data's struct traceeval_dynamic field when the corresponding
+ * traceeval_type type is set to TRACEEVAL_TYPE_DYNAMIC.
+ */
+struct traceeval_type {
+	char				*name;
+	traceeval_dyn_release_fn	dyn_release;
+	traceeval_dyn_cmp_fn		dyn_cmp;
+	enum traceeval_data_type	type;
+	size_t				flags;
+	size_t				id;

Let's reorder this a little. Normally function pointers come at the end of a structure. That's more of a guideline than a rule, but let's have it here.

struct traceeval_type {
	char				*name;
	enum traceeval_data_type	type;
	size_t				flags;
	size_t				id;
	traceeval_dyn_release_fn	dyn_release;
	traceeval_dyn_cmp_fn		dyn_cmp;
};
Quoted 2 lines

Especially since dynamic types are going to be rare, we don't want it in the hot cache.

Does the order of the fields in a struct definition not matter? I thought word-boundaries applied to struct definitions? Or does the compiler take care of this?

-- Stevie

Quoted 22 lines

-- Steve

Quoted 17 lines
include/traceeval-hist.h new file +17 -0
+};
+
+/* Statistics about a given entry element */
+struct traceeval_stat {
+	unsigned long long	max;
+	unsigned long long	min;
+	unsigned long long	total;
+	unsigned long long	avg;
+	unsigned long long	std;
+};
+
+/* Iterator over aggregated data */
+struct traceeval_iterator;
+
+struct traceeval;
+
+#endif /* __LIBTRACEEVAL_HIST_H__ */
#4 lore raw

Re: [PATCH v2 1/5] histograms: Initial histograms interface

Quoted 20 lines · On Fri, 4 Aug 2023 13:41:59 -0400 Stevie Alvarez <[email protected]>…
Quoted 20 lines

include/traceeval-hist.h line 111

Quoted 7 lines
include/traceeval-hist.h new file +7 -0
+struct traceeval_type {
+	char				*name;
+	traceeval_dyn_release_fn	dyn_release;
+	traceeval_dyn_cmp_fn		dyn_cmp;
+	enum traceeval_data_type	type;
+	size_t				flags;
+	size_t				id;  

Let's reorder this a little. Normally function pointers come at the end of a structure. That's more of a guideline than a rule, but let's have it here.

struct traceeval_type {
	char				*name;
	enum traceeval_data_type	type;
	size_t				flags;
	size_t				id;
	traceeval_dyn_release_fn	dyn_release;
	traceeval_dyn_cmp_fn		dyn_cmp;
};
Quoted 6 lines
Quoted 2 lines

Especially since dynamic types are going to be rare, we don't want it in the hot cache.

Does the order of the fields in a struct definition not matter? I thought word-boundaries applied to struct definitions? Or does the compiler take care of this?

They do matter. Word bounders are important, but the compiler will just make "holes" if needed. For example, let's say on 64 bit, everything above is 64 bits but the type. I would have created a "hole". But because the type is more important than the id, I kept it at the top.

struct traceeval_type {
	char				*name;			// offset 0

enum traceeval_data_type type; // offset 8

[ compiler adds 4 byte "hole" or "padding" ]

size_t flags; // offset 16
size_t id; // offset 24
traceeval_dyn_release_fn dyn_release; // offset 32
traceeval_dyn_cmp_fn dyn_cmp; // offset 40
};

If a cache line is 32 bytes (most is usually 128, but let's say on an older architecture) I don't care if the the dyn_release and dyn_cmp are in the same cache line as name.

-- Steve

#5 lore raw

Re: [PATCH v2 1/5] histograms: Initial histograms interface

Quoted 47 lines · Steven Rostedt
Quoted 20 lines · On Fri, 4 Aug 2023 13:41:59 -0400 Stevie Alvarez <[email protected]>…
Quoted 20 lines

include/traceeval-hist.h line 111

Quoted 7 lines
include/traceeval-hist.h new file +7 -0
+struct traceeval_type {
+	char				*name;
+	traceeval_dyn_release_fn	dyn_release;
+	traceeval_dyn_cmp_fn		dyn_cmp;
+	enum traceeval_data_type	type;
+	size_t				flags;
+	size_t				id;  

Let's reorder this a little. Normally function pointers come at the end of a structure. That's more of a guideline than a rule, but let's have it here.

struct traceeval_type {
	char				*name;
	enum traceeval_data_type	type;
	size_t				flags;
	size_t				id;
	traceeval_dyn_release_fn	dyn_release;
	traceeval_dyn_cmp_fn		dyn_cmp;
};
Quoted 6 lines
Quoted 2 lines

Especially since dynamic types are going to be rare, we don't want it in the hot cache.

Does the order of the fields in a struct definition not matter? I thought word-boundaries applied to struct definitions? Or does the compiler take care of this?

They do matter. Word bounders are important, but the compiler will just make "holes" if needed. For example, let's say on 64 bit, everything above is 64 bits but the type. I would have created a "hole". But because the type is more important than the id, I kept it at the top.

struct traceeval_type {
	char				*name;			// offset 0

enum traceeval_data_type type; // offset 8

[ compiler adds 4 byte "hole" or "padding" ]

(quoted diff) +0 -0
 	size_t				flags;			// offset 16
 	size_t				id;			// offset 24
 	traceeval_dyn_release_fn	dyn_release;		// offset 32
 	traceeval_dyn_cmp_fn		dyn_cmp;		// offset 40

};

Quoted 5 lines

If a cache line is 32 bytes (most is usually 128, but let's say on an older architecture) I don't care if the the dyn_release and dyn_cmp are in the same cache line as name.

-- Steve

This makes sense, I appreciate the explination! I was treating size_t as an int in my head by accident.... oops!

-- Stevie

#6 lore raw

PATCH v2 2/5 histograms: Add traceeval initialize

In reply to Stevie Alvarez

From: "Stevie Alvarez (Google)" <[email protected]>

traceeval_init() creates a new struct traceeval instance with regards to the struct traceeval_type array arguments keys and vals. These arrays define the structure of the histogram, with each describing the expected structure of inserted arrays of union traceeval_data. The keys and vals arguments are copied on the heap to ensure that the struct traceeval instance has access to the definition regardless of how the user initialized keys and vals.

---
 Makefile                 |   2 +-
 include/traceeval-hist.h |   5 +
 src/Makefile             |   1 +
 src/histograms.c         | 223 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 230 insertions(+), 1 deletion(-)
 create mode 100644 src/histograms.c
Makefile +1 -1
index 4a24d5a..3ea051c 100644
@@ -172,7 +172,7 @@ libs: $(LIBRARY_A) $(LIBRARY_SO)
 
 VALGRIND = $(shell which valgrind)
 UTEST_DIR = utest
-UTEST_BINARY = trace-utest
+UTEST_BINARY = eval-utest
 
 test: force $(LIBRARY_STATIC)
 ifneq ($(CUNIT_INSTALLED),1)
include/traceeval-hist.h +5 -0
index 4664974..5bea025 100644
@@ -125,4 +125,9 @@ struct traceeval_iterator;
 
 struct traceeval;
 
+/* Histogram interfaces */
+
+struct traceeval *traceeval_init(const struct traceeval_type *keys,
+		const struct traceeval_type *vals);
+
 #endif /* __LIBTRACEEVAL_HIST_H__ */
src/Makefile +1 -0
index b4b6e52..b32a389 100644
@@ -4,6 +4,7 @@ include $(src)/scripts/utils.mk
 
 OBJS =
 OBJS += trace-analysis.o
+OBJS += histograms.o
 
 OBJS := $(OBJS:%.o=$(bdir)/%.o)
 
src/histograms.c new file +223 -0

223 lines not on this page · raw message

Signature · 1 line
2.41.0
#7 lore raw

Re: [PATCH v2 2/5] histograms: Add traceeval initialize

Quoted 24 lines · On Thu, 3 Aug 2023 18:54:00 -0400 Stevie Alvarez <[email protected]>…

From: "Stevie Alvarez (Google)" <[email protected]>

traceeval_init() creates a new struct traceeval instance with regards to the struct traceeval_type array arguments keys and vals. These arrays define the structure of the histogram, with each describing the expected structure of inserted arrays of union traceeval_data. The keys and vals arguments are copied on the heap to ensure that the struct traceeval instance has access to the definition regardless of how the user initialized keys and vals.

---
 Makefile                 |   2 +-
 include/traceeval-hist.h |   5 +
 src/Makefile             |   1 +
 src/histograms.c         | 223 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 230 insertions(+), 1 deletion(-)
 create mode 100644 src/histograms.c
Makefile +0 -0
index 4a24d5a..3ea051c 100644

Makefile libs: $(LIBRARY_A) $(LIBRARY_SO) · line 175

Quoted 5 lines
Makefile +1 -1
@@ -172,7 +172,7 @@ libs: $(LIBRARY_A) $(LIBRARY_SO)
 
 VALGRIND = $(shell which valgrind)
 UTEST_DIR = utest
-UTEST_BINARY = trace-utest
+UTEST_BINARY = eval-utest

This seems like it doesn't belong in this patch.

Quoted 62 lines
Makefile +0 -0
 test: force $(LIBRARY_STATIC)
 ifneq ($(CUNIT_INSTALLED),1)
include/traceeval-hist.h +5 -0
index 4664974..5bea025 100644
@@ -125,4 +125,9 @@ struct traceeval_iterator;
 
 struct traceeval;
 
+/* Histogram interfaces */
+
+struct traceeval *traceeval_init(const struct traceeval_type *keys,
+		const struct traceeval_type *vals);
+
 #endif /* __LIBTRACEEVAL_HIST_H__ */
src/Makefile +1 -0
index b4b6e52..b32a389 100644
@@ -4,6 +4,7 @@ include $(src)/scripts/utils.mk
 
 OBJS =
 OBJS += trace-analysis.o
+OBJS += histograms.o
 
 OBJS := $(OBJS:%.o=$(bdir)/%.o)
 
src/histograms.c new file +27 -0
new file mode 100644
index 0000000..be17b89
@@ -0,0 +1,223 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * libtraceeval histogram interface implementation.
+ *
+ * Copyright (C) 2023 Google Inc, Stevie Alvarez <[email protected]>
+ */
+
+#include <stdbool.h>
+#include <string.h>
+#include <stdarg.h>
+#include <stdio.h>
+
+#include <traceeval-hist.h>
+
+/* A key-value pair */
+struct entry {
+	union traceeval_data	*keys;
+	union traceeval_data	*vals;
+};
+
+/* A table of key-value entries */
+struct hist_table {
+	struct entry	*map;
+	size_t		nr_entries;
+};
+
+/* Histogram */

src/histograms.c line 37

Quoted 10 lines
src/histograms.c new file +10 -0
+struct traceeval {
+	struct traceeval_type		*key_types;
+	struct traceeval_type		*val_types;
+	struct hist_table		*hist;
+	size_t				nr_key_types;
+	size_t				nr_val_types;
+};
+
+/* Iterate over results of histogram */
+struct traceeval_iterator {};

Add the traceeval_iterator struct when it is used.

It's not relevant to this patch.

Quoted 19 lines
src/histograms.c new file +19 -0
+
+/*
+ * print_err() - print an error message
+ * @fmt: String format
+ * @...: (optional) Additional arguments to print in conjunction with @format
+ */
+static void print_err(const char *fmt, ...)
+{
+	va_list ap;
+
+	va_start(ap, fmt);
+	vfprintf(stderr, fmt, ap);
+	va_end(ap);
+	fprintf(stderr, "\n");
+}
+
+/*
+ * type_release() - free a struct traceeval_type array
+ * @defs: The array to release

src/histograms.c line 66

Quoted 10 lines
src/histograms.c new file +10 -0
+ * @len: The length of @defs
+ *
+ * It is assumed that all elements of @defs, within the length of @len, have
+ * name fields initialized to valid pointers.
+ *
+ * This function was designed to be used on an array allocated by type_alloc().
+ * Note that type_alloc() initializes all name fields of elements within the
+ * returned size.
+ */
+static void type_release(struct traceeval_type *defs, size_t len)

You added this but did not add traceeval_release().

traceeval_release() should be added in this patch to clean up everything that traceeval_init() creates.

Quoted 39 lines
src/histograms.c new file +39 -0
+{
+	if (!defs)
+		return;
+
+	for (size_t i = 0; i < len; i++) {
+		free(defs[i].name);
+	}
+
+	free(defs);
+}
+
+/*
+ * type_alloc() - clone a struct traceeval_type array
+ * @defs: The original array
+ * @copy: A pointer to where to place the @defs copy
+ *
+ * Clone traceeval_type array @defs to the heap, and place in @copy.
+ * @defs must be terminated with an instance of type TRACEEVAL_TYPE_NONE.
+ *
+ * The size of the copy pointed to by @copy is returned. It counts all elements
+ * in @defs excluding the terminating TRACEEVAL_TYPE_NONE traceeval_type.
+ * The copy contains everything from @defs excluding the TRACEEVAL_TYPE_NONE
+ * traceeval_type.
+ *
+ * The name field of each element of @defs (except for the terminating
+ * TRACEEVAL_TYPE_NONE) must be a NULL-terminated string. The validity of the
+ * name field is not checked, but errors are returned upon finding unset name
+ * fields and string duplication failures. It is guaranteed that all elements
+ * of the copy within the returned size have valid pointers in their name
+ * fields.
+ *
+ * Returns the size of the array pointed to by @copy, or, if an error occurs,
+ * the negative of the size of what's been allocated so far.
+ * If the return value is negative, the user is responsible for freeing
+ * -1 * return value number of elements from @copy.
+ */
+static size_t type_alloc(const struct traceeval_type *defs,
+		struct traceeval_type **copy)
+{

src/histograms.c line 115

Quoted 10 lines
src/histograms.c new file +10 -0
+	struct traceeval_type *new_defs = NULL;
+	size_t size;
+	size_t i;
+
+	if (!defs) {
+		*copy = NULL;
+		return 0;
+	}
+
+	for (size = 0; defs && defs[size].type != TRACEEVAL_TYPE_NONE; size++);

For loops like the above, please do:

	for (size = 0; defs && defs[size].type != TRACEEVAL_TYPE_NONE; size++)
		;

This makes it obvious that it's an empty loop and we don't think it's looping over the line below it.

Quoted 5 lines
src/histograms.c new file +5 -0
+
+	if (size) {
+		new_defs = calloc(size, sizeof(*new_defs));
+	} else {
+		*copy = NULL;

src/histograms.c line 130

Quoted 10 lines
src/histograms.c new file +10 -0
+		return 0;
+	}
+
+	for (i = 0; i < size; i++) {
+		/* copy current def data to new_def */
+		new_defs[i] = defs[i];
+
+		/* copy name to heap, ensures name copied */
+		if (!defs[i].name)
+			goto fail_type_name;

No need to be so verbose in the labels.

			goto fail;

is good enough ;-)

src/histograms.c line 143

Quoted 13 lines
src/histograms.c new file +13 -0
+		new_defs[i].name = strdup(defs[i].name);
+
+		if (!new_defs[i].name)
+			goto fail_type_name;
+	}
+
+	*copy = new_defs;
+	return size;
+
+fail_type_name:
+	if (defs[size].name)
+		print_err("failed to allocate name for traceeval_type %s", defs[size].name);
+	print_err("failed to allocate name for traceeval_type index %zu", size);

I would change the above to:

	if (defs[i].name)
		print_err("Failed to allocate traceveal_type %zu", size);
	else
		print_err("traceeval_type list missing a name");

src/histograms.c line 145

Quoted 2 lines
src/histograms.c new file +2 -0
+	*copy = new_defs;
+	return i * -1;

Also, we need to clean up here and not pass that info back to the caller.

// need to make i: ssize_t i;

	for (; i >= 0; i--)
		free(new_defs[i].name);
	free(new_defs);
	*copy = NULL;
	return -1;
Quoted 26 lines
src/histograms.c new file +26 -0
+}
+
+/*
+ * traceeval_init - create a traceeval descriptor
+ * @keys: Defines the keys to differentiate traceeval entries
+ * @vals: Defines values attached to entries differentiated by @keys.
+ *
+ * The @keys and @vals define how the traceeval instance will be populated.
+ * The @keys will be used by traceeval_query() to find an instance within
+ * the "histogram". Note, both the @keys and @vals array must end with:
+ * { .type = TRACEEVAL_TYPE_NONE }.
+ *
+ * The @keys and @vals passed in are copied for internal use.
+ *
+ * For any member of @keys or @vals that isn't of type TRACEEVAL_TYPE_NONE,
+ * the name field must be a null-terminated string. For members of type
+ * TRACEEVAL_TYPE_NONE, the name is ignored.
+ *
+ * @vals can be NULL or start with its type field as TRACEEVAL_TYPE_NONE to
+ * define the values of the histogram to be empty.
+ * @keys must be populated with at least one element that is not
+ * TRACEEVAL_TYPE_NONE.
+ *
+ * Returns the descriptor on success, or NULL on error.
+ */
+struct traceeval *traceeval_init(const struct traceeval_type *keys,

src/histograms.c line 181

Quoted 10 lines
src/histograms.c new file +10 -0
+				 const struct traceeval_type *vals)
+{
+	struct traceeval *teval;
+	char *err_msg;
+
+	if (!keys)
+		return NULL;
+
+	if (keys->type == TRACEEVAL_TYPE_NONE) {
+		err_msg = "keys cannot start with type TRACEEVAL_TYPE_NONE";

BTW, all the error messages should start with a capital letter.

"Keys cannot .."

src/histograms.c line 182

Quoted 1 line
src/histograms.c new file +1 -0
+		goto fail_init_unalloced;

Just make this: goto fail;

src/histograms.c line 195

Quoted 13 lines
src/histograms.c new file +13 -0
+	}
+
+	/* alloc teval */
+	teval = calloc(1, sizeof(*teval));
+	if (!teval) {
+		err_msg = "failed to allocate memory for traceeval instance";
+		goto fail_init_unalloced;
+	}
+
+	/* alloc key types */
+	teval->nr_key_types = type_alloc(keys, &teval->key_types);
+	if (teval->nr_key_types <= 0) {
+		teval->nr_key_types *= -1;

Get rid of the above setting.

src/histograms.c line 197

Quoted 2 lines
src/histograms.c new file +2 -0
+		err_msg = "failed to allocate user defined keys";
+		goto fail_init;

and this: goto fail_release;

src/histograms.c line 203

Quoted 6 lines
src/histograms.c new file +6 -0
+	}
+
+	/* alloc val types */
+	teval->nr_val_types = type_alloc(vals, &teval->val_types);
+	if (teval->nr_val_types < 0) {
+		teval->nr_val_types *= -1;

And git rid of the above too.

Quoted 5 lines
src/histograms.c new file +5 -0
+		err_msg = "failed to allocate user defined values";
+		goto fail_init;
+	}
+
+	/* alloc hist */

src/histograms.c line 218

Quoted 10 lines
src/histograms.c new file +10 -0
+	teval->hist = calloc(1, sizeof(*teval->hist));
+	if (!teval->hist) {
+		err_msg = "failed to allocate memory for histogram";
+		goto fail_init;
+	}
+
+	return teval;
+
+fail_init:
+	traceeval_release(teval);

The above isn't defined. If you reference a function in a patch, it must be at least defined.

src/histograms.c line 223

Quoted 5 lines
src/histograms.c new file +5 -0
+
+fail_init_unalloced:
+	print_err(err_msg);
+	return NULL;
+}

-- Steve

#8 lore raw

Re: [PATCH v2 2/5] histograms: Add traceeval initialize

Quoted 301 lines · Steven Rostedt
Quoted 24 lines · On Thu, 3 Aug 2023 18:54:00 -0400 Stevie Alvarez <[email protected]>…

From: "Stevie Alvarez (Google)" <[email protected]>

traceeval_init() creates a new struct traceeval instance with regards to the struct traceeval_type array arguments keys and vals. These arrays define the structure of the histogram, with each describing the expected structure of inserted arrays of union traceeval_data. The keys and vals arguments are copied on the heap to ensure that the struct traceeval instance has access to the definition regardless of how the user initialized keys and vals.

---
 Makefile                 |   2 +-
 include/traceeval-hist.h |   5 +
 src/Makefile             |   1 +
 src/histograms.c         | 223 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 230 insertions(+), 1 deletion(-)
 create mode 100644 src/histograms.c
Makefile +0 -0
index 4a24d5a..3ea051c 100644

Makefile libs: $(LIBRARY_A) $(LIBRARY_SO) · line 175

Quoted 5 lines
Makefile +1 -1
@@ -172,7 +172,7 @@ libs: $(LIBRARY_A) $(LIBRARY_SO)
 
 VALGRIND = $(shell which valgrind)
 UTEST_DIR = utest
-UTEST_BINARY = trace-utest
+UTEST_BINARY = eval-utest

This seems like it doesn't belong in this patch.

Quoted 62 lines
Makefile +0 -0
 test: force $(LIBRARY_STATIC)
 ifneq ($(CUNIT_INSTALLED),1)
include/traceeval-hist.h +5 -0
index 4664974..5bea025 100644
@@ -125,4 +125,9 @@ struct traceeval_iterator;
 
 struct traceeval;
 
+/* Histogram interfaces */
+
+struct traceeval *traceeval_init(const struct traceeval_type *keys,
+		const struct traceeval_type *vals);
+
 #endif /* __LIBTRACEEVAL_HIST_H__ */
src/Makefile +1 -0
index b4b6e52..b32a389 100644
@@ -4,6 +4,7 @@ include $(src)/scripts/utils.mk
 
 OBJS =
 OBJS += trace-analysis.o
+OBJS += histograms.o
 
 OBJS := $(OBJS:%.o=$(bdir)/%.o)
 
src/histograms.c new file +27 -0
new file mode 100644
index 0000000..be17b89
@@ -0,0 +1,223 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * libtraceeval histogram interface implementation.
+ *
+ * Copyright (C) 2023 Google Inc, Stevie Alvarez <[email protected]>
+ */
+
+#include <stdbool.h>
+#include <string.h>
+#include <stdarg.h>
+#include <stdio.h>
+
+#include <traceeval-hist.h>
+
+/* A key-value pair */
+struct entry {
+	union traceeval_data	*keys;
+	union traceeval_data	*vals;
+};
+
+/* A table of key-value entries */
+struct hist_table {
+	struct entry	*map;
+	size_t		nr_entries;
+};
+
+/* Histogram */

src/histograms.c line 37

Quoted 10 lines
src/histograms.c new file +10 -0
+struct traceeval {
+	struct traceeval_type		*key_types;
+	struct traceeval_type		*val_types;
+	struct hist_table		*hist;
+	size_t				nr_key_types;
+	size_t				nr_val_types;
+};
+
+/* Iterate over results of histogram */
+struct traceeval_iterator {};

Add the traceeval_iterator struct when it is used.

It's not relevant to this patch.

Quoted 19 lines
src/histograms.c new file +19 -0
+
+/*
+ * print_err() - print an error message
+ * @fmt: String format
+ * @...: (optional) Additional arguments to print in conjunction with @format
+ */
+static void print_err(const char *fmt, ...)
+{
+	va_list ap;
+
+	va_start(ap, fmt);
+	vfprintf(stderr, fmt, ap);
+	va_end(ap);
+	fprintf(stderr, "\n");
+}
+
+/*
+ * type_release() - free a struct traceeval_type array
+ * @defs: The array to release

src/histograms.c line 66

Quoted 10 lines
src/histograms.c new file +10 -0
+ * @len: The length of @defs
+ *
+ * It is assumed that all elements of @defs, within the length of @len, have
+ * name fields initialized to valid pointers.
+ *
+ * This function was designed to be used on an array allocated by type_alloc().
+ * Note that type_alloc() initializes all name fields of elements within the
+ * returned size.
+ */
+static void type_release(struct traceeval_type *defs, size_t len)

You added this but did not add traceeval_release().

traceeval_release() should be added in this patch to clean up everything that traceeval_init() creates.

Quoted 39 lines
src/histograms.c new file +39 -0
+{
+	if (!defs)
+		return;
+
+	for (size_t i = 0; i < len; i++) {
+		free(defs[i].name);
+	}
+
+	free(defs);
+}
+
+/*
+ * type_alloc() - clone a struct traceeval_type array
+ * @defs: The original array
+ * @copy: A pointer to where to place the @defs copy
+ *
+ * Clone traceeval_type array @defs to the heap, and place in @copy.
+ * @defs must be terminated with an instance of type TRACEEVAL_TYPE_NONE.
+ *
+ * The size of the copy pointed to by @copy is returned. It counts all elements
+ * in @defs excluding the terminating TRACEEVAL_TYPE_NONE traceeval_type.
+ * The copy contains everything from @defs excluding the TRACEEVAL_TYPE_NONE
+ * traceeval_type.
+ *
+ * The name field of each element of @defs (except for the terminating
+ * TRACEEVAL_TYPE_NONE) must be a NULL-terminated string. The validity of the
+ * name field is not checked, but errors are returned upon finding unset name
+ * fields and string duplication failures. It is guaranteed that all elements
+ * of the copy within the returned size have valid pointers in their name
+ * fields.
+ *
+ * Returns the size of the array pointed to by @copy, or, if an error occurs,
+ * the negative of the size of what's been allocated so far.
+ * If the return value is negative, the user is responsible for freeing
+ * -1 * return value number of elements from @copy.
+ */
+static size_t type_alloc(const struct traceeval_type *defs,
+		struct traceeval_type **copy)
+{

src/histograms.c line 115

Quoted 10 lines
src/histograms.c new file +10 -0
+	struct traceeval_type *new_defs = NULL;
+	size_t size;
+	size_t i;
+
+	if (!defs) {
+		*copy = NULL;
+		return 0;
+	}
+
+	for (size = 0; defs && defs[size].type != TRACEEVAL_TYPE_NONE; size++);

For loops like the above, please do:

	for (size = 0; defs && defs[size].type != TRACEEVAL_TYPE_NONE; size++)
		;

This makes it obvious that it's an empty loop and we don't think it's looping over the line below it.

Quoted 5 lines
src/histograms.c new file +5 -0
+
+	if (size) {
+		new_defs = calloc(size, sizeof(*new_defs));
+	} else {
+		*copy = NULL;

src/histograms.c line 130

Quoted 10 lines
src/histograms.c new file +10 -0
+		return 0;
+	}
+
+	for (i = 0; i < size; i++) {
+		/* copy current def data to new_def */
+		new_defs[i] = defs[i];
+
+		/* copy name to heap, ensures name copied */
+		if (!defs[i].name)
+			goto fail_type_name;

No need to be so verbose in the labels.

			goto fail;

is good enough ;-)

src/histograms.c line 143

Quoted 13 lines
src/histograms.c new file +13 -0
+		new_defs[i].name = strdup(defs[i].name);
+
+		if (!new_defs[i].name)
+			goto fail_type_name;
+	}
+
+	*copy = new_defs;
+	return size;
+
+fail_type_name:
+	if (defs[size].name)
+		print_err("failed to allocate name for traceeval_type %s", defs[size].name);
+	print_err("failed to allocate name for traceeval_type index %zu", size);

I would change the above to:

	if (defs[i].name)
		print_err("Failed to allocate traceveal_type %zu", size);
	else
		print_err("traceeval_type list missing a name");

src/histograms.c line 145

Quoted 2 lines
src/histograms.c new file +2 -0
+	*copy = new_defs;
+	return i * -1;

Also, we need to clean up here and not pass that info back to the caller.

// need to make i: ssize_t i;

	for (; i >= 0; i--)
		free(new_defs[i].name);
	free(new_defs);
	*copy = NULL;
	return -1;
Quoted 26 lines
src/histograms.c new file +26 -0
+}
+
+/*
+ * traceeval_init - create a traceeval descriptor
+ * @keys: Defines the keys to differentiate traceeval entries
+ * @vals: Defines values attached to entries differentiated by @keys.
+ *
+ * The @keys and @vals define how the traceeval instance will be populated.
+ * The @keys will be used by traceeval_query() to find an instance within
+ * the "histogram". Note, both the @keys and @vals array must end with:
+ * { .type = TRACEEVAL_TYPE_NONE }.
+ *
+ * The @keys and @vals passed in are copied for internal use.
+ *
+ * For any member of @keys or @vals that isn't of type TRACEEVAL_TYPE_NONE,
+ * the name field must be a null-terminated string. For members of type
+ * TRACEEVAL_TYPE_NONE, the name is ignored.
+ *
+ * @vals can be NULL or start with its type field as TRACEEVAL_TYPE_NONE to
+ * define the values of the histogram to be empty.
+ * @keys must be populated with at least one element that is not
+ * TRACEEVAL_TYPE_NONE.
+ *
+ * Returns the descriptor on success, or NULL on error.
+ */
+struct traceeval *traceeval_init(const struct traceeval_type *keys,

src/histograms.c line 181

Quoted 10 lines
src/histograms.c new file +10 -0
+				 const struct traceeval_type *vals)
+{
+	struct traceeval *teval;
+	char *err_msg;
+
+	if (!keys)
+		return NULL;
+
+	if (keys->type == TRACEEVAL_TYPE_NONE) {
+		err_msg = "keys cannot start with type TRACEEVAL_TYPE_NONE";

BTW, all the error messages should start with a capital letter.

Quoted 5 lines

"Keys cannot .."

src/histograms.c line 182

Quoted 1 line
src/histograms.c new file +1 -0
+		goto fail_init_unalloced;

Just make this: goto fail;

Having multiple of the same label segfaults my tests. I'm relatively new to using C labels, is that behavior expected? And if so is it acceptable for me to distingush the two fail labels by a single character?

-- Stevie

Quoted 57 lines
Quoted 4 lines
src/histograms.c new file +3 -0
+	}
+
+	/* alloc teval */

src/histograms.c line 195

Quoted 10 lines
src/histograms.c new file +10 -0
+	teval = calloc(1, sizeof(*teval));
+	if (!teval) {
+		err_msg = "failed to allocate memory for traceeval instance";
+		goto fail_init_unalloced;
+	}
+
+	/* alloc key types */
+	teval->nr_key_types = type_alloc(keys, &teval->key_types);
+	if (teval->nr_key_types <= 0) {
+		teval->nr_key_types *= -1;

Get rid of the above setting.

src/histograms.c line 197

Quoted 2 lines
src/histograms.c new file +2 -0
+		err_msg = "failed to allocate user defined keys";
+		goto fail_init;

and this: goto fail_release;

src/histograms.c line 203

Quoted 6 lines
src/histograms.c new file +6 -0
+	}
+
+	/* alloc val types */
+	teval->nr_val_types = type_alloc(vals, &teval->val_types);
+	if (teval->nr_val_types < 0) {
+		teval->nr_val_types *= -1;

And git rid of the above too.

Quoted 5 lines
src/histograms.c new file +5 -0
+		err_msg = "failed to allocate user defined values";
+		goto fail_init;
+	}
+
+	/* alloc hist */

src/histograms.c line 218

Quoted 10 lines
src/histograms.c new file +10 -0
+	teval->hist = calloc(1, sizeof(*teval->hist));
+	if (!teval->hist) {
+		err_msg = "failed to allocate memory for histogram";
+		goto fail_init;
+	}
+
+	return teval;
+
+fail_init:
+	traceeval_release(teval);

The above isn't defined. If you reference a function in a patch, it must be at least defined.

src/histograms.c line 223

Quoted 5 lines
src/histograms.c new file +5 -0
+
+fail_init_unalloced:
+	print_err(err_msg);
+	return NULL;
+}

-- Steve

#9 lore raw

Re: [PATCH v2 2/5] histograms: Add traceeval initialize

Quoted 38 lines · On Fri, 4 Aug 2023 14:23:11 -0400 Stevie Alvarez <[email protected]>…
Quoted 38 lines
Quoted 23 lines

+ * traceeval_init - create a traceeval descriptor
+ * @keys: Defines the keys to differentiate traceeval entries
+ * @vals: Defines values attached to entries differentiated by @keys.
+ *
+ * The @keys and @vals define how the traceeval instance will be populated.
+ * The @keys will be used by traceeval_query() to find an instance within
+ * the "histogram". Note, both the @keys and @vals array must end with:
+ * { .type = TRACEEVAL_TYPE_NONE }.
+ *
+ * The @keys and @vals passed in are copied for internal use.
+ *
+ * For any member of @keys or @vals that isn't of type TRACEEVAL_TYPE_NONE,
+ * the name field must be a null-terminated string. For members of type
+ * TRACEEVAL_TYPE_NONE, the name is ignored.
+ *
+ * @vals can be NULL or start with its type field as TRACEEVAL_TYPE_NONE to
+ * define the values of the histogram to be empty.
+ * @keys must be populated with at least one element that is not
+ * TRACEEVAL_TYPE_NONE.
+ *
+ * Returns the descriptor on success, or NULL on error.
+ */

src/histograms.c new file +1 -0
+struct traceeval *traceeval_init(const struct traceeval_type *keys,

src/histograms.c line 181

Quoted 10 lines
src/histograms.c new file +10 -0
+				 const struct traceeval_type *vals)
+{
+	struct traceeval *teval;
+	char *err_msg;
+
+	if (!keys)
+		return NULL;
+
+	if (keys->type == TRACEEVAL_TYPE_NONE) {
+		err_msg = "keys cannot start with type TRACEEVAL_TYPE_NONE";  

BTW, all the error messages should start with a capital letter.

"Keys cannot .."

Quoted 7 lines
Quoted 3 lines

Review

Quoted 1 line
(quoted diff) +1 -0
+		goto fail_init_unalloced;  

Just make this: goto fail;

Having multiple of the same label segfaults my tests. I'm relatively new to using C labels, is that behavior expected? And if so is it acceptable for me to distingush the two fail labels by a single character?

labels must be unique per function. But I constantly use the same label multiple times per C file.

As I stated in the email, convert one to "fail:" and the other one to "fail_release:"

Quoted 34 lines

-- Stevie

Quoted 31 lines
Quoted 4 lines
src/histograms.c new file +3 -0
+	}
+
+	/* alloc teval */

src/histograms.c line 195

Quoted 10 lines
src/histograms.c new file +10 -0
+	teval = calloc(1, sizeof(*teval));
+	if (!teval) {
+		err_msg = "failed to allocate memory for traceeval instance";
+		goto fail_init_unalloced;
+	}
+
+	/* alloc key types */
+	teval->nr_key_types = type_alloc(keys, &teval->key_types);
+	if (teval->nr_key_types <= 0) {
+		teval->nr_key_types *= -1;  

Get rid of the above setting.

Review

Quoted 2 lines
(quoted diff) +2 -0
+		err_msg = "failed to allocate user defined keys";
+		goto fail_init;  

and this: goto fail_release;

src/histograms.c line 203

Quoted 6 lines
src/histograms.c new file +6 -0
+	}
+
+	/* alloc val types */
+	teval->nr_val_types = type_alloc(vals, &teval->val_types);
+	if (teval->nr_val_types < 0) {
+		teval->nr_val_types *= -1;  

And git rid of the above too.

Quoted 14 lines
Quoted 6 lines
Quoted 6 lines
src/histograms.c new file +6 -0
+		err_msg = "failed to allocate user defined values";
+		goto fail_init;
+	}
+
+	/* alloc hist */
+	teval->hist = calloc(1, sizeof(*teval->hist));
Quoted 8 lines

src/histograms.c line 217

Quoted 8 lines
src/histograms.c new file +8 -0
+	if (!teval->hist) {
+		err_msg = "failed to allocate memory for histogram";
+		goto fail_init;
+	}
+
+	return teval;
+
+fail_init:

This would be

fail_release:

as it releases the teval.

Quoted 7 lines
Quoted 7 lines

src/histograms.c line 218

Quoted 1 line
src/histograms.c new file +1 -0
+	traceeval_release(teval);  

The above isn't defined. If you reference a function in a patch, it must be at least defined.

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+fail_init_unalloced:

and this one be

fail:

As it is part of the fail path for both.

-- Steve

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
src/histograms.c new file +3 -0
+	print_err(err_msg);
+	return NULL;
+}  
#10 lore raw

PATCH v2 3/5 histograms: Add traceeval release

In reply to Stevie Alvarez

From: "Stevie Alvarez (Google)" <[email protected]>

traceeval_release() deconstructs a given struct traceeval instance. It frees any data allocated to the heap within the entries to the histogram, and the names allocated for the struct traceeval_type key-value definitions.

---
 include/traceeval-hist.h |  2 +
 src/histograms.c         | 84 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 86 insertions(+)
include/traceeval-hist.h +2 -0
index 5bea025..cd74d3e 100644
@@ -130,4 +130,6 @@ struct traceeval;
 struct traceeval *traceeval_init(const struct traceeval_type *keys,
 		const struct traceeval_type *vals);
 
+void traceeval_release(struct traceeval *teval);
+
 #endif /* __LIBTRACEEVAL_HIST_H__ */
src/histograms.c +84 -0

87 lines not on this page · raw message

Signature · 1 line
2.41.0
#11 lore raw

Re: [PATCH v2 3/5] histograms: Add traceeval release

Quoted 7 lines · On Thu, 3 Aug 2023 18:54:01 -0400 Stevie Alvarez <[email protected]>…

From: "Stevie Alvarez (Google)" <[email protected]>

traceeval_release() deconstructs a given struct traceeval instance. It frees any data allocated to the heap within the entries to the histogram, and the names allocated for the struct traceeval_type key-value definitions.

As this is referenced in the first patch, please fold this patch into the first one. It's fine to add both the init and the releaese (alloc and free) functions in one patch. In fact, they should be together because those functions are tightly coupled.

The idea is that every patch should do "one thing", but an alloc should always be accompanied by a free. As a whole, it's "one thing", and OK to be in one patch/commit.

It also makes it easier to review. When they are separate patches, I can't go and look to see if something allocated was later freed. Having them as two patches makes it more likely to introduce an error.

Quoted 26 lines
---
 include/traceeval-hist.h |  2 +
 src/histograms.c         | 84 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 86 insertions(+)
include/traceeval-hist.h +2 -0
index 5bea025..cd74d3e 100644
@@ -130,4 +130,6 @@ struct traceeval;
 struct traceeval *traceeval_init(const struct traceeval_type *keys,
 		const struct traceeval_type *vals);
 
+void traceeval_release(struct traceeval *teval);
+
 #endif /* __LIBTRACEEVAL_HIST_H__ */
src/histograms.c +1 -0
index be17b89..95243b0 100644
@@ -221,3 +221,87 @@ fail_init_unalloced:
 	print_err(err_msg);
 	return NULL;
 }
+

src/histograms.c fail_init_unalloced: · line 234

Quoted 10 lines
src/histograms.c +10 -0
+/*
+ * Free any specified dynamic data in @data.
+ */
+static void clean_data(union traceeval_data *data, struct traceeval_type *defs,
+		size_t size)
+{
+	size_t i;
+
+	if (!data || !defs)
+		return;

Perhaps the above should have:

	if (!data || !defs) {
		if (data)
			print_err("Data to be freed without accompanied types!");
	}

src/histograms.c fail_init_unalloced: · line 243

Quoted 9 lines
src/histograms.c +9 -0
+
+	for (i = 0; i < size; i++) {
+		switch (defs[i].type) {
+		case TRACEEVAL_TYPE_STRING:
+			free(data[i].string);
+			break;
+		case TRACEEVAL_TYPE_DYNAMIC:
+			if (defs[i].dyn_release)
+				defs[i].dyn_release(data[i].dyn_data, &defs[i]);

I just realized, you should make the release function define as:

typedef void (*traceeval_dyn_release_fn)(struct traceeval_type *,

					 struct traceeval_dynamic *);

As the function is really a method of the struct typeeval_type. And the usually way is to have the "this" pointer as the first parameter of the method function. The traceeval_type is the "this" in this case.

Also, for all functions, the parameters should line up with the open parenthesis.

Instead of:

static size_t type_alloc(const struct traceeval_type *defs,
struct traceeval_type **copy)

It should always be:

static size_t type_alloc(const struct traceeval_type *defs,
struct traceeval_type **copy)

See how easier it is to read the function name, and differentiate it from the parameters.

Same for the typedef (notice how I lined it up above?). It makes it easier to see what the name of the function type is.

-- Steve

Quoted 64 lines
src/histograms.c +64 -0
+			break;
+		default:
+			break;
+		}
+	}
+}
+
+/*
+ * Free all possible data stored within the entry.
+ */
+static void clean_entry(struct entry *entry, struct traceeval *teval)
+{
+	if (!entry)
+		return;
+
+	/* free dynamic traceeval_data */
+	clean_data(entry->keys, teval->key_types, teval->nr_key_types);
+	clean_data(entry->vals, teval->val_types, teval->nr_val_types);
+	free(entry->keys);
+	free(entry->vals);
+}
+
+/*
+ * Free the hist_table allocated to a traceeval instance.
+ */
+static void hist_table_release(struct traceeval *teval)
+{
+	struct hist_table *hist = teval->hist;
+
+	if (!hist)
+		return;
+
+	for (size_t i = 0; i < hist->nr_entries; i++) {
+		clean_entry(&hist->map[i], teval);
+	}
+
+	free(hist->map);
+	free(hist);
+	teval->hist = NULL;
+}
+
+/*
+ * traceeval_release - release a traceeval descriptor
+ * @teval: An instance of traceeval returned by traceeval_init()
+ *
+ * When the caller of traceeval_init() is done with the returned @teval,
+ * it must call traceeval_release().
+ *
+ * This frees all internally allocated data of @teval and will call the
+ * corresponding dyn_release() functions registered for keys and values of
+ * type TRACEEVAL_TYPE_DYNAMIC.
+ */
+void traceeval_release(struct traceeval *teval)
+{
+	if (!teval)
+		return;
+
+	hist_table_release(teval);
+	type_release(teval->key_types, teval->nr_key_types);
+	type_release(teval->val_types, teval->nr_val_types);
+	teval->key_types = NULL;
+	teval->val_types = NULL;
+	free(teval);
+}
#12 lore raw

PATCH v2 4/5 histograms: Add traceeval compare

In reply to Stevie Alvarez

From: "Stevie Alvarez (Google)" <[email protected]>

traceeval_compare() compares two struct traceeval instances for equality. This suite of comparitors was made for testing purposes.

---
 include/traceeval-test.h |  16 +++
 src/histograms.c         | 221 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 234 insertions(+), 3 deletions(-)
 create mode 100644 include/traceeval-test.h
include/traceeval-test.h new file +16 -0
new file mode 100644
index 0000000..bb8092a
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * libtraceeval interface for unit testing.
+ *
+ * Copyright (C) 2023 Google Inc, Steven Rostedt <[email protected]>
+ * Copyright (C) 2023 Google Inc, Stevie Alvarez <[email protected]>
+ */
+
+#ifndef __LIBTRACEEVAL_TEST_H__
+#define __LIBTRACEEVAL_TEST_H__
+
+#include <traceeval-hist.h>
+
+int traceeval_compare(struct traceeval *orig, struct traceeval *copy);
+
+#endif /* __LIBTRACEEVAL_TEST_H__ */
src/histograms.c +218 -3

239 lines not on this page · raw message

Signature · 1 line
2.41.0
#13 lore raw

Re: [PATCH v2 4/5] histograms: Add traceeval compare

Quoted 18 lines · On Thu, 3 Aug 2023 18:54:02 -0400 Stevie Alvarez <[email protected]>…

From: "Stevie Alvarez (Google)" <[email protected]>

traceeval_compare() compares two struct traceeval instances for equality. This suite of comparitors was made for testing purposes.

---
 include/traceeval-test.h |  16 +++
 src/histograms.c         | 221 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 234 insertions(+), 3 deletions(-)
 create mode 100644 include/traceeval-test.h
include/traceeval-test.h new file +0 -0
new file mode 100644
index 0000000..bb8092a

include/traceeval-test.h line 5

Quoted 5 lines
include/traceeval-test.h new file +5 -0
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * libtraceeval interface for unit testing.
+ *
+ * Copyright (C) 2023 Google Inc, Steven Rostedt <[email protected]>

I noticed that you deleted the above line in the next patch, it just shouldn't be added in this one.

Also, it shouldn't be in the include/ directory, as we don't want to add this to the system on install.

This is just for testing, let's keep the function in the utest/ directory, as well as the header file.

Feel free to add a file there to store this.

-- Steve

Quoted 257 lines
include/traceeval-test.h new file +11 -0
+ * Copyright (C) 2023 Google Inc, Stevie Alvarez <[email protected]>
+ */
+
+#ifndef __LIBTRACEEVAL_TEST_H__
+#define __LIBTRACEEVAL_TEST_H__
+
+#include <traceeval-hist.h>
+
+int traceeval_compare(struct traceeval *orig, struct traceeval *copy);
+
+#endif /* __LIBTRACEEVAL_TEST_H__ */
src/histograms.c +218 -3
index 95243b0..8094678 100644
@@ -11,6 +11,20 @@
 #include <stdio.h>
 
 #include <traceeval-hist.h>
+#include <traceeval-test.h>
+
+/*
+ * Compare two integers of variable length.
+ *
+ * Return 0 if @a and @b are the same, 1 if @a is greater than @b, and -1
+ * if @b is greater than @a.
+ */
+#define compare_numbers_return(a, b)	\
+do {					\
+	if ((a) < (b))			\
+		return -1;		\
+	return (a) != (b);		\
+} while (0)				\
 
 /* A key-value pair */
 struct entry {
@@ -158,12 +172,13 @@ fail_type_name:
  * The @keys and @vals passed in are copied for internal use.
  *
  * For any member of @keys or @vals that isn't of type TRACEEVAL_TYPE_NONE,
- * the name field must be a null-terminated string. For members of type
- * TRACEEVAL_TYPE_NONE, the name is ignored.
+ * the name field must be a null-terminated string. Members of type
+ * TRACEEVAL_TYPE_NONE are used to terminate the array, therefore their other
+ * fields are ignored.
  *
  * @vals can be NULL or start with its type field as TRACEEVAL_TYPE_NONE to
  * define the values of the histogram to be empty.
- * @keys must be populated with at least one element that is not
+ * @keys must be populated with at least one element that is not of type
  * TRACEEVAL_TYPE_NONE.
  *
  * Returns the descriptor on success, or NULL on error.
@@ -305,3 +320,203 @@ void traceeval_release(struct traceeval *teval)
 	teval->val_types = NULL;
 	free(teval);
 }
+
+/*
+ * Compare traceeval_type instances for equality.
+ *
+ * Return 0 if @orig and @copy are the same, 1 otherwise.
+ */
+static int compare_traceeval_type(struct traceeval_type *orig,
+		struct traceeval_type *copy, size_t orig_size, size_t copy_size)
+{
+	size_t i;
+
+	/* same memory/NULL */
+	if (orig == copy)
+		return 0;
+	if (!!orig != !!copy)
+		return 1;
+
+	if (orig_size != copy_size)
+		return 1;
+
+	for (i = 0; i < orig_size; i++) {
+		if (orig[i].type != copy[i].type)
+			return 1;
+		if (orig[i].flags != copy[i].flags)
+			return 1;
+		if (orig[i].id != copy[i].id)
+			return 1;
+		if (orig[i].dyn_release != copy[i].dyn_release)
+			return 1;
+		if (orig[i].dyn_cmp != copy[i].dyn_cmp)
+			return 1;
+
+		// make sure both names are same type
+		if (!!orig[i].name != !!copy[i].name)
+			return 1;
+		if (!orig[i].name)
+			continue;
+		if (strcmp(orig[i].name, copy[i].name) != 0)
+			return 1;
+	}
+
+	return 0;
+}
+
+/*
+ * Compare traceeval_data instances.
+ *
+ * Return 0 if @orig and @copy are the same, 1 if @orig is greater than @copy,
+ * -1 for the other way around, and -2 on error.
+ */
+static int compare_traceeval_data(union traceeval_data *orig,
+		const union traceeval_data *copy, struct traceeval_type *type)
+{
+	int i;
+
+	if (orig == copy)
+		return 0;
+
+	if (!orig)
+		return -1;
+
+	if (!copy)
+		return 1;
+
+	switch (type->type) {
+	case TRACEEVAL_TYPE_STRING:
+		i = strcmp(orig->string, copy->string);
+		compare_numbers_return(i, 0);
+
+	case TRACEEVAL_TYPE_NUMBER:
+		compare_numbers_return(orig->number, copy->number);
+
+	case TRACEEVAL_TYPE_NUMBER_64:
+		compare_numbers_return(orig->number_64, copy->number_64);
+
+	case TRACEEVAL_TYPE_NUMBER_32:
+		compare_numbers_return(orig->number_32, copy->number_32);
+
+	case TRACEEVAL_TYPE_NUMBER_16:
+		compare_numbers_return(orig->number_16, copy->number_16);
+
+	case TRACEEVAL_TYPE_NUMBER_8:
+		compare_numbers_return(orig->number_8, copy->number_8);
+
+	case TRACEEVAL_TYPE_DYNAMIC:
+		if (type->dyn_cmp)
+			return type->dyn_cmp(orig->dyn_data, copy->dyn_data, type);
+		return 0;
+
+	default:
+		print_err("%d is an invalid enum traceeval_data_type member",
+				type->type);
+		return -2;
+	}
+}
+
+/*
+ * Compare arrays of union traceeval_data's with respect to @def.
+ *
+ * Return 0 if @orig and @copy are the same, 1 if not, and -1 on error.
+ */
+static int compare_traceeval_data_set(union traceeval_data *orig,
+		const union traceeval_data *copy, struct traceeval_type *defs,
+		size_t size)
+{
+	int check;
+	size_t i;
+
+	/* compare data arrays */
+	for (i = 0; i < size; i++) {
+		if ((check = compare_traceeval_data(&orig[i], &copy[i], &defs[i])))
+			goto fail_c_set;
+	}
+
+	return 0;
+
+fail_c_set:
+	return check == -2 ? -1 : 1;
+}
+
+/*
+ * Return 0 if @orig and @copy are the same, 1 if not, and -1 on error.
+ */
+static int compare_entries(struct entry *orig, struct entry *copy,
+		struct traceeval *teval)
+{
+	int check;
+
+	/* compare keys */
+	check = compare_traceeval_data_set(orig->keys, copy->keys,
+			teval->key_types, teval->nr_key_types);
+	if (check)
+		return check;
+
+	/* compare values */
+	check = compare_traceeval_data_set(orig->vals, copy->vals,
+			teval->val_types, teval->nr_val_types);
+	return check;
+}
+
+/*
+ * Compares the hist fields of @orig and @copy for equality.
+ *
+ * Assumes all other aspects of @orig and @copy are the same.
+ *
+ * Return 0 if struct hist_table of @orig and @copy are the same, 1 if not,
+ * and -1 on error.
+ */
+static int compare_hist(struct traceeval *orig, struct traceeval *copy)
+{
+	struct hist_table *o_hist;
+	struct hist_table *c_hist;
+	int c;
+
+	o_hist = orig->hist;
+	c_hist = copy->hist;
+
+	if (o_hist->nr_entries != c_hist->nr_entries)
+		return 1;
+
+	for (size_t i = 0; i < o_hist->nr_entries; i++) {
+		if ((c = compare_entries(&o_hist->map[i], &c_hist->map[i], orig)))
+			return c;
+	}
+
+	return 0;
+}
+
+/*
+ * traceeval_compare - Check equality between two traceeval instances
+ * @orig: The first traceeval instance
+ * @copy: The second traceeval instance
+ *
+ * This compares the values of the key definitions, value definitions, and
+ * inserted data between @orig and @copy in order. It does not compare
+ * by memory address, except for struct traceeval_type's dyn_release() and
+ * dyn_cmp() fields.
+ *
+ * Returns 0 if @orig and @copy are the same, 1 if not, and -1 on error.
+ */
+ int traceeval_compare(struct traceeval *orig, struct traceeval *copy)
+{
+	int keys;
+	int vals;
+	int hists;
+
+	if (!orig || !copy)
+		return -1;
+
+	keys = compare_traceeval_type(orig->key_types, copy->key_types,
+			orig->nr_key_types, copy->nr_key_types);
+	vals = compare_traceeval_type(orig->val_types, copy->val_types,
+			orig->nr_val_types, copy->nr_val_types);
+	hists = compare_hist(orig, copy);
+
+	if (hists == -1)
+		return -1;
+
+	return keys || vals || hists;
+}
#14 lore raw

Re: [PATCH v2 4/5] histograms: Add traceeval compare

In reply to Stevie Alvarez

include/traceeval-test.h

Quoted 17 lines · Stevie Alvarez

From: "Stevie Alvarez (Google)" <[email protected]>

traceeval_compare() compares two struct traceeval instances for equality. This suite of comparitors was made for testing purposes.

---
 include/traceeval-test.h |  16 +++
 src/histograms.c         | 221 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 234 insertions(+), 3 deletions(-)
 create mode 100644 include/traceeval-test.h
include/traceeval-test.h new file +0 -0
new file mode 100644
index 0000000..bb8092a

<>

Quoted 4 lines
include/traceeval-test.h new file +0 -0
@@ -158,12 +172,13 @@ fail_type_name:
  * The @keys and @vals passed in are copied for internal use.
  *
  * For any member of @keys or @vals that isn't of type TRACEEVAL_TYPE_NONE,

include/traceeval-test.h fail_type_name: · line 181

Quoted 10 lines
include/traceeval-test.h new file +4 -3
- * the name field must be a null-terminated string. For members of type
- * TRACEEVAL_TYPE_NONE, the name is ignored.
+ * the name field must be a null-terminated string. Members of type
+ * TRACEEVAL_TYPE_NONE are used to terminate the array, therefore their other
+ * fields are ignored.
  *
  * @vals can be NULL or start with its type field as TRACEEVAL_TYPE_NONE to
  * define the values of the histogram to be empty.
- * @keys must be populated with at least one element that is not
+ * @keys must be populated with at least one element that is not of type

These two comment updates seem accidental, and can be just folded into the initial patch that creates them.

Quoted 48 lines
include/traceeval-test.h new file +41 -0
  * TRACEEVAL_TYPE_NONE.
  *
  * Returns the descriptor on success, or NULL on error.
@@ -305,3 +320,203 @@ void traceeval_release(struct traceeval *teval)
 	teval->val_types = NULL;
 	free(teval);
 }
+
+/*
+ * Compare traceeval_type instances for equality.
+ *
+ * Return 0 if @orig and @copy are the same, 1 otherwise.
+ */
+static int compare_traceeval_type(struct traceeval_type *orig,
+		struct traceeval_type *copy, size_t orig_size, size_t copy_size)
+{
+	size_t i;
+
+	/* same memory/NULL */
+	if (orig == copy)
+		return 0;
+	if (!!orig != !!copy)
+		return 1;
+
+	if (orig_size != copy_size)
+		return 1;
+
+	for (i = 0; i < orig_size; i++) {
+		if (orig[i].type != copy[i].type)
+			return 1;
+		if (orig[i].flags != copy[i].flags)
+			return 1;
+		if (orig[i].id != copy[i].id)
+			return 1;
+		if (orig[i].dyn_release != copy[i].dyn_release)
+			return 1;
+		if (orig[i].dyn_cmp != copy[i].dyn_cmp)
+			return 1;
+
+		// make sure both names are same type
+		if (!!orig[i].name != !!copy[i].name)
+			return 1;
+		if (!orig[i].name)
+			continue;
+		if (strcmp(orig[i].name, copy[i].name) != 0)
+			return 1;
+	}
+

include/traceeval-test.h void traceeval_release(struct traceeval *teval) · line 373

Quoted 10 lines
include/traceeval-test.h new file +10 -0
+	return 0;
+}
+
+/*
+ * Compare traceeval_data instances.
+ *
+ * Return 0 if @orig and @copy are the same, 1 if @orig is greater than @copy,
+ * -1 for the other way around, and -2 on error.
+ */
+static int compare_traceeval_data(union traceeval_data *orig,

For both this and compare_traceeval_data_set(), both 'orig' and 'copy' can be
const. Same with 'type', since compare functions shouldn't be modifying any
data. This applies to most or all of the pointer params to your compare functions.

Quoted 53 lines
include/traceeval-test.h new file +53 -0
+		const union traceeval_data *copy, struct traceeval_type *type)
+{
+	int i;
+
+	if (orig == copy)
+		return 0;
+
+	if (!orig)
+		return -1;
+
+	if (!copy)
+		return 1;
+
+	switch (type->type) {
+	case TRACEEVAL_TYPE_STRING:
+		i = strcmp(orig->string, copy->string);
+		compare_numbers_return(i, 0);
+
+	case TRACEEVAL_TYPE_NUMBER:
+		compare_numbers_return(orig->number, copy->number);
+
+	case TRACEEVAL_TYPE_NUMBER_64:
+		compare_numbers_return(orig->number_64, copy->number_64);
+
+	case TRACEEVAL_TYPE_NUMBER_32:
+		compare_numbers_return(orig->number_32, copy->number_32);
+
+	case TRACEEVAL_TYPE_NUMBER_16:
+		compare_numbers_return(orig->number_16, copy->number_16);
+
+	case TRACEEVAL_TYPE_NUMBER_8:
+		compare_numbers_return(orig->number_8, copy->number_8);
+
+	case TRACEEVAL_TYPE_DYNAMIC:
+		if (type->dyn_cmp)
+			return type->dyn_cmp(orig->dyn_data, copy->dyn_data, type);
+		return 0;
+
+	default:
+		print_err("%d is an invalid enum traceeval_data_type member",
+				type->type);
+		return -2;
+	}
+}
+
+/*
+ * Compare arrays of union traceeval_data's with respect to @def.
+ *
+ * Return 0 if @orig and @copy are the same, 1 if not, and -1 on error.
+ */
+static int compare_traceeval_data_set(union traceeval_data *orig,
+		const union traceeval_data *copy, struct traceeval_type *defs,
+		size_t size)
#15 lore raw

PATCH v2 5/5 histograms: Initial unit tests

In reply to Stevie Alvarez

From: "Stevie Alvarez (Google)" <[email protected]>

Add unit tests for traceeval_init(), traceeval_release(), and traceeval_compare() to check edge cases and ensure the interface is functional.

---
 include/traceeval-test.h |   1 -
 utest/.gitignore         |   1 +
 utest/Makefile           |  35 +++++++
 utest/eval-test.h        |  13 +++
 utest/eval-utest.c       |  28 +++++
 utest/traceeval-utest.c  | 217 +++++++++++++++++++++++++++++++++++++++
 6 files changed, 294 insertions(+), 1 deletion(-)
 create mode 100644 utest/.gitignore
 create mode 100644 utest/Makefile
 create mode 100644 utest/eval-test.h
 create mode 100644 utest/eval-utest.c
 create mode 100644 utest/traceeval-utest.c
include/traceeval-test.h +0 -1
index bb8092a..45cf13b 100644
@@ -2,7 +2,6 @@
 /*
  * libtraceeval interface for unit testing.
  *
- * Copyright (C) 2023 Google Inc, Steven Rostedt <[email protected]>
  * Copyright (C) 2023 Google Inc, Stevie Alvarez <[email protected]>
  */
 
utest/.gitignore new file +1 -0
new file mode 100644
index 0000000..ca0ac10
@@ -0,0 +1 @@
+eval-utest
utest/Makefile new file +35 -0
new file mode 100644
index 0000000..955f91e
@@ -0,0 +1,35 @@
+# SPDX-License-Identifier: MIT
+
+include $(src)/scripts/utils.mk
+
+bdir := $(obj)/utest
+eval_lib := $(obj)/lib/libtraceeval.a
+
+TARGETS = $(bdir)/eval-utest
+
+OBJS := eval-utest.o
+OBJS += traceeval-utest.o
+
+LIBS += -lcunit				\
+	-ldl				\
+	$(eval_lib)
+
+OBJS := $(OBJS:%.o=$(bdir)/%.o)
+
+$(bdir):
+	@mkdir -p $(bdir)
+
+$(OBJS): | $(bdir)
+
+$(bdir)/eval-utest: $(OBJS) $(eval_lib)
+	$(Q)$(do_app_build)
+
+$(bdir)/%.o: %.c
+	$(Q)$(call do_fpic_compile)
+
+-include .*.d
+
+test: $(TARGETS)
+
+clean:
+	$(Q)$(call do_clean,$(TARGETS) $(bdir)/*.o $(bdir)/.*.d)
utest/eval-test.h new file +13 -0
new file mode 100644
index 0000000..f6e594e
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * libtraceeval cunit interface.
+ *
+ * Copyright (C) 2023 Google Inc, Stevie Alvarez <[email protected]>
+ */
+
+#ifndef _EVAL_UTEST_H_
+#define _EVAL_UTEST_H_
+
+void test_traceeval_lib(void);
+
+#endif /* _EVAL_UTEST_H_ */
utest/eval-utest.c new file +28 -0
new file mode 100644
index 0000000..94d04c6
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * libtraceeval cunit handler.
+ *
+ * Copyright (C) 2023 Google Inc, Stevie Alvarez <[email protected]>
+ */
+#include <stdio.h>
+#include <stdlib.h>
+
+#include <CUnit/CUnit.h>
+#include <CUnit/Basic.h>
+
+#include "eval-test.h"
+
+int main(int argc, char **argv)
+{
+	if (CU_initialize_registry() != CUE_SUCCESS) {
+		printf("Test registry cannot be initialized\n");
+		return EXIT_FAILURE;
+	}
+
+	test_traceeval_lib();
+
+	CU_basic_set_mode(CU_BRM_VERBOSE);
+	CU_basic_run_tests();
+	CU_cleanup_registry();
+	return EXIT_SUCCESS;
+}
utest/traceeval-utest.c new file +217 -0

217 lines not on this page · raw message

Signature · 1 line
2.41.0
#16 lore raw
In reply to Stevie Alvarez
Quoted 21 lines · On Thu, 3 Aug 2023 18:53:58 -0400 Stevie Alvarez <[email protected]>…

From: "Stevie Alvarez (Google)" <[email protected]>

Changes in v2:
* Renamed interface from histograms.h to traceeval-hist.h.
* Internal traceeval_type arrays terminated by size rather than type.
* type_alloc() guarantees traceeval_type name field is a valid pointer.
* traceeval_release() sets released pointers to NULL.
* Conditional simplifications and fixes across the board.
* Removed function stubs and their interface definitions.
* Added a timestamp flag.
* Added typedefs for traceeval_dynamic helper function signatures.
* Documentation follows established conventions (kernel docs, line comments).
* Documentation moved from interface to source code.
* Internal testing interface.
* General documentation spell checking.

Since the addition of the traceeval_dynamic function signature typedefs, warnings pertaining to these function signatures and traceeval_type have popped up. Steven, Ross, and anyone else, any ideas on why these warnings are showing up?

Quoted 5 lines

Warning from function signatures in include/traceeval-hist.h, line 65 and 69:
warning: ‘struct traceeval_type’ declared inside parameter list will not be visible outside of this definition or declaration

struct traceeval_type *);

This is because the typedef is before the declaration. Just add:

struct traceeval_type;

above it.

Quoted 7 lines

Warning from dyn_release call in src/histograms.c line 258:

	warning: passing argument 2 of
	‘(defs + (sizetype)(i * 48))->dyn_release’ from incompatible
	pointer type

		defs[i].dyn_release(data[i].dyn_data, &defs[i]);

Try passing in: defs + i

I'll review these later today.

-- Steve

Quoted 43 lines

note: expected ‘struct traceeval_type *’ but argument is of type ‘struct traceeval_type *’

Warning from dyn_cmp call in src/histograms.c line 409:

	warning: passing argument 3 of ‘type->dyn_cmp’ from incompatible
	pointer type

		return type->dyn_cmp(orig->dyn_data, copy->dyn_data, type);

	note: expected ‘struct traceeval_type *’ but argument is of type
	‘struct traceeval_type *’

---
v1 discussion: https://lore.kernel.org/linux-trace-devel/[email protected]/T/

Stevie Alvarez (Google) (5):
histograms: Initial histograms interface
histograms: Add traceeval initialize
histograms: Add traceeval release
histograms: Add traceeval compare
histograms: Initial unit tests

 Makefile                 |   2 +-
 include/traceeval-hist.h | 135 ++++++++++
 include/traceeval-test.h |  15 ++
 src/Makefile             |   1 +
 src/histograms.c         | 522 +++++++++++++++++++++++++++++++++++++++
 utest/.gitignore         |   1 +
 utest/Makefile           |  35 +++
 utest/eval-test.h        |  13 +
 utest/eval-utest.c       |  28 +++
 utest/traceeval-utest.c  | 217 ++++++++++++++++
 10 files changed, 968 insertions(+), 1 deletion(-)
 create mode 100644 include/traceeval-hist.h
 create mode 100644 include/traceeval-test.h
 create mode 100644 src/histograms.c
 create mode 100644 utest/.gitignore
 create mode 100644 utest/Makefile
 create mode 100644 utest/eval-test.h
 create mode 100644 utest/eval-utest.c
 create mode 100644 utest/traceeval-utest.c