Skip to contentSkip to conversation

tracing: Have ring_buffer_event_time_stamp() work for all events

PATCH Superseded Steven Rostedt 2 participants 10 messages Mar 12, 2021 – Mar 13, 2021

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Steven Rostedt

author maintainer
#0 lore raw

cover tracing: Have ring_buffer_event_time_stamp() work for all events

Yordan has been working on a patch set that will not repeat functions in the function tracer but instead show a single event that states that the previous function repeated X amount of times. I recommended that this should also show the time of the last function that repeated. To save on space, Yordan suggested to use a 32 bit delta instead of a full 64 bit timestamp of the last event. But to create this delta, we need a way to extract the time of the repeat event and use that to calculate the delta of the last function event.

But currently the ring_buffer_event_time_stamp() only works if all events has an absolute time stamp attached to it. That would be counter productive to make all functions record the full time stamp because we wanted to use a 32 bit counter instead of a 64 bit counter for the rare repeat event!

This patch series makes ring_buffer_event_time_stamp() work for all non committed events (after reserve and before it is committed). The trick is that the per CPU descriptor of the buffer holds the time stamp used for the event being processed. It has a nesting of 5 levels to deal with interrupts that add events. Extracting the time for the event can only be done for the current event being processed and not yet committed.

To achieve this, the buffer that the event is on needs to be passed to the ring_buffer_event_time_stamp(). That caused a bit of churn for the histogram code, because that buffer wasn't available at the time the ring_buffer_event_time_stamp() was called. Although the hist_field contained a pointer to the trace_array that holds the ring buffer, if tracing was disabled, then a temporary buffer was used instead, and that temporary buffer would need to be passed to the ring_buffer_event_time_stamp(). That buffer is exposed to the trace event logic, but was not passed down into the trigger code. That had to be done to take advantage of the new code.

The second to last patch adds verifier logic (although commented out) that was used to debug the issue when a temporary buffer was in use. It proved to be very valuable and is kept in, in case this needs to be debugged again.

Finally the last patch is for use with Yordan's work, and is needed because when filtering is enabled, the event being processed may not even be on the buffer, and the tracing_event_time_stamp() checks for that and will just use the current time stamp if the event is not reserved yet. This could be used also for the histogram code, but wanted to hold off on that.

Steven Rostedt (VMware) (7):
ring-buffer: Separate out internal use of ring_buffer_event_time_stamp()
ring-buffer: Add a event_stamp to cpu_buffer for each level of nesting
tracing: Pass buffer of event to trigger operations
ring-buffer: Allow ring_buffer_event_time_stamp() to return time stamp of all events
tracing: Use a no_filter_buffering_ref to stop using the filter buffer
ring-buffer: Add verifier for using ring_buffer_event_time_stamp()
tracing: Add tracing_event_time_stamp() API

----

 include/linux/ring_buffer.h         |   3 +-
 include/linux/trace_events.h        |   5 +-
 kernel/trace/ring_buffer.c          | 138 ++++++++++++++++++++++++++++++------
 kernel/trace/trace.c                |  40 ++++++-----
 kernel/trace/trace.h                |   9 +--
 kernel/trace/trace_events_hist.c    |  97 ++++++++++++++++---------
 kernel/trace/trace_events_trigger.c |  45 +++++++-----
 7 files changed, 239 insertions(+), 98 deletions(-)
author maintainer
#1 lore raw

PATCH 1/7 ring-buffer: Separate out internal use of ring_buffer_event_time_stamp()

From: "Steven Rostedt (VMware)" <[email protected]>

The exported use of ring_buffer_event_time_stamp() is going to become different than how it is used internally. Move the internal logic out into a static function called rb_event_time_stamp(), and have the internal callers call that instead.

---
 kernel/trace/ring_buffer.c | 41 +++++++++++++++++++++-----------------
 1 file changed, 23 insertions(+), 18 deletions(-)
kernel/trace/ring_buffer.c +23 -18
index 68744c51517e..941ac2021b97 100644
@@ -287,6 +287,17 @@ EXPORT_SYMBOL_GPL(ring_buffer_event_data);
 #define TS_MASK		((1ULL << TS_SHIFT) - 1)
 #define TS_DELTA_TEST	(~TS_MASK)
 
+static u64 rb_event_time_stamp(struct ring_buffer_event *event)
+{
+	u64 ts;
+
+	ts = event->array[0];
+	ts <<= TS_SHIFT;
+	ts += event->time_delta;
+
+	return ts;
+}
+
 /**
  * ring_buffer_event_time_stamp - return the event's extended timestamp
  * @event: the event to get the timestamp of
@@ -299,13 +310,7 @@ EXPORT_SYMBOL_GPL(ring_buffer_event_data);
  */
 u64 ring_buffer_event_time_stamp(struct ring_buffer_event *event)
 {
-	u64 ts;
-
-	ts = event->array[0];
-	ts <<= TS_SHIFT;
-	ts += event->time_delta;
-
-	return ts;
+	return rb_event_time_stamp(event);
 }
 
 /* Flag when events were overwritten */
@@ -2766,7 +2771,7 @@ static u64 rb_time_delta(struct ring_buffer_event *event)
 		return 0;
 
 	case RINGBUF_TYPE_TIME_EXTEND:
-		return ring_buffer_event_time_stamp(event);
+		return rb_event_time_stamp(event);
 
 	case RINGBUF_TYPE_TIME_STAMP:
 		return 0;
@@ -3212,13 +3217,13 @@ static void dump_buffer_page(struct buffer_data_page *bpage,
 		switch (event->type_len) {
 
 		case RINGBUF_TYPE_TIME_EXTEND:
-			delta = ring_buffer_event_time_stamp(event);
+			delta = rb_event_time_stamp(event);
 			ts += delta;
 			pr_warn("  [%lld] delta:%lld TIME EXTEND\n", ts, delta);
 			break;
 
 		case RINGBUF_TYPE_TIME_STAMP:
-			delta = ring_buffer_event_time_stamp(event);
+			delta = rb_event_time_stamp(event);
 			ts = delta;
 			pr_warn("  [%lld] absolute:%lld TIME STAMP\n", ts, delta);
 			break;
@@ -3289,12 +3294,12 @@ static void check_buffer(struct ring_buffer_per_cpu *cpu_buffer,
 		switch (event->type_len) {
 
 		case RINGBUF_TYPE_TIME_EXTEND:
-			delta = ring_buffer_event_time_stamp(event);
+			delta = rb_event_time_stamp(event);
 			ts += delta;
 			break;
 
 		case RINGBUF_TYPE_TIME_STAMP:
-			delta = ring_buffer_event_time_stamp(event);
+			delta = rb_event_time_stamp(event);
 			ts = delta;
 			break;
 
@@ -4256,12 +4261,12 @@ rb_update_read_stamp(struct ring_buffer_per_cpu *cpu_buffer,
 		return;
 
 	case RINGBUF_TYPE_TIME_EXTEND:
-		delta = ring_buffer_event_time_stamp(event);
+		delta = rb_event_time_stamp(event);
 		cpu_buffer->read_stamp += delta;
 		return;
 
 	case RINGBUF_TYPE_TIME_STAMP:
-		delta = ring_buffer_event_time_stamp(event);
+		delta = rb_event_time_stamp(event);
 		cpu_buffer->read_stamp = delta;
 		return;
 
@@ -4286,12 +4291,12 @@ rb_update_iter_read_stamp(struct ring_buffer_iter *iter,
 		return;
 
 	case RINGBUF_TYPE_TIME_EXTEND:
-		delta = ring_buffer_event_time_stamp(event);
+		delta = rb_event_time_stamp(event);
 		iter->read_stamp += delta;
 		return;
 
 	case RINGBUF_TYPE_TIME_STAMP:
-		delta = ring_buffer_event_time_stamp(event);
+		delta = rb_event_time_stamp(event);
 		iter->read_stamp = delta;
 		return;
 
@@ -4544,7 +4549,7 @@ rb_buffer_peek(struct ring_buffer_per_cpu *cpu_buffer, u64 *ts,
 
 	case RINGBUF_TYPE_TIME_STAMP:
 		if (ts) {
-			*ts = ring_buffer_event_time_stamp(event);
+			*ts = rb_event_time_stamp(event);
 			ring_buffer_normalize_time_stamp(cpu_buffer->buffer,
 							 cpu_buffer->cpu, ts);
 		}
@@ -4635,7 +4640,7 @@ rb_iter_peek(struct ring_buffer_iter *iter, u64 *ts)
 
 	case RINGBUF_TYPE_TIME_STAMP:
 		if (ts) {
-			*ts = ring_buffer_event_time_stamp(event);
+			*ts = rb_event_time_stamp(event);
 			ring_buffer_normalize_time_stamp(cpu_buffer->buffer,
 							 cpu_buffer->cpu, ts);
 		}
Signature · 1 line
2.30.1
author maintainer
#2 lore raw

PATCH 2/7 ring-buffer: Add a event_stamp to cpu_buffer for each level of nesting

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

Add a place to save the current event time stamp for each level of nesting. This will be used to retrieve the time stamp of the current event before it is committed.

---
 kernel/trace/ring_buffer.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
kernel/trace/ring_buffer.c +10 -1
index 941ac2021b97..470d97169081 100644
@@ -492,6 +492,8 @@ struct rb_time_struct {
 #endif
 typedef struct rb_time_struct rb_time_t;
 
+#define MAX_NEST	5
+
 /*
  * head_page == tail_page && head == tail then buffer is empty.
  */
@@ -529,6 +531,7 @@ struct ring_buffer_per_cpu {
 	unsigned long			read_bytes;
 	rb_time_t			write_stamp;
 	rb_time_t			before_stamp;
+	u64				event_stamp[MAX_NEST];
 	u64				read_stamp;
 	/* ring buffer pages to update, > 0 to add, < 0 to remove */
 	long				nr_pages_to_update;
@@ -2715,6 +2718,10 @@ rb_update_event(struct ring_buffer_per_cpu *cpu_buffer,
 {
 	unsigned length = info->length;
 	u64 delta = info->delta;
+	unsigned int nest = local_read(&cpu_buffer->committing) - 1;
+
+	if (nest < MAX_NEST)
+		cpu_buffer->event_stamp[nest] = info->ts;
 
 	/*
 	 * If we need to add a timestamp, then we
@@ -3456,7 +3463,6 @@ __rb_reserve_next(struct ring_buffer_per_cpu *cpu_buffer,
 				    info->after, ts)) {
 			/* Nothing came after this event between C and E */
 			info->delta = ts - info->after;
-			info->ts = ts;
 		} else {
 			/*
 			 * Interrupted between C and E:
@@ -3468,6 +3474,7 @@ __rb_reserve_next(struct ring_buffer_per_cpu *cpu_buffer,
 			 */
 			info->delta = 0;
 		}
+		info->ts = ts;
 		info->add_timestamp &= ~RB_ADD_STAMP_FORCE;
 	}
 
@@ -5026,6 +5033,8 @@ rb_reset_cpu(struct ring_buffer_per_cpu *cpu_buffer)
 	rb_time_set(&cpu_buffer->write_stamp, 0);
 	rb_time_set(&cpu_buffer->before_stamp, 0);
 
+	memset(cpu_buffer->event_stamp, 0, sizeof(cpu_buffer->event_stamp));
+
 	cpu_buffer->lost_events = 0;
 	cpu_buffer->last_overrun = 0;
 
Signature · 1 line
2.30.1
author maintainer
#3 lore raw

PATCH 3/7 tracing: Pass buffer of event to trigger operations

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

The ring_buffer_event_time_stamp() is going to be updated to extract the time stamp for the event without needing it to be set to have absolute values for all events. But to do so, it needs the buffer that the event is on as the buffer saves information for the event before it is committed to the buffer.

If the trace buffer is disabled, a temporary buffer is used, and there's no access to this buffer from the current histogram triggers, even though it is passed to the trace event code.

Pass the buffer that the event is on all the way down to the histogram triggers.

---
 include/linux/trace_events.h        |  5 +-
 kernel/trace/trace.c                |  2 +-
 kernel/trace/trace.h                |  4 +-
 kernel/trace/trace_events_hist.c    | 89 +++++++++++++++++++----------
 kernel/trace/trace_events_trigger.c | 45 +++++++++------
 5 files changed, 93 insertions(+), 52 deletions(-)
include/linux/trace_events.h +3 -2
index 28e7af1406f2..8cba64ce23a4 100644
@@ -640,7 +640,8 @@ enum event_trigger_type {
 extern int filter_match_preds(struct event_filter *filter, void *rec);
 
 extern enum event_trigger_type
-event_triggers_call(struct trace_event_file *file, void *rec,
+event_triggers_call(struct trace_event_file *file,
+		    struct trace_buffer *buffer, void *rec,
 		    struct ring_buffer_event *event);
 extern void
 event_triggers_post_call(struct trace_event_file *file,
@@ -664,7 +665,7 @@ trace_trigger_soft_disabled(struct trace_event_file *file)
 
 	if (!(eflags & EVENT_FILE_FL_TRIGGER_COND)) {
 		if (eflags & EVENT_FILE_FL_TRIGGER_MODE)
-			event_triggers_call(file, NULL, NULL);
+			event_triggers_call(file, NULL, NULL, NULL);
 		if (eflags & EVENT_FILE_FL_SOFT_DISABLED)
 			return true;
 		if (eflags & EVENT_FILE_FL_PID_FILTER)
kernel/trace/trace.c +1 -1
index 8a79bd416da5..c138891eab86 100644
@@ -6911,7 +6911,7 @@ tracing_mark_write(struct file *filp, const char __user *ubuf,
 	if (tr->trace_marker_file && !list_empty(&tr->trace_marker_file->triggers)) {
 		/* do not add \n before testing triggers, but add \0 */
 		entry->buf[cnt] = '\0';
-		tt = event_triggers_call(tr->trace_marker_file, entry, event);
+		tt = event_triggers_call(tr->trace_marker_file, buffer, entry, event);
 	}
 
 	if (entry->buf[cnt - 1] != '\n') {
kernel/trace/trace.h +2 -2
index 9240c3b5bf09..16bedaaface4 100644
@@ -1331,7 +1331,7 @@ __event_trigger_test_discard(struct trace_event_file *file,
 	unsigned long eflags = file->flags;
 
 	if (eflags & EVENT_FILE_FL_TRIGGER_COND)
-		*tt = event_triggers_call(file, entry, event);
+		*tt = event_triggers_call(file, buffer, entry, event);
 
 	if (test_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &file->flags) ||
 	    (unlikely(file->flags & EVENT_FILE_FL_FILTERED) &&
@@ -1628,7 +1628,7 @@ extern int register_trigger_hist_enable_disable_cmds(void);
  */
 struct event_trigger_ops {
 	void			(*func)(struct event_trigger_data *data,
-					void *rec,
+					struct trace_buffer *buffer, void *rec,
 					struct ring_buffer_event *rbe);
 	int			(*init)(struct event_trigger_ops *ops,
 					struct event_trigger_data *data);
kernel/trace/trace_events_hist.c +59 -30

333 lines not on this page · raw message

kernel/trace/trace_events_trigger.c +28 -17

147 lines not on this page · raw message

Signature · 1 line
2.30.1
#4 lore raw

Re: [PATCH 3/7] tracing: Pass buffer of event to trigger operations

Thank you for the patch! Yet something to improve:

Hi Steven,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linux/master]
[also build test ERROR on linus/master v5.12-rc2 next-20210312] [cannot apply to tip/perf/core hnaz-linux-mm/master]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Steven-Rostedt/tracing-Have-ring_buffer_event_time_stamp-work-for-all-events/20210313-051807
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git a74e6a014c9d4d4161061f770c9b4f98372ac778
config: x86_64-randconfig-a004-20210312 (attached as .config)
compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project dfd27ebbd0eb137c9a439b7c537bb87ba903efd3)

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/e0953acfef6e9234feed2877ed1b0a796755ceb6
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Steven-Rostedt/tracing-Have-ring_buffer_event_time_stamp-work-for-all-events/20210313-051807 git checkout e0953acfef6e9234feed2877ed1b0a796755ceb6
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

>> kernel/trace/trace_events_hist.c:3191:30: error: incompatible function pointer types assigning to 'action_fn_t' (aka 'void (*)(struct hist_trigger_data *, struct tracing_map_elt *, struct trace_buffer *, void *, struct ring_buffer_event *, void *, struct action_data *, unsigned long long *)') from 'void (struct hist_trigger_data *, struct tracing_map_elt *, void *, struct ring_buffer_event *, void *, struct action_data *, u64 *)' (aka 'void (struct hist_trigger_data *, struct tracing_map_elt *, void *, struct ring_buffer_event *, void *, struct action_data *, unsigned long long *)') [-Werror,-Wincompatible-function-pointer-types]
                   data->track_data.save_data = save_track_data_snapshot;
                                              ^ ~~~~~~~~~~~~~~~~~~~~~~~~

1 error generated.

vim +3191 kernel/trace/trace_events_hist.c

50450603ec9cb8 Tom Zanussi             2018-01-15  3125  
d0cd871ba0d613 Steven Rostedt (VMware  2019-04-01  3126) static int action_parse(struct trace_array *tr, char *str, struct action_data *data,
7d18a10c316783 Tom Zanussi             2019-02-13  3127  			enum handler_id handler)
7d18a10c316783 Tom Zanussi             2019-02-13  3128  {
7d18a10c316783 Tom Zanussi             2019-02-13  3129  	char *action_name;
7d18a10c316783 Tom Zanussi             2019-02-13  3130  	int ret = 0;
7d18a10c316783 Tom Zanussi             2019-02-13  3131  
7d18a10c316783 Tom Zanussi             2019-02-13  3132  	strsep(&str, ".");
7d18a10c316783 Tom Zanussi             2019-02-13  3133  	if (!str) {
d0cd871ba0d613 Steven Rostedt (VMware  2019-04-01  3134) 		hist_err(tr, HIST_ERR_ACTION_NOT_FOUND, 0);
7d18a10c316783 Tom Zanussi             2019-02-13  3135  		ret = -EINVAL;
7d18a10c316783 Tom Zanussi             2019-02-13  3136  		goto out;
7d18a10c316783 Tom Zanussi             2019-02-13  3137  	}
7d18a10c316783 Tom Zanussi             2019-02-13  3138  
7d18a10c316783 Tom Zanussi             2019-02-13  3139  	action_name = strsep(&str, "(");
7d18a10c316783 Tom Zanussi             2019-02-13  3140  	if (!action_name || !str) {
d0cd871ba0d613 Steven Rostedt (VMware  2019-04-01  3141) 		hist_err(tr, HIST_ERR_ACTION_NOT_FOUND, 0);
7d18a10c316783 Tom Zanussi             2019-02-13  3142  		ret = -EINVAL;
7d18a10c316783 Tom Zanussi             2019-02-13  3143  		goto out;
7d18a10c316783 Tom Zanussi             2019-02-13  3144  	}
7d18a10c316783 Tom Zanussi             2019-02-13  3145  
7d18a10c316783 Tom Zanussi             2019-02-13  3146  	if (str_has_prefix(action_name, "save")) {
7d18a10c316783 Tom Zanussi             2019-02-13  3147  		char *params = strsep(&str, ")");
7d18a10c316783 Tom Zanussi             2019-02-13  3148  
7d18a10c316783 Tom Zanussi             2019-02-13  3149  		if (!params) {
d0cd871ba0d613 Steven Rostedt (VMware  2019-04-01  3150) 			hist_err(tr, HIST_ERR_NO_SAVE_PARAMS, 0);
7d18a10c316783 Tom Zanussi             2019-02-13  3151  			ret = -EINVAL;
7d18a10c316783 Tom Zanussi             2019-02-13  3152  			goto out;
7d18a10c316783 Tom Zanussi             2019-02-13  3153  		}
7d18a10c316783 Tom Zanussi             2019-02-13  3154  
d0cd871ba0d613 Steven Rostedt (VMware  2019-04-01  3155) 		ret = parse_action_params(tr, params, data);
7d18a10c316783 Tom Zanussi             2019-02-13  3156  		if (ret)
7d18a10c316783 Tom Zanussi             2019-02-13  3157  			goto out;
7d18a10c316783 Tom Zanussi             2019-02-13  3158  
7d18a10c316783 Tom Zanussi             2019-02-13  3159  		if (handler == HANDLER_ONMAX)
466f4528fbc692 Tom Zanussi             2019-02-13  3160  			data->track_data.check_val = check_track_val_max;
dff81f559285b5 Tom Zanussi             2019-02-13  3161  		else if (handler == HANDLER_ONCHANGE)
dff81f559285b5 Tom Zanussi             2019-02-13  3162  			data->track_data.check_val = check_track_val_changed;
466f4528fbc692 Tom Zanussi             2019-02-13  3163  		else {
d0cd871ba0d613 Steven Rostedt (VMware  2019-04-01  3164) 			hist_err(tr, HIST_ERR_ACTION_MISMATCH, errpos(action_name));
466f4528fbc692 Tom Zanussi             2019-02-13  3165  			ret = -EINVAL;
466f4528fbc692 Tom Zanussi             2019-02-13  3166  			goto out;
466f4528fbc692 Tom Zanussi             2019-02-13  3167  		}
7d18a10c316783 Tom Zanussi             2019-02-13  3168  
466f4528fbc692 Tom Zanussi             2019-02-13  3169  		data->track_data.save_data = save_track_data_vars;
466f4528fbc692 Tom Zanussi             2019-02-13  3170  		data->fn = ontrack_action;
7d18a10c316783 Tom Zanussi             2019-02-13  3171  		data->action = ACTION_SAVE;
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3172  	} else if (str_has_prefix(action_name, "snapshot")) {
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3173  		char *params = strsep(&str, ")");
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3174  
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3175  		if (!str) {
d0cd871ba0d613 Steven Rostedt (VMware  2019-04-01  3176) 			hist_err(tr, HIST_ERR_NO_CLOSING_PAREN, errpos(params));
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3177  			ret = -EINVAL;
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3178  			goto out;
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3179  		}
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3180  
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3181  		if (handler == HANDLER_ONMAX)
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3182  			data->track_data.check_val = check_track_val_max;
dff81f559285b5 Tom Zanussi             2019-02-13  3183  		else if (handler == HANDLER_ONCHANGE)
dff81f559285b5 Tom Zanussi             2019-02-13  3184  			data->track_data.check_val = check_track_val_changed;
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3185  		else {
d0cd871ba0d613 Steven Rostedt (VMware  2019-04-01  3186) 			hist_err(tr, HIST_ERR_ACTION_MISMATCH, errpos(action_name));
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3187  			ret = -EINVAL;
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3188  			goto out;
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3189  		}
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3190  
a3785b7eca8fd4 Tom Zanussi             2019-02-13 @3191  		data->track_data.save_data = save_track_data_snapshot;
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3192  		data->fn = ontrack_action;
a3785b7eca8fd4 Tom Zanussi             2019-02-13  3193  		data->action = ACTION_SNAPSHOT;
7d18a10c316783 Tom Zanussi             2019-02-13  3194  	} else {
7d18a10c316783 Tom Zanussi             2019-02-13  3195  		char *params = strsep(&str, ")");
7d18a10c316783 Tom Zanussi             2019-02-13  3196  
e91eefd731d933 Tom Zanussi             2019-02-13  3197  		if (str_has_prefix(action_name, "trace"))
e91eefd731d933 Tom Zanussi             2019-02-13  3198  			data->use_trace_keyword = true;
e91eefd731d933 Tom Zanussi             2019-02-13  3199  
7d18a10c316783 Tom Zanussi             2019-02-13  3200  		if (params) {
d0cd871ba0d613 Steven Rostedt (VMware  2019-04-01  3201) 			ret = parse_action_params(tr, params, data);
7d18a10c316783 Tom Zanussi             2019-02-13  3202  			if (ret)
7d18a10c316783 Tom Zanussi             2019-02-13  3203  				goto out;
7d18a10c316783 Tom Zanussi             2019-02-13  3204  		}
7d18a10c316783 Tom Zanussi             2019-02-13  3205  
466f4528fbc692 Tom Zanussi             2019-02-13  3206  		if (handler == HANDLER_ONMAX)
466f4528fbc692 Tom Zanussi             2019-02-13  3207  			data->track_data.check_val = check_track_val_max;
dff81f559285b5 Tom Zanussi             2019-02-13  3208  		else if (handler == HANDLER_ONCHANGE)
dff81f559285b5 Tom Zanussi             2019-02-13  3209  			data->track_data.check_val = check_track_val_changed;
466f4528fbc692 Tom Zanussi             2019-02-13  3210  
466f4528fbc692 Tom Zanussi             2019-02-13  3211  		if (handler != HANDLER_ONMATCH) {
466f4528fbc692 Tom Zanussi             2019-02-13  3212  			data->track_data.save_data = action_trace;
466f4528fbc692 Tom Zanussi             2019-02-13  3213  			data->fn = ontrack_action;
466f4528fbc692 Tom Zanussi             2019-02-13  3214  		} else
7d18a10c316783 Tom Zanussi             2019-02-13  3215  			data->fn = action_trace;
466f4528fbc692 Tom Zanussi             2019-02-13  3216  
7d18a10c316783 Tom Zanussi             2019-02-13  3217  		data->action = ACTION_TRACE;
7d18a10c316783 Tom Zanussi             2019-02-13  3218  	}
7d18a10c316783 Tom Zanussi             2019-02-13  3219  
7d18a10c316783 Tom Zanussi             2019-02-13  3220  	data->action_name = kstrdup(action_name, GFP_KERNEL);
7d18a10c316783 Tom Zanussi             2019-02-13  3221  	if (!data->action_name) {
7d18a10c316783 Tom Zanussi             2019-02-13  3222  		ret = -ENOMEM;
7d18a10c316783 Tom Zanussi             2019-02-13  3223  		goto out;
7d18a10c316783 Tom Zanussi             2019-02-13  3224  	}
7d18a10c316783 Tom Zanussi             2019-02-13  3225  
7d18a10c316783 Tom Zanussi             2019-02-13  3226  	data->handler = handler;
7d18a10c316783 Tom Zanussi             2019-02-13  3227   out:
7d18a10c316783 Tom Zanussi             2019-02-13  3228  	return ret;
7d18a10c316783 Tom Zanussi             2019-02-13  3229  }
7d18a10c316783 Tom Zanussi             2019-02-13  3230  

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

author maintainer
#5 lore raw

PATCH 4/7 ring-buffer: Allow ring_buffer_event_time_stamp() to return time stamp of all events

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

Currently, ring_buffer_event_time_stamp() only returns an accurate time stamp of the event if it has an absolute extended time stamp attached to it. To make it more robust, use the event_stamp() in case the event does not have an absolute value attached to it.

This will allow ring_buffer_event_time_stamp() to be used in more cases than just histograms, and it will also allow histograms to not require including absolute values all the time.

---
 include/linux/ring_buffer.h      |  3 +-
 kernel/trace/ring_buffer.c       | 60 ++++++++++++++++++++++++--------
 kernel/trace/trace_events_hist.c |  2 +-
 3 files changed, 48 insertions(+), 17 deletions(-)
include/linux/ring_buffer.h +2 -1
index 136ea0997e6d..057b7ed4fe24 100644
@@ -61,7 +61,8 @@ enum ring_buffer_type {
 
 unsigned ring_buffer_event_length(struct ring_buffer_event *event);
 void *ring_buffer_event_data(struct ring_buffer_event *event);
-u64 ring_buffer_event_time_stamp(struct ring_buffer_event *event);
+u64 ring_buffer_event_time_stamp(struct trace_buffer *buffer,
+				 struct ring_buffer_event *event);
 
 /*
  * ring_buffer_discard_commit will remove an event that has not
kernel/trace/ring_buffer.c +45 -15

72 lines not on this page · raw message

kernel/trace/trace_events_hist.c +1 -1
index aabad2c93e3a..05ead88cfb66 100644
@@ -644,7 +644,7 @@ static u64 hist_field_timestamp(struct hist_field *hist_field,
 	struct hist_trigger_data *hist_data = hist_field->hist_data;
 	struct trace_array *tr = hist_data->event_file->tr;
 
-	u64 ts = ring_buffer_event_time_stamp(rbe);
+	u64 ts = ring_buffer_event_time_stamp(buffer, rbe);
 
 	if (hist_data->attrs->ts_in_usecs && trace_clock_in_ns(tr))
 		ts = ns2usecs(ts);
Signature · 1 line
2.30.1
author maintainer
#6 lore raw

PATCH 5/7 tracing: Use a no_filter_buffering_ref to stop using the filter buffer

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

Currently, the trace histograms relies on it using absolute time stamps to trigger the tracing to not use the temp buffer if filters are set. That's because the histograms need the full timestamp that is saved in the ring buffer. That is no longer the case, as the ring_buffer_event_time_stamp() can now return the time stamp for all events without all triggering a full absolute time stamp.

Now that the absolute time stamp is an unrelated dependency to not using the filters. There's nothing about having absolute timestamps to keep from using the filter buffer. Instead, change the interface to explicitly state to disable filter buffering that the histogram logic can use.

---
 kernel/trace/trace.c             | 28 ++++++++++++----------------
 kernel/trace/trace.h             |  4 ++--
 kernel/trace/trace_events_hist.c |  6 +++---
 3 files changed, 17 insertions(+), 21 deletions(-)
kernel/trace/trace.c +12 -16
index c138891eab86..8cf7397f3f56 100644
@@ -2737,12 +2737,13 @@ trace_event_buffer_lock_reserve(struct trace_buffer **current_rb,
 			  unsigned int trace_ctx)
 {
 	struct ring_buffer_event *entry;
+	struct trace_array *tr = trace_file->tr;
 	int val;
 
-	*current_rb = trace_file->tr->array_buffer.buffer;
+	*current_rb = tr->array_buffer.buffer;
 
-	if (!ring_buffer_time_stamp_abs(*current_rb) && (trace_file->flags &
-	     (EVENT_FILE_FL_SOFT_DISABLED | EVENT_FILE_FL_FILTERED)) &&
+	if (!tr->no_filter_buffering_ref &&
+	    (trace_file->flags & (EVENT_FILE_FL_SOFT_DISABLED | EVENT_FILE_FL_FILTERED)) &&
 	    (entry = this_cpu_read(trace_buffered_event))) {
 		/* Try to use the per cpu buffer first */
 		val = this_cpu_inc_return(trace_buffered_event_cnt);
@@ -7119,31 +7120,26 @@ static int tracing_time_stamp_mode_open(struct inode *inode, struct file *file)
 	return ret;
 }
 
-int tracing_set_time_stamp_abs(struct trace_array *tr, bool abs)
+/*
+ * Set or disable using the per CPU trace_buffer_event when possible.
+ */
+int tracing_set_filter_buffering(struct trace_array *tr, bool set)
 {
 	int ret = 0;
 
 	mutex_lock(&trace_types_lock);
 
-	if (abs && tr->time_stamp_abs_ref++)
+	if (set && tr->no_filter_buffering_ref++)
 		goto out;
 
-	if (!abs) {
-		if (WARN_ON_ONCE(!tr->time_stamp_abs_ref)) {
+	if (!set) {
+		if (WARN_ON_ONCE(!tr->no_filter_buffering_ref)) {
 			ret = -EINVAL;
 			goto out;
 		}
 
-		if (--tr->time_stamp_abs_ref)
-			goto out;
+		--tr->no_filter_buffering_ref;
 	}
-
-	ring_buffer_set_time_stamp_abs(tr->array_buffer.buffer, abs);
-
-#ifdef CONFIG_TRACER_MAX_TRACE
-	if (tr->max_buffer.buffer)
-		ring_buffer_set_time_stamp_abs(tr->max_buffer.buffer, abs);
-#endif
  out:
 	mutex_unlock(&trace_types_lock);
 
kernel/trace/trace.h +2 -2
index 16bedaaface4..de717706e863 100644
@@ -352,7 +352,7 @@ struct trace_array {
 	/* function tracing enabled */
 	int			function_enabled;
 #endif
-	int			time_stamp_abs_ref;
+	int			no_filter_buffering_ref;
 	struct list_head	hist_vars;
 #ifdef CONFIG_TRACER_SNAPSHOT
 	struct cond_snapshot	*cond_snapshot;
@@ -372,7 +372,7 @@ extern int tracing_check_open_get_tr(struct trace_array *tr);
 extern struct trace_array *trace_array_find(const char *instance);
 extern struct trace_array *trace_array_find_get(const char *instance);
 
-extern int tracing_set_time_stamp_abs(struct trace_array *tr, bool abs);
+extern int tracing_set_filter_buffering(struct trace_array *tr, bool set);
 extern int tracing_set_clock(struct trace_array *tr, const char *clockstr);
 
 extern bool trace_clock_in_ns(struct trace_array *tr);
kernel/trace/trace_events_hist.c +3 -3
index 05ead88cfb66..3090498acd98 100644
@@ -5483,7 +5483,7 @@ static int hist_register_trigger(char *glob, struct event_trigger_ops *ops,
 			goto out;
 		}
 
-		tracing_set_time_stamp_abs(file->tr, true);
+		tracing_set_filter_buffering(file->tr, true);
 	}
 
 	if (named_data)
@@ -5591,7 +5591,7 @@ static void hist_unregister_trigger(char *glob, struct event_trigger_ops *ops,
 
 	if (hist_data->enable_timestamps) {
 		if (!hist_data->remove || unregistered)
-			tracing_set_time_stamp_abs(file->tr, false);
+			tracing_set_filter_buffering(file->tr, false);
 	}
 }
 
@@ -5638,7 +5638,7 @@ static void hist_unreg_all(struct trace_event_file *file)
 
 			update_cond_flag(file);
 			if (hist_data->enable_timestamps)
-				tracing_set_time_stamp_abs(file->tr, false);
+				tracing_set_filter_buffering(file->tr, false);
 			if (test->ops->free)
 				test->ops->free(test->ops, test);
 		}
Signature · 1 line
2.30.1
author maintainer
#7 lore raw

PATCH 6/7 ring-buffer: Add verifier for using ring_buffer_event_time_stamp()

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

The ring_buffer_event_time_stamp() must be only called by an event that has not been committed yet, and is on the buffer that is passed in. This was used to help debug converting the histogram logic over to using the new time stamp code, and was proven to be very useful.

Add a verifier that can check that this is the case, and extra WARN_ONs to catch unexpected use cases.

---
 kernel/trace/ring_buffer.c | 56 +++++++++++++++++++++++++++++++++++---
 1 file changed, 52 insertions(+), 4 deletions(-)
kernel/trace/ring_buffer.c +52 -4
index 8fa2a84f714f..1c61a8cd7b99 100644
@@ -742,6 +742,48 @@ static bool rb_time_cmpxchg(rb_time_t *t, u64 expect, u64 set)
 }
 #endif
 
+/*
+ * Enable this to make sure that the event passed to
+ * ring_buffer_event_time_stamp() is not committed and also
+ * is on the buffer that it passed in.
+ */
+//#define RB_VERIFY_EVENT
+#ifdef RB_VERIFY_EVENT
+static struct list_head *rb_list_head(struct list_head *list);
+static void verify_event(struct ring_buffer_per_cpu *cpu_buffer,
+			 void *event)
+{
+	struct buffer_page *page = cpu_buffer->commit_page;
+	struct buffer_page *tail_page = READ_ONCE(cpu_buffer->tail_page);
+	struct list_head *next;
+	long commit, write;
+	unsigned long addr = (unsigned long)event;
+	bool done = false;
+	int stop = 0;
+
+	/* Make sure the event exists and is not committed yet */
+	do {
+		if (page == tail_page || WARN_ON_ONCE(stop++ > 100))
+			done = true;
+		commit = local_read(&page->page->commit);
+		write = local_read(&page->write);
+		if (addr >= (unsigned long)&page->page->data[commit] &&
+		    addr < (unsigned long)&page->page->data[write])
+			return;
+
+		next = rb_list_head(page->list.next);
+		page = list_entry(next, struct buffer_page, list);
+	} while (!done);
+	WARN_ON_ONCE(1);
+}
+#else
+static inline void verify_event(struct ring_buffer_per_cpu *cpu_buffer,
+			 void *event)
+{
+}
+#endif
+
+
 static inline u64 rb_time_stamp(struct trace_buffer *buffer);
 
 /**
@@ -772,13 +814,19 @@ u64 ring_buffer_event_time_stamp(struct trace_buffer *buffer,
 	if (event->type_len == RINGBUF_TYPE_TIME_STAMP)
 		return rb_event_time_stamp(event);
 
+	nest = local_read(&cpu_buffer->committing);
+	verify_event(cpu_buffer, event);
+	if (WARN_ON_ONCE(!nest))
+		goto fail;
+
 	/* Read the current saved nesting level time stamp */
-	nest = local_read(&cpu_buffer->committing) - 1;
-	if (likely(nest < MAX_NEST))
+	if (likely(--nest < MAX_NEST))
 		return cpu_buffer->event_stamp[nest];
 
-	WARN_ON_ONCE(1);
+	/* Shouldn't happen, warn if it does */
+	WARN_ONCE(1, "nest (%d) greater than max", nest);
 
+ fail:
 	/* Can only fail on 32 bit */
 	if (!rb_time_read(&cpu_buffer->write_stamp, &ts))
 		/* Screw it, just read the current time */
@@ -2750,7 +2798,7 @@ rb_update_event(struct ring_buffer_per_cpu *cpu_buffer,
 	u64 delta = info->delta;
 	unsigned int nest = local_read(&cpu_buffer->committing) - 1;
 
-	if (nest < MAX_NEST)
+	if (!WARN_ON_ONCE(nest >= MAX_NEST))
 		cpu_buffer->event_stamp[nest] = info->ts;
 
 	/*
Signature · 1 line
2.30.1
author maintainer
#8 lore raw

PATCH 7/7 tracing: Add tracing_event_time_stamp() API

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

Add a tracing_event_time_stamp() API that checks if the event passed in is not on the ring buffer but a pointer to the per CPU trace_buffered_event which does not have its time stamp set yet.

If it is a pointer to the trace_buffered_event, then just return the current time stamp that the ring buffer would produce.

Otherwise, return the time stamp from the event.

---
 kernel/trace/trace.c | 10 ++++++++++
 kernel/trace/trace.h |  1 +
 2 files changed, 11 insertions(+)
kernel/trace/trace.c +10 -0
index 8cf7397f3f56..a8d5356ed327 100644
@@ -7120,6 +7120,16 @@ static int tracing_time_stamp_mode_open(struct inode *inode, struct file *file)
 	return ret;
 }
 
+u64 tracing_event_time_stamp(struct trace_array *tr, struct ring_buffer_event *rbe)
+{
+	struct trace_buffer *buffer = tr->array_buffer.buffer;
+
+	if (rbe == this_cpu_read(trace_buffered_event))
+		return ring_buffer_time_stamp(buffer, smp_processor_id());
+
+	return ring_buffer_event_time_stamp(buffer, rbe);
+}
+
 /*
  * Set or disable using the per CPU trace_buffer_event when possible.
  */
kernel/trace/trace.h +1 -0
index de717706e863..23d85e78bef4 100644
@@ -372,6 +372,7 @@ extern int tracing_check_open_get_tr(struct trace_array *tr);
 extern struct trace_array *trace_array_find(const char *instance);
 extern struct trace_array *trace_array_find_get(const char *instance);
 
+extern u64 tracing_event_time_stamp(struct trace_array *tr, struct ring_buffer_event *rbe);
 extern int tracing_set_filter_buffering(struct trace_array *tr, bool set);
 extern int tracing_set_clock(struct trace_array *tr, const char *clockstr);
 
Signature · 1 line
2.30.1
author maintainer
#9 lore raw

PATCH 7/7 tracing: Add tracing_event_time_stamp() API

From: "Steven Rostedt (VMware)" <[email protected]>

Add a tracing_event_time_stamp() API that checks if the event passed in is not on the ring buffer but a pointer to the per CPU trace_buffered_event which does not have its time stamp set yet.

If it is a pointer to the trace_buffered_event, then just return the current time stamp that the ring buffer would produce.

Otherwise, return the time stamp from the event.

---

I updated this patch but never committed the changes, before sending to the list. There's no reason to pass in the trace array, when the trace buffer would suffice.

 kernel/trace/trace.c | 8 ++++++++
 kernel/trace/trace.h | 1 +
 2 files changed, 9 insertions(+)
kernel/trace/trace.c +8 -0
index 8cf7397f3f56..00797a58971d 100644
@@ -7120,6 +7120,14 @@ static int tracing_time_stamp_mode_open(struct inode *inode, struct file *file)
 	return ret;
 }
 
+u64 tracing_event_time_stamp(struct trace_buffer *buffer, struct ring_buffer_event *rbe)
+{
+	if (rbe == this_cpu_read(trace_buffered_event))
+		return ring_buffer_time_stamp(buffer, smp_processor_id());
+
+	return ring_buffer_event_time_stamp(buffer, rbe);
+}
+
 /*
  * Set or disable using the per CPU trace_buffer_event when possible.
  */
kernel/trace/trace.h +1 -0
index de717706e863..50f944e0b5a1 100644
@@ -372,6 +372,7 @@ extern int tracing_check_open_get_tr(struct trace_array *tr);
 extern struct trace_array *trace_array_find(const char *instance);
 extern struct trace_array *trace_array_find_get(const char *instance);
 
+extern u64 tracing_event_time_stamp(struct trace_buffer *buffer, struct ring_buffer_event *rbe);
 extern int tracing_set_filter_buffering(struct trace_array *tr, bool set);
 extern int tracing_set_clock(struct trace_array *tr, const char *clockstr);
 
Signature · 1 line
2.29.2