Skip to contentSkip to conversation

perf_event: fix loss of notification with multi-event

PATCH Stephane Eranian 3 participants 5 messages Nov 26, 2011 – Dec 5, 2011 perf events

lore
Share & tools
lore

Stephane Eranian

author Reviewed-by
#0 lore raw

Here is a fixed version of Peter's patch. I fixed the compilation errors + added the missing linked list + spinlock initializations.

I also added the mmap_mutex in perf_poll() to avoid the race with perf_set_output(). It's not pretty but it solves the problem of a process getting stuck on a sampling buffer which is never going to get any samples. This was an unlikely scenario but we should take care of it anyway.

This patch solves the issue with loss of buffer notifications when sampling on multiple events at the same time. In that case fewer than expected samples would be captured.

---

include/linux/perf_event.h +1 -0
index 1e9ebe5..b1f8912 100644
@@ -822,6 +822,7 @@ struct perf_event {
 	int				mmap_locked;
 	struct user_struct		*mmap_user;
 	struct ring_buffer		*rb;
+	struct list_head		rb_entry;
 
 	/* poll related */
 	wait_queue_head_t		waitq;
kernel/events/core.c +84 -2

142 lines not on this page · raw message

kernel/events/internal.h +3 -0
index be4a43f..b0b107f 100644
@@ -26,6 +26,9 @@ struct ring_buffer {
 	local_t				lost;		/* nr records lost   */
 
 	long				watermark;	/* wakeup watermark  */
+	/* poll crap */
+	spinlock_t			event_lock;
+	struct list_head		event_list;
 
 	struct perf_event_mmap_page	*user_page;
 	void				*data_pages[0];
kernel/events/ring_buffer.c +3 -0
index a2a2920..7f3011c 100644
@@ -209,6 +209,9 @@ ring_buffer_init(struct ring_buffer *rb, long watermark, int flags)
 		rb->writable = 1;
 
 	atomic_set(&rb->refcount, 1);
+
+	INIT_LIST_HEAD(&rb->event_list);
+	spin_lock_init(&rb->event_lock);
 }
 
 #ifndef CONFIG_PERF_USE_VMALLOC
maintainer
#1 lore raw
Quoted 10 lines · Stephane Eranian

Here is a fixed version of Peter's patch. I fixed the compilation errors + added the missing linked list + spinlock initializations.

I also added the mmap_mutex in perf_poll() to avoid the race with perf_set_output(). It's not pretty but it solves the problem of a process getting stuck on a sampling buffer which is never going to get any samples. This was an unlikely scenario but we should take care of it anyway.

Quoted 3 lines

This patch solves the issue with loss of buffer notifications when sampling on multiple events at the same time. In that case fewer than expected samples would be captured.

Hehe, so it actually works eh.. amazing stuff ;-)

OK, I'll stick it in perf/urgent.. Thanks Stephane!

If anybody happens upon a prettier solution, don't be shy and please do tell us ;-)

maintainer
#2 lore raw
Quoted 1 line · Peter Zijlstra

OK, I'll stick it in perf/urgent.. Thanks Stephane!

I'll use you as test-subject for my quilt2git script ( http://programming.kicks-ass.net/misc/quilt2git.sh )

My exploded quilt tree lives at:

git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git

With this particular commit:

http://git.kernel.org/?p=linux/kernel/git/peterz/queue.git;a=commitdiff;h=85d947a9b8113814ee0b8469ff9563d110223874

in the perf/urgent branch.

maintainer
#3 lore raw
Quoted 9 lines · Peter Zijlstra
Quoted 1 line · Peter Zijlstra

OK, I'll stick it in perf/urgent.. Thanks Stephane!

I'll use you as test-subject for my quilt2git script ( http://programming.kicks-ass.net/misc/quilt2git.sh )

My exploded quilt tree lives at:

git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git

With a note that this tree is completely regenerated every time I run this script, so its a complete rebase. Its only purpose in life is to expose my quilt tree in git topic branch form for whoemever thinks this is more convenient than grabbing my tarball and fiddling with quilt.

Also note that the master branch of the tree includes all topic master branches, which in turn include all topic subject branches. IOW the master branch contains my complete quilt tree of the moment (which might or might not actually compile, eat your pet or cook you dinner).

#4 lore raw

[tip:perf/urgent] perf: Fix loss of notification with multi-event

In reply to Stephane Eranian
Commit-ID:  10c6db110d0eb4466b59812c49088ab56218fc2e
Gitweb:     http://git.kernel.org/tip/10c6db110d0eb4466b59812c49088ab56218fc2e
Author:     Peter Zijlstra <[email protected]>
AuthorDate: Sat, 26 Nov 2011 02:47:31 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Mon, 5 Dec 2011 09:33:03 +0100

perf: Fix loss of notification with multi-event

When you do:

        $ perf record -e cycles,cycles,cycles noploop 10

You expect about 10,000 samples for each event, i.e., 10s at 1000samples/sec. However, this is not what's happening. You get much fewer samples, maybe 3700 samples/event:

$ perf report -D | tail -15
Aggregated stats:
           TOTAL events:      10998
            MMAP events:         66
            COMM events:          2
          SAMPLE events:      10930
cycles stats:
           TOTAL events:       3644
          SAMPLE events:       3644
cycles stats:
           TOTAL events:       3642
          SAMPLE events:       3642
cycles stats:
           TOTAL events:       3644
          SAMPLE events:       3644

On a Intel Nehalem or even AMD64, there are 4 counters capable of measuring cycles, so there is plenty of space to measure those events without multiplexing (even with the NMI watchdog active). And even with multiplexing, we'd expect roughly the same number of samples per event.

The root of the problem was that when the event that caused the buffer to become full was not the first event passed on the cmdline, the user notification would get lost. The notification was sent to the file descriptor of the overflowed event but the perf tool was not polling on it. The perf tool aggregates all samples into a single buffer, i.e., the buffer of the first event. Consequently, it assumes notifications for any event will come via that descriptor.

The seemingly straight forward solution of moving the waitq into the ringbuffer object doesn't work because of life-time issues. One could perf_event_set_output() on a fd that you're also blocking on and cause the old rb object to be freed while its waitq would still be referenced by the blocked thread -> FAIL.

Therefore link all events to the ringbuffer and broadcast the wakeup from the ringbuffer object to all possible events that could be waited upon. This is rather ugly, and we're open to better solutions but it works for now.

---
 include/linux/perf_event.h  |    1 +
 kernel/events/core.c        |   86 ++++++++++++++++++++++++++++++++++++++++++-
 kernel/events/internal.h    |    3 +
 kernel/events/ring_buffer.c |    3 +
 4 files changed, 91 insertions(+), 2 deletions(-)
include/linux/perf_event.h +1 -0
index 1e9ebe5..b1f8912 100644
@@ -822,6 +822,7 @@ struct perf_event {
 	int				mmap_locked;
 	struct user_struct		*mmap_user;
 	struct ring_buffer		*rb;
+	struct list_head		rb_entry;
 
 	/* poll related */
 	wait_queue_head_t		waitq;
kernel/events/core.c +84 -2

142 lines not on this page · raw message

kernel/events/internal.h +3 -0
index 09097dd..64568a6 100644
@@ -22,6 +22,9 @@ struct ring_buffer {
 	local_t				lost;		/* nr records lost   */
 
 	long				watermark;	/* wakeup watermark  */
+	/* poll crap */
+	spinlock_t			event_lock;
+	struct list_head		event_list;
 
 	struct perf_event_mmap_page	*user_page;
 	void				*data_pages[0];
kernel/events/ring_buffer.c +3 -0
index a2a2920..7f3011c 100644
@@ -209,6 +209,9 @@ ring_buffer_init(struct ring_buffer *rb, long watermark, int flags)
 		rb->writable = 1;
 
 	atomic_set(&rb->refcount, 1);
+
+	INIT_LIST_HEAD(&rb->event_list);
+	spin_lock_init(&rb->event_lock);
 }
 
 #ifndef CONFIG_PERF_USE_VMALLOC