Skip to contentSkip to conversation

perf/aux: Only update aux_wakeup in non-overwrite mode

PATCH In review Alexander Shishkin 4 participants 6 messages Sep 6, 2017 – Sep 29, 2017 perf events

lore
Share & tools
lore

Alexander Shishkin

author reviewer
#0 lore raw

PATCH 1/1 perf/aux: Only update aux_wakeup in non-overwrite mode

Commit d9a50b0256 ("perf/aux: Ensure aux_wakeup represents most recent wakeup index") changed aux wakeup position calculation to rounddown(), which causes a division-by-zero in AUX overwrite mode (aka "snapshot mode").

The zero denominator results from the fact that perf record doesn't set aux_watermark to anything, in which case the kernel will set it to half the AUX buffer size, but only for non-overwrite mode. In the overwrite mode aux_watermark stays zero.

The good news is that, AUX overwrite mode, wakeups don't happen and related bookkeeping is not relevant, so we can simply forego the whole wakeup updates.

---
 kernel/events/ring_buffer.c | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)
kernel/events/ring_buffer.c +15 -5
index f4ebe42879..4dae1da4c1 100644
@@ -412,6 +412,19 @@ void *perf_aux_output_begin(struct perf_output_handle *handle,
 	return NULL;
 }
 
+static bool __always_inline rb_need_aux_wakeup(struct ring_buffer *rb)
+{
+	if (rb->aux_overwrite)
+		return false;
+
+	if (rb->aux_head - rb->aux_wakeup >= rb->aux_watermark) {
+		rb->aux_wakeup = rounddown(rb->aux_head, rb->aux_watermark);
+		return true;
+	}
+
+	return false;
+}
+
 /*
  * Commit the data written by hardware into the ring buffer by adjusting
  * aux_head and posting a PERF_RECORD_AUX into the perf buffer. It is the
@@ -451,10 +464,8 @@ void perf_aux_output_end(struct perf_output_handle *handle, unsigned long size)
 	}
 
 	rb->user_page->aux_head = rb->aux_head;
-	if (rb->aux_head - rb->aux_wakeup >= rb->aux_watermark) {
+	if (rb_need_aux_wakeup(rb))
 		wakeup = true;
-		rb->aux_wakeup = rounddown(rb->aux_head, rb->aux_watermark);
-	}
 
 	if (wakeup) {
 		if (handle->aux_flags & PERF_AUX_FLAG_TRUNCATED)
@@ -484,9 +495,8 @@ int perf_aux_output_skip(struct perf_output_handle *handle, unsigned long size)
 	rb->aux_head += size;
 
 	rb->user_page->aux_head = rb->aux_head;
-	if (rb->aux_head - rb->aux_wakeup >= rb->aux_watermark) {
+	if (rb_need_aux_wakeup(rb)) {
 		perf_output_wakeup(handle);
-		rb->aux_wakeup = rounddown(rb->aux_head, rb->aux_watermark);
 		handle->wakeup = rb->aux_wakeup + rb->aux_watermark;
 	}
 
Signature · 1 line
2.14.1
Acked-by on v1 1/1
#1 lore raw

Hi Alexander,

Quoted 10 lines · Alexander Shishkin

Commit d9a50b0256 ("perf/aux: Ensure aux_wakeup represents most recent wakeup index") changed aux wakeup position calculation to rounddown(), which causes a division-by-zero in AUX overwrite mode (aka "snapshot mode").

The zero denominator results from the fact that perf record doesn't set aux_watermark to anything, in which case the kernel will set it to half the AUX buffer size, but only for non-overwrite mode. In the overwrite mode aux_watermark stays zero.

Quoted 8 lines

The good news is that, AUX overwrite mode, wakeups don't happen and related bookkeeping is not relevant, so we can simply forego the whole wakeup updates.

---
 kernel/events/ring_buffer.c | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

Damn, sorry about that. How did you spot the problem? Anyway, I think the code is much better with this factored out:

Thanks,

Will

Quoted 50 lines
kernel/events/ring_buffer.c +15 -5
index f4ebe42879..4dae1da4c1 100644
@@ -412,6 +412,19 @@ void *perf_aux_output_begin(struct perf_output_handle *handle,
 	return NULL;
 }
 
+static bool __always_inline rb_need_aux_wakeup(struct ring_buffer *rb)
+{
+	if (rb->aux_overwrite)
+		return false;
+
+	if (rb->aux_head - rb->aux_wakeup >= rb->aux_watermark) {
+		rb->aux_wakeup = rounddown(rb->aux_head, rb->aux_watermark);
+		return true;
+	}
+
+	return false;
+}
+
 /*
  * Commit the data written by hardware into the ring buffer by adjusting
  * aux_head and posting a PERF_RECORD_AUX into the perf buffer. It is the
@@ -451,10 +464,8 @@ void perf_aux_output_end(struct perf_output_handle *handle, unsigned long size)
 	}
 
 	rb->user_page->aux_head = rb->aux_head;
-	if (rb->aux_head - rb->aux_wakeup >= rb->aux_watermark) {
+	if (rb_need_aux_wakeup(rb))
 		wakeup = true;
-		rb->aux_wakeup = rounddown(rb->aux_head, rb->aux_watermark);
-	}
 
 	if (wakeup) {
 		if (handle->aux_flags & PERF_AUX_FLAG_TRUNCATED)
@@ -484,9 +495,8 @@ int perf_aux_output_skip(struct perf_output_handle *handle, unsigned long size)
 	rb->aux_head += size;
 
 	rb->user_page->aux_head = rb->aux_head;
-	if (rb->aux_head - rb->aux_wakeup >= rb->aux_watermark) {
+	if (rb_need_aux_wakeup(rb)) {
 		perf_output_wakeup(handle);
-		rb->aux_wakeup = rounddown(rb->aux_head, rb->aux_watermark);
 		handle->wakeup = rb->aux_wakeup + rb->aux_watermark;
 	}
 
Signature · 1 line
2.14.1
author reviewer
#2 lore raw
Quoted 17 lines · Will Deacon

Hi Alexander,

Quoted 14 lines · Alexander Shishkin

Commit d9a50b0256 ("perf/aux: Ensure aux_wakeup represents most recent wakeup index") changed aux wakeup position calculation to rounddown(), which causes a division-by-zero in AUX overwrite mode (aka "snapshot mode").

The zero denominator results from the fact that perf record doesn't set aux_watermark to anything, in which case the kernel will set it to half the AUX buffer size, but only for non-overwrite mode. In the overwrite mode aux_watermark stays zero.

The good news is that, AUX overwrite mode, wakeups don't happen and related bookkeeping is not relevant, so we can simply forego the whole wakeup updates.

Quoted 6 lines
Quoted 4 lines
---
 kernel/events/ring_buffer.c | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

Damn, sorry about that. How did you spot the problem?

A normal perf record with -S should trigger it, I don't remember what exactly I was doing at that moment. But no worries, we all missed it the first time around. :)

Quoted 3 lines

Anyway, I think the code is much better with this factored out:

Thanks!

#3 lore raw

Hi Alexander,

Quoted 27 lines · Alexander Shishkin
Quoted 15 lines · Will Deacon
Quoted 14 lines · Alexander Shishkin

Commit d9a50b0256 ("perf/aux: Ensure aux_wakeup represents most recent wakeup index") changed aux wakeup position calculation to rounddown(), which causes a division-by-zero in AUX overwrite mode (aka "snapshot mode").

The zero denominator results from the fact that perf record doesn't set aux_watermark to anything, in which case the kernel will set it to half the AUX buffer size, but only for non-overwrite mode. In the overwrite mode aux_watermark stays zero.

The good news is that, AUX overwrite mode, wakeups don't happen and related bookkeeping is not relevant, so we can simply forego the whole wakeup updates.

Quoted 6 lines
Quoted 4 lines
---
 kernel/events/ring_buffer.c | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

Damn, sorry about that. How did you spot the problem?

A normal perf record with -S should trigger it, I don't remember what exactly I was doing at that moment. But no worries, we all missed it the first time around. :)

Quoted 5 lines
Quoted 3 lines

Anyway, I think the code is much better with this factored out:

Thanks!

What's the status on this patch? I don't see it in mainline or linux-next, but it would ideally get in as a fix for 4.14.

Will

maintainer
#4 lore raw
Quoted 2 lines · Will Deacon

What's the status on this patch? I don't see it in mainline or linux-next, but it would ideally get in as a fix for 4.14.

Its in the patch queue I'm trying to get mingo to merge ;-) It is indeed queued for /urgent.

#5 lore raw

[tip:perf/urgent] perf/aux: Only update ->aux_wakeup in non-overwrite mode

In reply to Alexander Shishkin
perf/aux: Only update ->aux_wakeup in non-overwrite mode
Commit-ID:  441430eb54a00586f95f1aefc48e0801bbd6a923
Gitweb:     https://git.kernel.org/tip/441430eb54a00586f95f1aefc48e0801bbd6a923
Author:     Alexander Shishkin <[email protected]>
AuthorDate: Wed, 6 Sep 2017 19:08:11 +0300
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 29 Sep 2017 10:06:45 +0200

perf/aux: Only update ->aux_wakeup in non-overwrite mode

The following commit:

d9a50b0256 ("perf/aux: Ensure aux_wakeup represents most recent wakeup index")

changed the AUX wakeup position calculation to rounddown(), which causes a division-by-zero in AUX overwrite mode (aka "snapshot mode").

The zero denominator results from the fact that perf record doesn't set aux_watermark to anything, in which case the kernel will set it to half the AUX buffer size, but only for non-overwrite mode. In the overwrite mode aux_watermark stays zero.

The good news is that, AUX overwrite mode, wakeups don't happen and related bookkeeping is not relevant, so we can simply forego the whole wakeup updates.

---
 kernel/events/ring_buffer.c | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)
kernel/events/ring_buffer.c +15 -5
index af71a84e..f684d8e 100644
@@ -412,6 +412,19 @@ err:
 	return NULL;
 }
 
+static bool __always_inline rb_need_aux_wakeup(struct ring_buffer *rb)
+{
+	if (rb->aux_overwrite)
+		return false;
+
+	if (rb->aux_head - rb->aux_wakeup >= rb->aux_watermark) {
+		rb->aux_wakeup = rounddown(rb->aux_head, rb->aux_watermark);
+		return true;
+	}
+
+	return false;
+}
+
 /*
  * Commit the data written by hardware into the ring buffer by adjusting
  * aux_head and posting a PERF_RECORD_AUX into the perf buffer. It is the
@@ -451,10 +464,8 @@ void perf_aux_output_end(struct perf_output_handle *handle, unsigned long size)
 	}
 
 	rb->user_page->aux_head = rb->aux_head;
-	if (rb->aux_head - rb->aux_wakeup >= rb->aux_watermark) {
+	if (rb_need_aux_wakeup(rb))
 		wakeup = true;
-		rb->aux_wakeup = rounddown(rb->aux_head, rb->aux_watermark);
-	}
 
 	if (wakeup) {
 		if (handle->aux_flags & PERF_AUX_FLAG_TRUNCATED)
@@ -484,9 +495,8 @@ int perf_aux_output_skip(struct perf_output_handle *handle, unsigned long size)
 	rb->aux_head += size;
 
 	rb->user_page->aux_head = rb->aux_head;
-	if (rb->aux_head - rb->aux_wakeup >= rb->aux_watermark) {
+	if (rb_need_aux_wakeup(rb)) {
 		perf_output_wakeup(handle);
-		rb->aux_wakeup = rounddown(rb->aux_head, rb->aux_watermark);
 		handle->wakeup = rb->aux_wakeup + rb->aux_watermark;
 	}