Skip to contentSkip to conversation

perf: Fix race in perf_mmap_close function

PATCH In review Jiri Olsa 6 participants 13 messages Sep 10, 2020 – Oct 12, 2020 perf events

lore
Share & tools
lore

Jiri Olsa

author reviewer
#0 lore raw

PATCH 1/1 perf: Fix race in perf_mmap_close function

There's a possible race in perf_mmap_close when checking ring buffer's mmap_count refcount value. The problem is that the mmap_count check is not atomic because we call atomic_dec and atomic_read separately.

  perf_mmap_close:
  ...
   atomic_dec(&rb->mmap_count);
   ...
   if (atomic_read(&rb->mmap_count))
      goto out_put;

<ring buffer detach>
free_uid

out_put:
ring_buffer_put(rb); /* could be last */

The race can happen when we have two (or more) events sharing same ring buffer and they go through atomic_dec and then they both see 0 as refcount value later in atomic_read. Then both will go on and execute code which is meant to be run just once.

The code that detaches ring buffer is probably fine to be executed more than once, but the problem is in calling free_uid, which will later on demonstrate in related crashes and refcount warnings, like:

refcount_t: addition on 0; use-after-free.
...

  RIP: 0010:refcount_warn_saturate+0x6d/0xf
  ...
  Call Trace:
  prepare_creds+0x190/0x1e0
  copy_creds+0x35/0x172
  copy_process+0x471/0x1a80
  _do_fork+0x83/0x3a0
  __do_sys_wait4+0x83/0x90
  __do_sys_clone+0x85/0xa0
  do_syscall_64+0x5b/0x1e0
  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Using atomic decrease and check instead of separated calls. This fixes CVE-2020-14351.

---
 kernel/events/core.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
kernel/events/core.c +1 -3
index 7ed5248f0445..29313cc54d9e 100644
@@ -5903,8 +5903,6 @@ static void perf_mmap_close(struct vm_area_struct *vma)
 		mutex_unlock(&event->mmap_mutex);
 	}
 
-	atomic_dec(&rb->mmap_count);
-
 	if (!atomic_dec_and_mutex_lock(&event->mmap_count, &event->mmap_mutex))
 		goto out_put;
 
@@ -5912,7 +5910,7 @@ static void perf_mmap_close(struct vm_area_struct *vma)
 	mutex_unlock(&event->mmap_mutex);
 
 	/* If there's still other mmap()s of this buffer, we're done. */
-	if (atomic_read(&rb->mmap_count))
+	if (!atomic_dec_and_test(&rb->mmap_count))
 		goto out_put;
 
 	/*
Signature · 1 line
2.26.2
maintainer
#1 lore raw
Quoted 54 lines · Jiri Olsa

There's a possible race in perf_mmap_close when checking ring buffer's mmap_count refcount value. The problem is that the mmap_count check is not atomic because we call atomic_dec and atomic_read separately.

  perf_mmap_close:
  ...
   atomic_dec(&rb->mmap_count);
   ...
   if (atomic_read(&rb->mmap_count))
      goto out_put;

<ring buffer detach>
free_uid

out_put:
ring_buffer_put(rb); /* could be last */

The race can happen when we have two (or more) events sharing same ring buffer and they go through atomic_dec and then they both see 0 as refcount value later in atomic_read. Then both will go on and execute code which is meant to be run just once.

The code that detaches ring buffer is probably fine to be executed more than once, but the problem is in calling free_uid, which will later on demonstrate in related crashes and refcount warnings, like:

refcount_t: addition on 0; use-after-free.
...

  RIP: 0010:refcount_warn_saturate+0x6d/0xf
  ...
  Call Trace:
  prepare_creds+0x190/0x1e0
  copy_creds+0x35/0x172
  copy_process+0x471/0x1a80
  _do_fork+0x83/0x3a0
  __do_sys_wait4+0x83/0x90
  __do_sys_clone+0x85/0xa0
  do_syscall_64+0x5b/0x1e0
  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Using atomic decrease and check instead of separated calls. This fixes CVE-2020-14351.

---
 kernel/events/core.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
kernel/events/core.c +0 -0
index 7ed5248f0445..29313cc54d9e 100644

kernel/events/core.c static void perf_mmap_close(struct vm_area_struct *vma) · line 5907

Quoted 7 lines
kernel/events/core.c +0 -2
@@ -5903,8 +5903,6 @@ static void perf_mmap_close(struct vm_area_struct *vma)
                mutex_unlock(&event->mmap_mutex);
        }
 
-       atomic_dec(&rb->mmap_count);
-
        if (!atomic_dec_and_mutex_lock(&event->mmap_count, &event->mmap_mutex))
                goto out_put;

But when it takes the goto, rb->mmap_count won't decrement anymore..

Thanks
Namhyung

Quoted 13 lines
kernel/events/core.c +1 -2
@@ -5912,7 +5910,7 @@ static void perf_mmap_close(struct vm_area_struct *vma)
        mutex_unlock(&event->mmap_mutex);
 
        /* If there's still other mmap()s of this buffer, we're done. */
-       if (atomic_read(&rb->mmap_count))
+       if (!atomic_dec_and_test(&rb->mmap_count))
                goto out_put;
 
        /*
--

2.26.2

#2 lore raw

On Thu, Sep 10, 2020 at 10:48:02PM +0900, Namhyung Kim wrote:

SNIP

Quoted 22 lines
Quoted 22 lines
  _do_fork+0x83/0x3a0
  __do_sys_wait4+0x83/0x90
  __do_sys_clone+0x85/0xa0
  do_syscall_64+0x5b/0x1e0
  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Using atomic decrease and check instead of separated calls. This fixes CVE-2020-14351.

---
 kernel/events/core.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
kernel/events/core.c +0 -0
index 7ed5248f0445..29313cc54d9e 100644
@@ -5903,8 +5903,6 @@ static void perf_mmap_close(struct vm_area_struct *vma)
                mutex_unlock(&event->mmap_mutex);
        }
 
Quoted 6 lines

kernel/events/core.c static void perf_mmap_close(struct vm_area_struct *vma) · line 5907

Quoted 4 lines
kernel/events/core.c +0 -2
-       atomic_dec(&rb->mmap_count);
-
        if (!atomic_dec_and_mutex_lock(&event->mmap_count, &event->mmap_mutex))
                goto out_put;

But when it takes the goto, rb->mmap_count won't decrement anymore..

event->mmap_count is per event, so if we have have race in here, 2 threads can go through with each event->mmap_count reaching zero

jirka

maintainer
#3 lore raw

Hi Jiri,

Quoted 34 lines · Jiri Olsa

On Thu, Sep 10, 2020 at 10:48:02PM +0900, Namhyung Kim wrote:

SNIP

Quoted 22 lines
Quoted 22 lines
  _do_fork+0x83/0x3a0
  __do_sys_wait4+0x83/0x90
  __do_sys_clone+0x85/0xa0
  do_syscall_64+0x5b/0x1e0
  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Using atomic decrease and check instead of separated calls. This fixes CVE-2020-14351.

---
 kernel/events/core.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
kernel/events/core.c +0 -0
index 7ed5248f0445..29313cc54d9e 100644
@@ -5903,8 +5903,6 @@ static void perf_mmap_close(struct vm_area_struct *vma)
                mutex_unlock(&event->mmap_mutex);
        }
 
Quoted 6 lines

kernel/events/core.c static void perf_mmap_close(struct vm_area_struct *vma) · line 5907

Quoted 4 lines
kernel/events/core.c +0 -2
-       atomic_dec(&rb->mmap_count);
-
        if (!atomic_dec_and_mutex_lock(&event->mmap_count, &event->mmap_mutex))
                goto out_put;

But when it takes the goto, rb->mmap_count won't decrement anymore..

Quoted 2 lines

event->mmap_count is per event, so if we have have race in here, 2 threads can go through with each event->mmap_count reaching zero

Maybe I'm missing something.

But as far as I can see, perf_mmap_close() always decremented both rb->mmap_count and event->mmap_count. But with this change, it seems not decrement rb->mmap_count when event->mmap_count doesn't go to zero, right?

Thanks
Namhyung

#4 lore raw
Quoted 40 lines · Namhyung Kim

Hi Jiri,

Quoted 34 lines · Jiri Olsa

On Thu, Sep 10, 2020 at 10:48:02PM +0900, Namhyung Kim wrote:

SNIP

Quoted 22 lines
Quoted 22 lines
  _do_fork+0x83/0x3a0
  __do_sys_wait4+0x83/0x90
  __do_sys_clone+0x85/0xa0
  do_syscall_64+0x5b/0x1e0
  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Using atomic decrease and check instead of separated calls. This fixes CVE-2020-14351.

---
 kernel/events/core.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
kernel/events/core.c +0 -0
index 7ed5248f0445..29313cc54d9e 100644
@@ -5903,8 +5903,6 @@ static void perf_mmap_close(struct vm_area_struct *vma)
                mutex_unlock(&event->mmap_mutex);
        }
 
Quoted 6 lines

kernel/events/core.c static void perf_mmap_close(struct vm_area_struct *vma) · line 5907

Quoted 4 lines
kernel/events/core.c +0 -2
-       atomic_dec(&rb->mmap_count);
-
        if (!atomic_dec_and_mutex_lock(&event->mmap_count, &event->mmap_mutex))
                goto out_put;

But when it takes the goto, rb->mmap_count won't decrement anymore..

Quoted 2 lines

event->mmap_count is per event, so if we have have race in here, 2 threads can go through with each event->mmap_count reaching zero

Quoted 6 lines

Maybe I'm missing something.

But as far as I can see, perf_mmap_close() always decremented both rb->mmap_count and event->mmap_count. But with this change, it seems not decrement rb->mmap_count when event->mmap_count doesn't go to zero, right?

ugh, that's right.. how about change below

jirka

---

kernel/events/core.c +4 -3
index 7ed5248f0445..8ab2400aef55 100644
@@ -5868,11 +5868,11 @@ static void perf_pmu_output_stop(struct perf_event *event);
 static void perf_mmap_close(struct vm_area_struct *vma)
 {
 	struct perf_event *event = vma->vm_file->private_data;
-
 	struct perf_buffer *rb = ring_buffer_get(event);
 	struct user_struct *mmap_user = rb->mmap_user;
 	int mmap_locked = rb->mmap_locked;
 	unsigned long size = perf_data_size(rb);
+	bool detach_rest = false;
 
 	if (event->pmu->event_unmapped)
 		event->pmu->event_unmapped(event, vma->vm_mm);
@@ -5903,7 +5903,8 @@ static void perf_mmap_close(struct vm_area_struct *vma)
 		mutex_unlock(&event->mmap_mutex);
 	}
 
-	atomic_dec(&rb->mmap_count);
+	if (atomic_dec_and_test(&rb->mmap_count))
+		detach_rest = true;
 
 	if (!atomic_dec_and_mutex_lock(&event->mmap_count, &event->mmap_mutex))
 		goto out_put;
@@ -5912,7 +5913,7 @@ static void perf_mmap_close(struct vm_area_struct *vma)
 	mutex_unlock(&event->mmap_mutex);
 
 	/* If there's still other mmap()s of this buffer, we're done. */
-	if (atomic_read(&rb->mmap_count))
+	if (!detach_rest)
 		goto out_put;
 
 	/*
maintainer Acked-by
#5 lore raw
Quoted 1 line · Jiri Olsa

ugh, that's right.. how about change below

Thanks
Namhyung

Quoted 42 lines

jirka

---

kernel/events/core.c +4 -3
index 7ed5248f0445..8ab2400aef55 100644
@@ -5868,11 +5868,11 @@ static void perf_pmu_output_stop(struct perf_event *event);
 static void perf_mmap_close(struct vm_area_struct *vma)
 {
        struct perf_event *event = vma->vm_file->private_data;
-
        struct perf_buffer *rb = ring_buffer_get(event);
        struct user_struct *mmap_user = rb->mmap_user;
        int mmap_locked = rb->mmap_locked;
        unsigned long size = perf_data_size(rb);
+       bool detach_rest = false;
 
        if (event->pmu->event_unmapped)
                event->pmu->event_unmapped(event, vma->vm_mm);
@@ -5903,7 +5903,8 @@ static void perf_mmap_close(struct vm_area_struct *vma)
                mutex_unlock(&event->mmap_mutex);
        }
 
-       atomic_dec(&rb->mmap_count);
+       if (atomic_dec_and_test(&rb->mmap_count))
+               detach_rest = true;
 
        if (!atomic_dec_and_mutex_lock(&event->mmap_count, &event->mmap_mutex))
                goto out_put;
@@ -5912,7 +5913,7 @@ static void perf_mmap_close(struct vm_area_struct *vma)
        mutex_unlock(&event->mmap_mutex);
 
        /* If there's still other mmap()s of this buffer, we're done. */
-       if (atomic_read(&rb->mmap_count))
+       if (!detach_rest)
                goto out_put;
 
        /*
#6 lore raw
Quoted 4 lines · Namhyung Kim
Quoted 1 line · Jiri Olsa

ugh, that's right.. how about change below

thanks, I'll send full patch after we're done testing this

jirka

Quoted 48 lines

Thanks
Namhyung

Quoted 42 lines

jirka

---

kernel/events/core.c +4 -3
index 7ed5248f0445..8ab2400aef55 100644
@@ -5868,11 +5868,11 @@ static void perf_pmu_output_stop(struct perf_event *event);
 static void perf_mmap_close(struct vm_area_struct *vma)
 {
        struct perf_event *event = vma->vm_file->private_data;
-
        struct perf_buffer *rb = ring_buffer_get(event);
        struct user_struct *mmap_user = rb->mmap_user;
        int mmap_locked = rb->mmap_locked;
        unsigned long size = perf_data_size(rb);
+       bool detach_rest = false;
 
        if (event->pmu->event_unmapped)
                event->pmu->event_unmapped(event, vma->vm_mm);
@@ -5903,7 +5903,8 @@ static void perf_mmap_close(struct vm_area_struct *vma)
                mutex_unlock(&event->mmap_mutex);
        }
 
-       atomic_dec(&rb->mmap_count);
+       if (atomic_dec_and_test(&rb->mmap_count))
+               detach_rest = true;
 
        if (!atomic_dec_and_mutex_lock(&event->mmap_count, &event->mmap_mutex))
                goto out_put;
@@ -5912,7 +5913,7 @@ static void perf_mmap_close(struct vm_area_struct *vma)
        mutex_unlock(&event->mmap_mutex);
 
        /* If there's still other mmap()s of this buffer, we're done. */
-       if (atomic_read(&rb->mmap_count))
+       if (!detach_rest)
                goto out_put;
 
        /*
Tested-by
#7 lore raw
Quoted 9 lines · Jiri Olsa
Quoted 4 lines · Namhyung Kim
Quoted 1 line · Jiri Olsa

ugh, that's right.. how about change below

thanks, I'll send full patch after we're done testing this

jirka

I've tested the change and seems OK to me.

Quoted 50 lines
Quoted 49 lines

Thanks
Namhyung

Quoted 42 lines

jirka

---

kernel/events/core.c +4 -3
index 7ed5248f0445..8ab2400aef55 100644
@@ -5868,11 +5868,11 @@ static void perf_pmu_output_stop(struct perf_event *event);
 static void perf_mmap_close(struct vm_area_struct *vma)
 {
        struct perf_event *event = vma->vm_file->private_data;
-
        struct perf_buffer *rb = ring_buffer_get(event);
        struct user_struct *mmap_user = rb->mmap_user;
        int mmap_locked = rb->mmap_locked;
        unsigned long size = perf_data_size(rb);
+       bool detach_rest = false;
 
        if (event->pmu->event_unmapped)
                event->pmu->event_unmapped(event, vma->vm_mm);
@@ -5903,7 +5903,8 @@ static void perf_mmap_close(struct vm_area_struct *vma)
                mutex_unlock(&event->mmap_mutex);
        }
 
-       atomic_dec(&rb->mmap_count);
+       if (atomic_dec_and_test(&rb->mmap_count))
+               detach_rest = true;
 
        if (!atomic_dec_and_mutex_lock(&event->mmap_count, &event->mmap_mutex))
                goto out_put;
@@ -5912,7 +5913,7 @@ static void perf_mmap_close(struct vm_area_struct *vma)
        mutex_unlock(&event->mmap_mutex);
 
        /* If there's still other mmap()s of this buffer, we're done. */
-       if (atomic_read(&rb->mmap_count))
+       if (!detach_rest)
                goto out_put;
 
        /*
#8 lore raw

PATCH v2 1/1 perf: Fix race in perf_mmap_close function

There's a possible race in perf_mmap_close when checking ring buffer's mmap_count refcount value. The problem is that the mmap_count check is not atomic because we call atomic_dec and atomic_read separately.

  perf_mmap_close:
  ...
   atomic_dec(&rb->mmap_count);
   ...
   if (atomic_read(&rb->mmap_count))
      goto out_put;

<ring buffer detach>
free_uid

out_put:
ring_buffer_put(rb); /* could be last */

The race can happen when we have two (or more) events sharing same ring buffer and they go through atomic_dec and then they both see 0 as refcount value later in atomic_read. Then both will go on and execute code which is meant to be run just once.

The code that detaches ring buffer is probably fine to be executed more than once, but the problem is in calling free_uid, which will later on demonstrate in related crashes and refcount warnings, like:

refcount_t: addition on 0; use-after-free.
...

  RIP: 0010:refcount_warn_saturate+0x6d/0xf
  ...
  Call Trace:
  prepare_creds+0x190/0x1e0
  copy_creds+0x35/0x172
  copy_process+0x471/0x1a80
  _do_fork+0x83/0x3a0
  __do_sys_wait4+0x83/0x90
  __do_sys_clone+0x85/0xa0
  do_syscall_64+0x5b/0x1e0
  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Using atomic decrease and check instead of separated calls. This fixes CVE-2020-14351.

---
 kernel/events/core.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
kernel/events/core.c +4 -3
index 7ed5248f0445..8ab2400aef55 100644
@@ -5868,11 +5868,11 @@ static void perf_pmu_output_stop(struct perf_event *event);
 static void perf_mmap_close(struct vm_area_struct *vma)
 {
 	struct perf_event *event = vma->vm_file->private_data;
-
 	struct perf_buffer *rb = ring_buffer_get(event);
 	struct user_struct *mmap_user = rb->mmap_user;
 	int mmap_locked = rb->mmap_locked;
 	unsigned long size = perf_data_size(rb);
+	bool detach_rest = false;
 
 	if (event->pmu->event_unmapped)
 		event->pmu->event_unmapped(event, vma->vm_mm);
@@ -5903,7 +5903,8 @@ static void perf_mmap_close(struct vm_area_struct *vma)
 		mutex_unlock(&event->mmap_mutex);
 	}
 
-	atomic_dec(&rb->mmap_count);
+	if (atomic_dec_and_test(&rb->mmap_count))
+		detach_rest = true;
 
 	if (!atomic_dec_and_mutex_lock(&event->mmap_count, &event->mmap_mutex))
 		goto out_put;
@@ -5912,7 +5913,7 @@ static void perf_mmap_close(struct vm_area_struct *vma)
 	mutex_unlock(&event->mmap_mutex);
 
 	/* If there's still other mmap()s of this buffer, we're done. */
-	if (atomic_read(&rb->mmap_count))
+	if (!detach_rest)
 		goto out_put;
 
 	/*
Signature · 1 line
2.26.2
#9 lore raw
Quoted 40 lines · Jiri Olsa

There's a possible race in perf_mmap_close when checking ring buffer's mmap_count refcount value. The problem is that the mmap_count check is not atomic because we call atomic_dec and atomic_read separately.

  perf_mmap_close:
  ...
   atomic_dec(&rb->mmap_count);
   ...
   if (atomic_read(&rb->mmap_count))
      goto out_put;

<ring buffer detach>
free_uid

out_put:
ring_buffer_put(rb); /* could be last */

The race can happen when we have two (or more) events sharing same ring buffer and they go through atomic_dec and then they both see 0 as refcount value later in atomic_read. Then both will go on and execute code which is meant to be run just once.

The code that detaches ring buffer is probably fine to be executed more than once, but the problem is in calling free_uid, which will later on demonstrate in related crashes and refcount warnings, like:

refcount_t: addition on 0; use-after-free.
...

  RIP: 0010:refcount_warn_saturate+0x6d/0xf
  ...
  Call Trace:
  prepare_creds+0x190/0x1e0
  copy_creds+0x35/0x172
  copy_process+0x471/0x1a80
  _do_fork+0x83/0x3a0
  __do_sys_wait4+0x83/0x90
  __do_sys_clone+0x85/0xa0
  do_syscall_64+0x5b/0x1e0
  entry_SYSCALL_64_after_hwframe+0x44/0xa9
Quoted 2 lines

Using atomic decrease and check instead of separated calls. This fixes CVE-2020-14351.

I'm tempted to remove that line; I can never seem to find anything useful in a CVE.

Fixes ?
Quoted 3 lines
#10 lore raw
Quoted 41 lines · On Wed, Sep 16, 2020 at 03:54:02PM +0200, [email protected] wrote:
Quoted 40 lines · Jiri Olsa

There's a possible race in perf_mmap_close when checking ring buffer's mmap_count refcount value. The problem is that the mmap_count check is not atomic because we call atomic_dec and atomic_read separately.

  perf_mmap_close:
  ...
   atomic_dec(&rb->mmap_count);
   ...
   if (atomic_read(&rb->mmap_count))
      goto out_put;

<ring buffer detach>
free_uid

out_put:
ring_buffer_put(rb); /* could be last */

The race can happen when we have two (or more) events sharing same ring buffer and they go through atomic_dec and then they both see 0 as refcount value later in atomic_read. Then both will go on and execute code which is meant to be run just once.

The code that detaches ring buffer is probably fine to be executed more than once, but the problem is in calling free_uid, which will later on demonstrate in related crashes and refcount warnings, like:

refcount_t: addition on 0; use-after-free.
...

  RIP: 0010:refcount_warn_saturate+0x6d/0xf
  ...
  Call Trace:
  prepare_creds+0x190/0x1e0
  copy_creds+0x35/0x172
  copy_process+0x471/0x1a80
  _do_fork+0x83/0x3a0
  __do_sys_wait4+0x83/0x90
  __do_sys_clone+0x85/0xa0
  do_syscall_64+0x5b/0x1e0
  entry_SYSCALL_64_after_hwframe+0x44/0xa9
Quoted 5 lines
Quoted 2 lines

Using atomic decrease and check instead of separated calls. This fixes CVE-2020-14351.

I'm tempted to remove that line; I can never seem to find anything useful in a CVE.

I was asked by security guys to add this, Wade?

Quoted 2 lines
Fixes ?

right, sry..

Fixes 9bb5d40cd93c ("perf: Fix mmap() accounting hole");

thanks,
jirka

Quoted 5 lines
Quoted 4 lines
#11 lore raw
In reply to Jiri Olsa
Quoted 14 lines · Jiri Olsa

There's a possible race in perf_mmap_close when checking ring buffer's mmap_count refcount value. The problem is that the mmap_count check is not atomic because we call atomic_dec and atomic_read separately.

  perf_mmap_close:
  ...
   atomic_dec(&rb->mmap_count);
   ...
   if (atomic_read(&rb->mmap_count))
      goto out_put;

<ring buffer detach>
free_uid

Quoted 7 lines

out_put:
ring_buffer_put(rb); /* could be last */

The race can happen when we have two (or more) events sharing same ring buffer and they go through atomic_dec and then they both see 0 as refcount value later in atomic_read. Then both will go on and execute code which is meant to be run just once.

The trival case should be protected by mmap_sem, we call vm_ops->close() with mmap_sem held for writing IIRC. But yes, I think it's possible to construct a failure here.

merge bot Applied
#12 lore raw

[tip: perf/core] perf/core: Fix race in the perf_mmap_close() function

In reply to Jiri Olsa
The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     f91072ed1b7283b13ca57fcfbece5a3b92726143
Gitweb:        https://git.kernel.org/tip/f91072ed1b7283b13ca57fcfbece5a3b92726143
Author:        Jiri Olsa <[email protected]>
AuthorDate:    Wed, 16 Sep 2020 13:53:11 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Mon, 12 Oct 2020 13:24:26 +02:00

perf/core: Fix race in the perf_mmap_close() function

There's a possible race in perf_mmap_close() when checking ring buffer's mmap_count refcount value. The problem is that the mmap_count check is not atomic because we call atomic_dec() and atomic_read() separately.

  perf_mmap_close:
  ...
   atomic_dec(&rb->mmap_count);
   ...
   if (atomic_read(&rb->mmap_count))
      goto out_put;

<ring buffer detach>
free_uid

out_put:
ring_buffer_put(rb); /* could be last */

The race can happen when we have two (or more) events sharing same ring buffer and they go through atomic_dec() and then they both see 0 as refcount value later in atomic_read(). Then both will go on and execute code which is meant to be run just once.

The code that detaches ring buffer is probably fine to be executed more than once, but the problem is in calling free_uid(), which will later on demonstrate in related crashes and refcount warnings, like:

refcount_t: addition on 0; use-after-free.
...

  RIP: 0010:refcount_warn_saturate+0x6d/0xf
  ...
  Call Trace:
  prepare_creds+0x190/0x1e0
  copy_creds+0x35/0x172
  copy_process+0x471/0x1a80
  _do_fork+0x83/0x3a0
  __do_sys_wait4+0x83/0x90
  __do_sys_clone+0x85/0xa0
  do_syscall_64+0x5b/0x1e0
  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Using atomic decrease and check instead of separated calls.

---
 kernel/events/core.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
kernel/events/core.c +4 -3
index 45edb85..fb662eb 100644
@@ -5895,11 +5895,11 @@ static void perf_pmu_output_stop(struct perf_event *event);
 static void perf_mmap_close(struct vm_area_struct *vma)
 {
 	struct perf_event *event = vma->vm_file->private_data;
-
 	struct perf_buffer *rb = ring_buffer_get(event);
 	struct user_struct *mmap_user = rb->mmap_user;
 	int mmap_locked = rb->mmap_locked;
 	unsigned long size = perf_data_size(rb);
+	bool detach_rest = false;
 
 	if (event->pmu->event_unmapped)
 		event->pmu->event_unmapped(event, vma->vm_mm);
@@ -5930,7 +5930,8 @@ static void perf_mmap_close(struct vm_area_struct *vma)
 		mutex_unlock(&event->mmap_mutex);
 	}
 
-	atomic_dec(&rb->mmap_count);
+	if (atomic_dec_and_test(&rb->mmap_count))
+		detach_rest = true;
 
 	if (!atomic_dec_and_mutex_lock(&event->mmap_count, &event->mmap_mutex))
 		goto out_put;
@@ -5939,7 +5940,7 @@ static void perf_mmap_close(struct vm_area_struct *vma)
 	mutex_unlock(&event->mmap_mutex);
 
 	/* If there's still other mmap()s of this buffer, we're done. */
-	if (atomic_read(&rb->mmap_count))
+	if (!detach_rest)
 		goto out_put;
 
 	/*