Skip to contentSkip to conversation

kcov: don't lose track of remote references during softirqs

PATCH Superseded Aleksandr Nogikh 4 participants 7 messages Jun 11, 2024 – Jun 14, 2024 KCOV

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kcov: don't lose track of remote references during softirqs

In kcov_remote_start()/kcov_remote_stop(), we swap the previous KCOV metadata of the current task into a per-CPU variable. However, the kcov_mode_enabled(mode) check is not sufficient in the case of remote KCOV coverage: current->kcov_mode always remains KCOV_MODE_DISABLED for remote KCOV objects.

If the original task that has invoked the KCOV_REMOTE_ENABLE ioctl happens to get interrupted and kcov_remote_start() is called, it ultimately leads to kcov_remote_stop() NOT restoring the original KCOV reference. So when the task exits, all registered remote KCOV handles remain active forever.

Fix it by introducing a special kcov_mode that is assigned to the task that owns a KCOV remote object. It makes kcov_mode_enabled() return true and yet does not trigger coverage collection in __sanitizer_cov_trace_pc() and write_comp_data().

Fixes 5ff3b30ab57d ("kcov: collect coverage from interrupts")
---
 include/linux/kcov.h | 2 ++
 kernel/kcov.c        | 1 +
 2 files changed, 3 insertions(+)
include/linux/kcov.h +2 -0
index b851ba415e03..3b479a3d235a 100644
@@ -21,6 +21,8 @@ enum kcov_mode {
 	KCOV_MODE_TRACE_PC = 2,
 	/* Collecting comparison operands mode. */
 	KCOV_MODE_TRACE_CMP = 3,
+	/* The process owns a KCOV remote reference. */
+	KCOV_MODE_REMOTE = 4,
 };
 
 #define KCOV_IN_CTXSW	(1 << 30)
kernel/kcov.c +1 -0
index c3124f6d5536..5371d3f7b5c3 100644
@@ -632,6 +632,7 @@ static int kcov_ioctl_locked(struct kcov *kcov, unsigned int cmd,
 			return -EINVAL;
 		kcov->mode = mode;
 		t->kcov = kcov;
+		WRITE_ONCE(t->kcov_mode, KCOV_MODE_REMOTE);
 		kcov->t = t;
 		kcov->remote = true;
 		kcov->remote_size = remote_arg->area_size;
Signature · 1 line
2.45.2.505.gda0bf45e8d-goog
reviewer Reviewed-by on v1 1/1
#1 lore raw
Quoted 13 lines · Aleksandr Nogikh

In kcov_remote_start()/kcov_remote_stop(), we swap the previous KCOV metadata of the current task into a per-CPU variable. However, the kcov_mode_enabled(mode) check is not sufficient in the case of remote KCOV coverage: current->kcov_mode always remains KCOV_MODE_DISABLED for remote KCOV objects.

If the original task that has invoked the KCOV_REMOTE_ENABLE ioctl happens to get interrupted and kcov_remote_start() is called, it ultimately leads to kcov_remote_stop() NOT restoring the original KCOV reference. So when the task exits, all registered remote KCOV handles remain active forever.

Quoted 7 lines

Fix it by introducing a special kcov_mode that is assigned to the task that owns a KCOV remote object. It makes kcov_mode_enabled() return true and yet does not trigger coverage collection in __sanitizer_cov_trace_pc() and write_comp_data().

Fixes 5ff3b30ab57d ("kcov: collect coverage from interrupts")
Quoted 33 lines
---
 include/linux/kcov.h | 2 ++
 kernel/kcov.c        | 1 +
 2 files changed, 3 insertions(+)
include/linux/kcov.h +2 -0
index b851ba415e03..3b479a3d235a 100644
@@ -21,6 +21,8 @@ enum kcov_mode {
        KCOV_MODE_TRACE_PC = 2,
        /* Collecting comparison operands mode. */
        KCOV_MODE_TRACE_CMP = 3,
+       /* The process owns a KCOV remote reference. */
+       KCOV_MODE_REMOTE = 4,
 };
 
 #define KCOV_IN_CTXSW  (1 << 30)
kernel/kcov.c +1 -1
index c3124f6d5536..5371d3f7b5c3 100644
@@ -632,6 +632,7 @@ static int kcov_ioctl_locked(struct kcov *kcov, unsigned int cmd,
                        return -EINVAL;
                kcov->mode = mode;
                t->kcov = kcov;
+               WRITE_ONCE(t->kcov_mode, KCOV_MODE_REMOTE);
                kcov->t = t;
                kcov->remote = true;
                kcov->remote_size = remote_arg->area_size;
--

2.45.2.505.gda0bf45e8d-goog

#2 lore raw
In reply to Aleksandr Nogikh
Quoted 12 lines · On Tue, 11 Jun 2024 15:32:29 +0200 Aleksandr Nogikh <[email protected]> wrote:

In kcov_remote_start()/kcov_remote_stop(), we swap the previous KCOV metadata of the current task into a per-CPU variable. However, the kcov_mode_enabled(mode) check is not sufficient in the case of remote KCOV coverage: current->kcov_mode always remains KCOV_MODE_DISABLED for remote KCOV objects.

If the original task that has invoked the KCOV_REMOTE_ENABLE ioctl happens to get interrupted and kcov_remote_start() is called, it ultimately leads to kcov_remote_stop() NOT restoring the original KCOV reference. So when the task exits, all registered remote KCOV handles remain active forever.

Quoted 4 lines

Fix it by introducing a special kcov_mode that is assigned to the task that owns a KCOV remote object. It makes kcov_mode_enabled() return true and yet does not trigger coverage collection in __sanitizer_cov_trace_pc() and write_comp_data().

What are the userspace visible effects of this bug? I *think* it's just an efficiency thing, but how significant? In other words, should we backport this fix?

#3 lore raw
Quoted 15 lines · Andrew Morton
Quoted 12 lines · On Tue, 11 Jun 2024 15:32:29 +0200 Aleksandr Nogikh <[email protected]> wrote:

In kcov_remote_start()/kcov_remote_stop(), we swap the previous KCOV metadata of the current task into a per-CPU variable. However, the kcov_mode_enabled(mode) check is not sufficient in the case of remote KCOV coverage: current->kcov_mode always remains KCOV_MODE_DISABLED for remote KCOV objects.

If the original task that has invoked the KCOV_REMOTE_ENABLE ioctl happens to get interrupted and kcov_remote_start() is called, it ultimately leads to kcov_remote_stop() NOT restoring the original KCOV reference. So when the task exits, all registered remote KCOV handles remain active forever.

Quoted 9 lines
Quoted 4 lines

Fix it by introducing a special kcov_mode that is assigned to the task that owns a KCOV remote object. It makes kcov_mode_enabled() return true and yet does not trigger coverage collection in __sanitizer_cov_trace_pc() and write_comp_data().

What are the userspace visible effects of this bug? I *think* it's just an efficiency thing, but how significant? In other words, should we backport this fix?

The most uncomfortable effect (at least for syzkaller) is that the bug prevents the reuse of the same /sys/kernel/debug/kcov descriptor. If we obtain it in the parent process and then e.g. drop some capabilities and continuously fork to execute individual programs, at some point current->kcov of the forked process is lost, kcov_task_exit() takes no action, and all KCOV_REMOTE_ENABLE ioctls calls from subsequent forks fail.

And, yes, the efficiency is also affected if we keep on losing remote kcov objects.
a) kcov_remote_map keeps on growing forever.
b) (If I'm not mistaken), we're also not freeing the memory referenced by kcov->area.

I think it would be nice to backport the fix to the stable trees.

Signature · 1 line
Aleksandr
reviewer Reviewed-by Tested-by on v1 1/1
#4 lore raw
In reply to Aleksandr Nogikh
Quoted 51 lines · Aleksandr Nogikh

In kcov_remote_start()/kcov_remote_stop(), we swap the previous KCOV metadata of the current task into a per-CPU variable. However, the kcov_mode_enabled(mode) check is not sufficient in the case of remote KCOV coverage: current->kcov_mode always remains KCOV_MODE_DISABLED for remote KCOV objects.

If the original task that has invoked the KCOV_REMOTE_ENABLE ioctl happens to get interrupted and kcov_remote_start() is called, it ultimately leads to kcov_remote_stop() NOT restoring the original KCOV reference. So when the task exits, all registered remote KCOV handles remain active forever.

Fix it by introducing a special kcov_mode that is assigned to the task that owns a KCOV remote object. It makes kcov_mode_enabled() return true and yet does not trigger coverage collection in __sanitizer_cov_trace_pc() and write_comp_data().

Fixes 5ff3b30ab57d ("kcov: collect coverage from interrupts")
---
 include/linux/kcov.h | 2 ++
 kernel/kcov.c        | 1 +
 2 files changed, 3 insertions(+)
include/linux/kcov.h +2 -0
index b851ba415e03..3b479a3d235a 100644
@@ -21,6 +21,8 @@ enum kcov_mode {
        KCOV_MODE_TRACE_PC = 2,
        /* Collecting comparison operands mode. */
        KCOV_MODE_TRACE_CMP = 3,
+       /* The process owns a KCOV remote reference. */
+       KCOV_MODE_REMOTE = 4,
 };
 
 #define KCOV_IN_CTXSW  (1 << 30)
kernel/kcov.c +1 -1
index c3124f6d5536..5371d3f7b5c3 100644
@@ -632,6 +632,7 @@ static int kcov_ioctl_locked(struct kcov *kcov, unsigned int cmd,
                        return -EINVAL;
                kcov->mode = mode;
                t->kcov = kcov;
+               WRITE_ONCE(t->kcov_mode, KCOV_MODE_REMOTE);
                kcov->t = t;
                kcov->remote = true;
                kcov->remote_size = remote_arg->area_size;
--
Quoted 2 lines

2.45.2.505.gda0bf45e8d-goog

Thank you for fixing this!

#5 lore raw
In reply to Aleksandr Nogikh
Quoted 43 lines · Aleksandr Nogikh

In kcov_remote_start()/kcov_remote_stop(), we swap the previous KCOV metadata of the current task into a per-CPU variable. However, the kcov_mode_enabled(mode) check is not sufficient in the case of remote KCOV coverage: current->kcov_mode always remains KCOV_MODE_DISABLED for remote KCOV objects.

If the original task that has invoked the KCOV_REMOTE_ENABLE ioctl happens to get interrupted and kcov_remote_start() is called, it ultimately leads to kcov_remote_stop() NOT restoring the original KCOV reference. So when the task exits, all registered remote KCOV handles remain active forever.

Fix it by introducing a special kcov_mode that is assigned to the task that owns a KCOV remote object. It makes kcov_mode_enabled() return true and yet does not trigger coverage collection in __sanitizer_cov_trace_pc() and write_comp_data().

Fixes 5ff3b30ab57d ("kcov: collect coverage from interrupts")
---
 include/linux/kcov.h | 2 ++
 kernel/kcov.c        | 1 +
 2 files changed, 3 insertions(+)
include/linux/kcov.h +2 -0
index b851ba415e03..3b479a3d235a 100644
@@ -21,6 +21,8 @@ enum kcov_mode {
        KCOV_MODE_TRACE_PC = 2,
        /* Collecting comparison operands mode. */
        KCOV_MODE_TRACE_CMP = 3,
+       /* The process owns a KCOV remote reference. */
+       KCOV_MODE_REMOTE = 4,
 };
 
 #define KCOV_IN_CTXSW  (1 << 30)
kernel/kcov.c +0 -0
index c3124f6d5536..5371d3f7b5c3 100644

kernel/kcov.c static int kcov_ioctl_locked(struct kcov *kcov, unsigned int cmd, · line 635

Quoted 4 lines
kernel/kcov.c +1 -0
@@ -632,6 +632,7 @@ static int kcov_ioctl_locked(struct kcov *kcov, unsigned int cmd,
                        return -EINVAL;
                kcov->mode = mode;
                t->kcov = kcov;
+               WRITE_ONCE(t->kcov_mode, KCOV_MODE_REMOTE);

Looking at this again, I don't think we need this WRITE_ONCE here, as we have interrupts disabled. But if we do, perhaps it makes sense to add a comment explaining why.

Quoted 6 lines
kernel/kcov.c +0 -1
                kcov->t = t;
                kcov->remote = true;
                kcov->remote_size = remote_arg->area_size;
--

2.45.2.505.gda0bf45e8d-goog

#6 lore raw
Quoted 39 lines · Andrey Konovalov
Quoted 37 lines · Aleksandr Nogikh

In kcov_remote_start()/kcov_remote_stop(), we swap the previous KCOV metadata of the current task into a per-CPU variable. However, the kcov_mode_enabled(mode) check is not sufficient in the case of remote KCOV coverage: current->kcov_mode always remains KCOV_MODE_DISABLED for remote KCOV objects.

If the original task that has invoked the KCOV_REMOTE_ENABLE ioctl happens to get interrupted and kcov_remote_start() is called, it ultimately leads to kcov_remote_stop() NOT restoring the original KCOV reference. So when the task exits, all registered remote KCOV handles remain active forever.

Fix it by introducing a special kcov_mode that is assigned to the task that owns a KCOV remote object. It makes kcov_mode_enabled() return true and yet does not trigger coverage collection in __sanitizer_cov_trace_pc() and write_comp_data().

Fixes 5ff3b30ab57d ("kcov: collect coverage from interrupts")
---
 include/linux/kcov.h | 2 ++
 kernel/kcov.c        | 1 +
 2 files changed, 3 insertions(+)
include/linux/kcov.h +2 -0
index b851ba415e03..3b479a3d235a 100644
@@ -21,6 +21,8 @@ enum kcov_mode {
        KCOV_MODE_TRACE_PC = 2,
        /* Collecting comparison operands mode. */
        KCOV_MODE_TRACE_CMP = 3,
+       /* The process owns a KCOV remote reference. */
+       KCOV_MODE_REMOTE = 4,
 };
 
Quoted 14 lines

kernel/kcov.c static int kcov_ioctl_locked(struct kcov *kcov, unsigned int cmd, · line 635

Quoted 10 lines
include/linux/kcov.h +0 -0
 #define KCOV_IN_CTXSW  (1 << 30)
kernel/kcov.c +1 -0
index c3124f6d5536..5371d3f7b5c3 100644
@@ -632,6 +632,7 @@ static int kcov_ioctl_locked(struct kcov *kcov, unsigned int cmd,
                        return -EINVAL;
                kcov->mode = mode;
                t->kcov = kcov;
+               WRITE_ONCE(t->kcov_mode, KCOV_MODE_REMOTE);

Looking at this again, I don't think we need this WRITE_ONCE here, as we have interrupts disabled. But if we do, perhaps it makes sense to add a comment explaining why.

Thank you!
I've sent a v2:
https://lore.kernel.org/all/[email protected]/

Quoted 8 lines
Quoted 7 lines
kernel/kcov.c +0 -1
                kcov->t = t;
                kcov->remote = true;
                kcov->remote_size = remote_arg->area_size;
--

2.45.2.505.gda0bf45e8d-goog