Skip to contentSkip to conversation

/sys/kernel/tracing/events permissions

DISCUSSION Milian Wolff 3 participants 4 messages Nov 22, 2021

lore
Share & tools
lore
#0 lore raw

Hello,

a recent kernel update on my arch system has lead to breakage with regard to accessing trace events from perf. For example:

```
$ perf record -e sched:sched_switch ls 
event syntax error: 'sched:sched_switch'
                     \___ can't access trace events

Error:  No permissions to read /sys/kernel/tracing/events/sched/sched_switch
Hint:   Try 'sudo mount -o remount,mode=755 /sys/kernel/tracing/'

Run 'perf list' for a list of valid events

 Usage: perf record [<options>] [<command>]
    or: perf record [<options>] -- <command> [<options>]

    -e, --event <event>   event selector. use 'perf list' to list available 
events
```

When I then try to run the remount hint from the message above, which used to work just fine, then the error persists, because:

```
$ mount | grep tracing
tracefs on /sys/kernel/tracing type tracefs 
(rw,nosuid,nodev,noexec,relatime,mode=755)
tracefs on /sys/kernel/debug/tracing type tracefs 
(rw,nosuid,nodev,noexec,relatime,mode=755)
```

and:

```
$ ls -ld /sys/kernel/tracing/
drwxr-xr-x root root 0 B Mon Nov 22 08:39:15 2021 /sys/kernel/tracing/
$ ls -ld /sys/kernel/tracing/events/
drwxr-x--- root root 0 B Mon Nov 22 08:39:26 2021 /sys/kernel/tracing/events/
```

How should this be handled? Is it now required to manually chmod the events folder, or is there another means to allow non-root to record kernel trace events?

Thanks

Signature · 4 lines
Milian Wolff | [email protected] | Senior Software Engineer
KDAB (Deutschland) GmbH, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt, C++ and OpenGL Experts
#1 lore raw

Em Mon, Nov 22, 2021 at 11:55:11AM +0100, Milian Wolff escreveu:

Quoted 4 lines

Hello,

a recent kernel update on my arch system has lead to breakage with regard to accessing trace events from perf. For example:

I saw some recent changes but haven't yet tried to reflect it into the perf tools:

commit 79ef0c00142519bc34e1341447f3797436cc48bf
Merge: d54f486035fd89f1 feea69ec121f0670
Author: Linus Torvalds <[email protected]>
Date: Mon Nov 1 20:05:19 2021 -0700

Merge tag 'trace-v5.16' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace

Pull tracing updates from Steven Rostedt:
<SNIP>
- Have tracefs allow owner and group permissions by default (only denying others). There's been pressure to allow non root to tracefs in a controlled fashion, and using groups is probably the safest.
<SNIP>

Quoted 37 lines
```
$ perf record -e sched:sched_switch ls 
event syntax error: 'sched:sched_switch'
                     \___ can't access trace events

Error:  No permissions to read /sys/kernel/tracing/events/sched/sched_switch
Hint:   Try 'sudo mount -o remount,mode=755 /sys/kernel/tracing/'

Run 'perf list' for a list of valid events

 Usage: perf record [<options>] [<command>]
    or: perf record [<options>] -- <command> [<options>]

    -e, --event <event>   event selector. use 'perf list' to list available 
events
```

When I then try to run the remount hint from the message above, which used to work just fine, then the error persists, because:

```
$ mount | grep tracing
tracefs on /sys/kernel/tracing type tracefs 
(rw,nosuid,nodev,noexec,relatime,mode=755)
tracefs on /sys/kernel/debug/tracing type tracefs 
(rw,nosuid,nodev,noexec,relatime,mode=755)
```

and:

```
$ ls -ld /sys/kernel/tracing/
drwxr-xr-x root root 0 B Mon Nov 22 08:39:15 2021 /sys/kernel/tracing/
$ ls -ld /sys/kernel/tracing/events/
drwxr-x--- root root 0 B Mon Nov 22 08:39:26 2021 /sys/kernel/tracing/events/
```
Quoted 3 lines

How should this be handled? Is it now required to manually chmod the events folder, or is there another means to allow non-root to record kernel trace events?

Rostedt, I looked for documentation for this on Documentation/trace/ but couldn't find, so its just a matter of creating a 'tracing' group and adding users to this group that then can use tracefs files?

- Arnaldo

#2 lore raw
Quoted 6 lines · On Mon, 22 Nov 2021 12:02:37 -0300 Arnaldo Carvalho de Melo <[email protected]>…

Em Mon, Nov 22, 2021 at 11:55:11AM +0100, Milian Wolff escreveu:

Quoted 4 lines

Hello,

a recent kernel update on my arch system has lead to breakage with regard to accessing trace events from perf. For example:

Interesting. The change should have made the access to tracefs less restrictive, not more. Ah, I think it's the default permissions that caused this.

Quoted 8 lines

I saw some recent changes but haven't yet tried to reflect it into the perf tools:

commit 79ef0c00142519bc34e1341447f3797436cc48bf
Merge: d54f486035fd89f1 feea69ec121f0670
Author: Linus Torvalds <[email protected]>
Date: Mon Nov 1 20:05:19 2021 -0700

Quoted 17 lines

Merge tag 'trace-v5.16' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace

Pull tracing updates from Steven Rostedt:
<SNIP>
- Have tracefs allow owner and group permissions by default (only denying others). There's been pressure to allow non root to tracefs in a controlled fashion, and using groups is probably the safest.
<SNIP>

Quoted 7 lines

```

$ perf record -e sched:sched_switch ls 
event syntax error: 'sched:sched_switch'
                     \___ can't access trace events
Error:  No permissions to read /sys/kernel/tracing/events/sched/sched_switch
Hint:   Try 'sudo mount -o remount,mode=755 /sys/kernel/tracing/'

It's not the one commit, but the a clean up to not allow other access by default.

To get the old access you need to add:

sudo chmod -R o+r /sys/kernel/tracing
sudo find /sys/kernel/tracing -type d -exec chmod o+x {} \;

Quoted 8 lines
Quoted 3 lines

How should this be handled? Is it now required to manually chmod the events folder, or is there another means to allow non-root to record kernel trace events?

Rostedt, I looked for documentation for this on Documentation/trace/ but couldn't find, so its just a matter of creating a 'tracing' group and adding users to this group that then can use tracefs files?

I should add that to the documentation.

I believe the commit was the change to the default permissions (for group) and not for other.

But to add a group, you can simply add:

sudo addgroup tracing
sudo chgrp -R /sys/kernel/tracing

And if you only do the above (not the o+* commands explained earlier), then only those in the tracing group have access to tracefs.

-- Steve

#3 lore raw

Em Mon, Nov 22, 2021 at 12:01:55PM -0500, Steven Rostedt escreveu:

Quoted 53 lines · On Mon, 22 Nov 2021 12:02:37 -0300 Arnaldo Carvalho de Melo <[email protected]>…
Quoted 4 lines

Em Mon, Nov 22, 2021 at 11:55:11AM +0100, Milian Wolff escreveu:

Quoted 2 lines

a recent kernel update on my arch system has lead to breakage with regard to accessing trace events from perf. For example:

Interesting. The change should have made the access to tracefs less restrictive, not more. Ah, I think it's the default permissions that caused this.

Quoted 8 lines

I saw some recent changes but haven't yet tried to reflect it into the perf tools:

commit 79ef0c00142519bc34e1341447f3797436cc48bf
Merge: d54f486035fd89f1 feea69ec121f0670
Author: Linus Torvalds <[email protected]>
Date: Mon Nov 1 20:05:19 2021 -0700

Quoted 16 lines

Merge tag 'trace-v5.16' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace

Pull tracing updates from Steven Rostedt:
<SNIP>
- Have tracefs allow owner and group permissions by default (only denying others). There's been pressure to allow non root to tracefs in a controlled fashion, and using groups is probably the safest.
<SNIP>

Quoted 7 lines

```

$ perf record -e sched:sched_switch ls 
event syntax error: 'sched:sched_switch'
                     \___ can't access trace events
Error:  No permissions to read /sys/kernel/tracing/events/sched/sched_switch
Hint:   Try 'sudo mount -o remount,mode=755 /sys/kernel/tracing/'

It's not the one commit, but the a clean up to not allow other access by default.

To get the old access you need to add:

sudo chmod -R o+r /sys/kernel/tracing
sudo find /sys/kernel/tracing -type d -exec chmod o+x {} \;

Quoted 7 lines
Quoted 3 lines

How should this be handled? Is it now required to manually chmod the events folder, or is there another means to allow non-root to record kernel trace events?

Rostedt, I looked for documentation for this on Documentation/trace/ but couldn't find, so its just a matter of creating a 'tracing' group and adding users to this group that then can use tracefs files?

I should add that to the documentation.

Quoted 7 lines

I believe the commit was the change to the default permissions (for group) and not for other.

But to add a group, you can simply add:

sudo addgroup tracing
sudo chgrp -R /sys/kernel/tracing

sudo chgrp -R tracing /sys/kernel/tracing

Quoted 2 lines

And if you only do the above (not the o+* commands explained earlier), then only those in the tracing group have access to tracefs.

Ok, I'll try to autodetect this new behaviour and provide a hint for the recommended behaviour, to create a group and add users to it.

- Arnaldo