Skip to contentSkip to conversation

lttng-modules system call tracing filtering

RFC Julien Desfossez 3 participants 3 messages Jul 20, 2014 – Jul 21, 2014 3 messages not in the archive

lore
Share & tools
lore
#1 lore raw
In reply to a message not in the archive
Quoted 11 lines · Mathieu Desnoyers

Hi!

I just create a dev branch with system call filtering within LTTng modules. Currently, only enable-event is supported at the lttng-tools level for this. With this feature, you can specify exactly which system calls you want to trace.

Here are the dev branches:

https://github.com/compudj/lttng-modules-dev branch: syscall-filtering
https://github.com/compudj/lttng-tools-dev/ branch: syscall-filtering

Great !

Just to clarify, what happens to the exit_syscall event ? Is it still emitted for every syscall exit or only for the ones we selected ?
Is it renamed ?

For the CLI usage, I think having a coma separated list would be more friendly that having to enable each syscall in a separate command, but I'm sure it is easy to have that.

Thanks !

Julien

Quoted 70 lines

To use it:

#ex 1
lttng create
lttng enable-event -k --syscall sys_open lttng enable-event -k --syscall sys_close lttng start; sleep 3; lttng stop; lttng view -> should only show sys_open and sys_close syscalls.

#ex 2
lttng create
lttng enable-event -k --syscall -a
lttng start; sleep 3; lttng stop; lttng view
-> should show all syscalls

#ex 3
lttng create
lttng enable-event -k --syscall -a
lttng enable-event -k --syscall sys_open
-> returns that the system call is already enabled. lttng start; sleep 3; lttng stop; lttng view -> should show all syscalls

#ex 4
lttng create
lttng enable-event -k --syscall sys_open
lttng enable-event -k --syscall -a
lttng start; sleep 3; lttng stop; lttng view
-> should show all syscalls

### Now about the disable-event part (not implemented in tools yet) ### This is the behavior I would like:

#ex 5 (TODO)
lttng create
lttng enable-event -k --syscall -a
lttng disable-event -k --syscall sys_open
lttng start; sleep 3; lttng stop; lttng view
-> should show all syscalls except sys_open

#ex 6 (TODO)
lttng create
lttng disable-event -k --syscall sys_open
-> should fail.

For the curious, I implement this "filtering" with a per-channel bitmap that represents which system calls to trace. We might need to double-check that I got the NR_syscalls right for each architecture, especially those with compatibility system call tables (64-bit archs having 32-bit compat syscalls). For the common case (all system calls are traced), the pointer to the array is NULL, so this is a simple pointer check, which is less expensive cache-wise than looking up within the bitmap.

As far as lttng-tools is concerned, what is a bit different is that system calls don't each get a file descriptor assigned, unlike other tracepoint events. Therefore, we interact with them at the channel level. If we can find a way to send the disable-event command directly to the channel, with the new "u.syscall.disable" flag I added to the lttng ABI, we should be able to use disable-event with syscalls. However, I'm not sure how deeply we need to modify lttng-tools for this.

Feedback is welcome,

Thanks !

Mathieu

#3 lore raw
In reply to a message not in the archive

----- Original Message -----

Quoted 16 lines

From: "Julien Desfossez" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>, "lttng-dev" <[email protected]>
Cc: "David Goulet ([email protected])" <[email protected]>
Sent: Sunday, July 20, 2014 11:54:53 AM
Subject: Re: [RFC] lttng-modules system call tracing filtering

Quoted 9 lines · Mathieu Desnoyers

Hi!

I just create a dev branch with system call filtering within LTTng modules. Currently, only enable-event is supported at the lttng-tools level for this. With this feature, you can specify exactly which system calls you want to trace.

Here are the dev branches:

Quoted 8 lines
Quoted 2 lines

https://github.com/compudj/lttng-modules-dev branch: syscall-filtering
https://github.com/compudj/lttng-tools-dev/ branch: syscall-filtering

Great !

Just to clarify, what happens to the exit_syscall event ? Is it still emitted for every syscall exit or only for the ones we selected ?

Just for the ones selected.

Quoted 1 line

Is it renamed ?

Yes, I'm planning to rename it, given that I want to extract the system call output parameter at syscall exit anyway, so we need to specialize this event.

Quoted 4 lines

For the CLI usage, I think having a coma separated list would be more friendly that having to enable each syscall in a separate command, but I'm sure it is easy to have that.

Perhaps this already works actually, I have not tried it though.

Thanks for the feedback!

Mathieu

Quoted 76 lines

Thanks !

Julien

Quoted 70 lines

To use it:

#ex 1
lttng create
lttng enable-event -k --syscall sys_open lttng enable-event -k --syscall sys_close lttng start; sleep 3; lttng stop; lttng view -> should only show sys_open and sys_close syscalls.

#ex 2
lttng create
lttng enable-event -k --syscall -a
lttng start; sleep 3; lttng stop; lttng view
-> should show all syscalls

#ex 3
lttng create
lttng enable-event -k --syscall -a
lttng enable-event -k --syscall sys_open
-> returns that the system call is already enabled. lttng start; sleep 3; lttng stop; lttng view -> should show all syscalls

#ex 4
lttng create
lttng enable-event -k --syscall sys_open
lttng enable-event -k --syscall -a
lttng start; sleep 3; lttng stop; lttng view
-> should show all syscalls

### Now about the disable-event part (not implemented in tools yet) ### This is the behavior I would like:

#ex 5 (TODO)
lttng create
lttng enable-event -k --syscall -a
lttng disable-event -k --syscall sys_open
lttng start; sleep 3; lttng stop; lttng view
-> should show all syscalls except sys_open

#ex 6 (TODO)
lttng create
lttng disable-event -k --syscall sys_open
-> should fail.

For the curious, I implement this "filtering" with a per-channel bitmap that represents which system calls to trace. We might need to double-check that I got the NR_syscalls right for each architecture, especially those with compatibility system call tables (64-bit archs having 32-bit compat syscalls). For the common case (all system calls are traced), the pointer to the array is NULL, so this is a simple pointer check, which is less expensive cache-wise than looking up within the bitmap.

As far as lttng-tools is concerned, what is a bit different is that system calls don't each get a file descriptor assigned, unlike other tracepoint events. Therefore, we interact with them at the channel level. If we can find a way to send the disable-event command directly to the channel, with the new "u.syscall.disable" flag I added to the lttng ABI, we should be able to use disable-event with syscalls. However, I'm not sure how deeply we need to modify lttng-tools for this.

Feedback is welcome,

Thanks !

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#5 lore raw
In reply to a message not in the archive
Quoted 35 lines · Mathieu Desnoyers

----- Original Message -----

Quoted 16 lines

From: "Julien Desfossez" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>, "lttng-dev" <[email protected]>
Cc: "David Goulet ([email protected])" <[email protected]>
Sent: Sunday, July 20, 2014 11:54:53 AM
Subject: Re: [RFC] lttng-modules system call tracing filtering

Quoted 9 lines · Mathieu Desnoyers

Hi!

I just create a dev branch with system call filtering within LTTng modules. Currently, only enable-event is supported at the lttng-tools level for this. With this feature, you can specify exactly which system calls you want to trace.

Here are the dev branches:

Quoted 8 lines
Quoted 2 lines

https://github.com/compudj/lttng-modules-dev branch: syscall-filtering
https://github.com/compudj/lttng-tools-dev/ branch: syscall-filtering

Great !

Just to clarify, what happens to the exit_syscall event ? Is it still emitted for every syscall exit or only for the ones we selected ?

Just for the ones selected.

Quoted 1 line

Is it renamed ?

Yes, I'm planning to rename it, given that I want to extract the system call output parameter at syscall exit anyway, so we need to specialize this event.

Quoted 1 line
Quoted 5 lines
Quoted 3 lines

For the CLI usage, I think having a coma separated list would be more friendly that having to enable each syscall in a separate command, but I'm sure it is easy to have that.

Perhaps this already works actually, I have not tried it though.

No reason this shouldn’t work out of the box :). You're inside the event striping loop so everything is fine.

Cheers!

Quoted 82 lines

Thanks for the feedback!

Mathieu

Quoted 76 lines

Thanks !

Julien

Quoted 70 lines

To use it:

#ex 1
lttng create
lttng enable-event -k --syscall sys_open lttng enable-event -k --syscall sys_close lttng start; sleep 3; lttng stop; lttng view -> should only show sys_open and sys_close syscalls.

#ex 2
lttng create
lttng enable-event -k --syscall -a
lttng start; sleep 3; lttng stop; lttng view
-> should show all syscalls

#ex 3
lttng create
lttng enable-event -k --syscall -a
lttng enable-event -k --syscall sys_open
-> returns that the system call is already enabled. lttng start; sleep 3; lttng stop; lttng view -> should show all syscalls

#ex 4
lttng create
lttng enable-event -k --syscall sys_open
lttng enable-event -k --syscall -a
lttng start; sleep 3; lttng stop; lttng view
-> should show all syscalls

### Now about the disable-event part (not implemented in tools yet) ### This is the behavior I would like:

#ex 5 (TODO)
lttng create
lttng enable-event -k --syscall -a
lttng disable-event -k --syscall sys_open
lttng start; sleep 3; lttng stop; lttng view
-> should show all syscalls except sys_open

#ex 6 (TODO)
lttng create
lttng disable-event -k --syscall sys_open
-> should fail.

For the curious, I implement this "filtering" with a per-channel bitmap that represents which system calls to trace. We might need to double-check that I got the NR_syscalls right for each architecture, especially those with compatibility system call tables (64-bit archs having 32-bit compat syscalls). For the common case (all system calls are traced), the pointer to the array is NULL, so this is a simple pointer check, which is less expensive cache-wise than looking up within the bitmap.

As far as lttng-tools is concerned, what is a bit different is that system calls don't each get a file descriptor assigned, unlike other tracepoint events. Therefore, we interact with them at the channel level. If we can find a way to send the disable-event command directly to the channel, with the new "u.syscall.disable" flag I added to the lttng ABI, we should be able to use disable-event with syscalls. However, I'm not sure how deeply we need to modify lttng-tools for this.

Feedback is welcome,

Thanks !

Mathieu