Skip to contentSkip to conversation

LTTng logger ABI

RFC PATCH Karim Yaghmour 2 participants 7 messages Feb 17, 2014 7 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

That was quick :)

Quoted 3 lines · Mathieu Desnoyers

Add a user-space ABI (new file /proc/lttng-logger) to lttng-modules which can be written into by any user on the system. The content is saved into the kernel trace stream into the "lttng_logger" kernel event.

Nice. I think this would do for the use-case we discussed (i.e. feedback Android's "ATRACE" events into lttng instead of ftrace's "trace_marker" file.)

Quoted 5 lines

The content of a single write is written into an lttng_logger event, provided the packet size is large enough to store the content. If it is not large enough, it won't be logged. However we cannot return any error to the caller, since there may be more than one trace session listening to this event.

Can we just truncate instead of dropping if the event is too large?

Signature · 3 lines
Karim Yaghmour
CEO - Opersys inc. / www.opersys.com
http://twitter.com/karimyaghmour

A message that is not in the archive (sent off-list or not yet received): [email protected]

#3 lore raw

----- Original Message -----

Quoted 9 lines

From: "Karim Yaghmour" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: [email protected]
Sent: Monday, February 17, 2014 10:09:37 AM
Subject: Re: [RFC PATCH lttng-modules] LTTng logger ABI

That was quick :)

Quoted 8 lines
Quoted 3 lines · Mathieu Desnoyers

Add a user-space ABI (new file /proc/lttng-logger) to lttng-modules which can be written into by any user on the system. The content is saved into the kernel trace stream into the "lttng_logger" kernel event.

Nice. I think this would do for the use-case we discussed (i.e. feedback Android's "ATRACE" events into lttng instead of ftrace's "trace_marker" file.)

Great!

Quoted 8 lines
Quoted 6 lines

The content of a single write is written into an lttng_logger event, provided the packet size is large enough to store the content. If it is not large enough, it won't be logged. However we cannot return any error to the caller, since there may be more than one trace session listening to this event.

Can we just truncate instead of dropping if the event is too large?

I'd be tempted to ask: is that the behavior of ftrace when receiving an event larger than a page through trace_marker ?

We could indeed truncate the event, but it would increase complexity compared to the current patch. So I'm tempted to wait a bit before doing so.

Thanks,

Mathieu

Quoted 7 lines

--
Karim Yaghmour
CEO - Opersys inc. / www.opersys.com
http://twitter.com/karimyaghmour

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

A message that is not in the archive (sent off-list or not yet received): [email protected]

#5 lore raw
Quoted 2 lines · Mathieu Desnoyers

I'd be tempted to ask: is that the behavior of ftrace when receiving an event larger than a page through trace_marker ?

I've never bothered looking it up to be quite honest.

Quoted 2 lines

We could indeed truncate the event, but it would increase complexity compared to the current patch. So I'm tempted to wait a bit before doing so.

What's the maximum length of events lttng-logger will take? We just truncate in the Android user-space library that writes the atrace events.

Signature · 3 lines
Karim Yaghmour
CEO - Opersys inc. / www.opersys.com
http://twitter.com/karimyaghmour

A message that is not in the archive (sent off-list or not yet received): [email protected]

#7 lore raw

----- Original Message -----

Quoted 7 lines

From: "Karim Yaghmour" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: [email protected]
Sent: Monday, February 17, 2014 11:13:42 AM
Subject: Re: [RFC PATCH lttng-modules] LTTng logger ABI

Quoted 5 lines
Quoted 2 lines · Mathieu Desnoyers

I'd be tempted to ask: is that the behavior of ftrace when receiving an event larger than a page through trace_marker ?

I've never bothered looking it up to be quite honest.

I'm seeing here that the event in trace_marker is limited to 1 page (it can cross a page boundary, so ftrace deals with 2 pages at most). It makes sense, since Ftrace's buffers are split into pages.

Its truncation happens here:

if (cnt > TRACE_BUF_SIZE)
cnt = TRACE_BUF_SIZE;

Quoted 7 lines
Quoted 4 lines

We could indeed truncate the event, but it would increase complexity compared to the current patch. So I'm tempted to wait a bit before doing so.

What's the maximum length of events lttng-logger will take? We just truncate in the Android user-space library that writes the atrace events.

The minimum size of sub-buffers (trace packets) LTTng can allocate is a page. However, the available space for event records is slightly less, since we have a packet header at the beginning of the packet. Since the packet header can be extended between LTTng versions, it's hard to say. The space can be slightly less if event contexts are enabled (this can be done dynamically at runtime, per-buffer).

Thoughts ?

Thanks,

Mathieu

Quoted 7 lines

--
Karim Yaghmour
CEO - Opersys inc. / www.opersys.com
http://twitter.com/karimyaghmour

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

A message that is not in the archive (sent off-list or not yet received): [email protected]

#9 lore raw
Quoted 8 lines · Mathieu Desnoyers

I'm seeing here that the event in trace_marker is limited to 1 page (it can cross a page boundary, so ftrace deals with 2 pages at most). It makes sense, since Ftrace's buffers are split into pages.

Its truncation happens here:

if (cnt > TRACE_BUF_SIZE)
cnt = TRACE_BUF_SIZE;

Good to know. So at least we know ftrace truncates by default.

Quoted 8 lines

The minimum size of sub-buffers (trace packets) LTTng can allocate is a page. However, the available space for event records is slightly less, since we have a packet header at the beginning of the packet. Since the packet header can be extended between LTTng versions, it's hard to say. The space can be slightly less if event contexts are enabled (this can be done dynamically at runtime, per-buffer).

Thoughts ?

I'd be tempted to say that anything that big should be good enough. But from the sounds of it, even doing strlen's in user-space wouldn't be good enough to ensure that the data fits in lttng buffer. I can't say that I have a good solution here. Ideally there should be some form of automatic truncation somewhere without having to go through the expense of strlens or the likes.

Signature · 3 lines
Karim Yaghmour
CEO - Opersys inc. / www.opersys.com
http://twitter.com/karimyaghmour

A message that is not in the archive (sent off-list or not yet received): [email protected]

#11 lore raw

----- Original Message -----

Quoted 29 lines

From: "Karim Yaghmour" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: [email protected]
Sent: Monday, February 17, 2014 11:26:26 AM
Subject: Re: [RFC PATCH lttng-modules] LTTng logger ABI

Quoted 9 lines · Mathieu Desnoyers

I'm seeing here that the event in trace_marker is limited to 1 page (it can cross a page boundary, so ftrace deals with 2 pages at most). It makes sense,
since Ftrace's buffers are split into pages.

Its truncation happens here:

if (cnt > TRACE_BUF_SIZE)
cnt = TRACE_BUF_SIZE;

Good to know. So at least we know ftrace truncates by default.

Quoted 9 lines

The minimum size of sub-buffers (trace packets) LTTng can allocate is a page.
However, the available space for event records is slightly less, since we have
a packet header at the beginning of the packet. Since the packet header can be extended between LTTng versions, it's hard to say. The space can be slightly less if event contexts are enabled (this can be done dynamically at runtime, per-buffer).

Quoted 8 lines
Quoted 1 line

Thoughts ?

I'd be tempted to say that anything that big should be good enough. But from the sounds of it, even doing strlen's in user-space wouldn't be good enough to ensure that the data fits in lttng buffer. I can't say that I have a good solution here. Ideally there should be some form of automatic truncation somewhere without having to go through the expense of strlens or the likes.

trace.h:#define TRACE_BUF_SIZE 1024

so actually, ftrace truncates at 1024 bytes. We could very well do the same within the lttng-logger write() op.

Thoughts ?

Thanks,

Mathieu

Quoted 7 lines

--
Karim Yaghmour
CEO - Opersys inc. / www.opersys.com
http://twitter.com/karimyaghmour

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

A message that is not in the archive (sent off-list or not yet received): [email protected]

#13 lore raw
Quoted 6 lines · Mathieu Desnoyers

trace.h:#define TRACE_BUF_SIZE 1024

so actually, ftrace truncates at 1024 bytes. We could very well do the same within the lttng-logger write() op.

Thoughts ?

Sold.

Signature · 3 lines
Karim Yaghmour
CEO - Opersys inc. / www.opersys.com
http://twitter.com/karimyaghmour