Skip to contentSkip to conversation

Emitting events from shared object constructors is (currently) not possible

DISCUSSION Mathieu Desnoyers 2 participants 6 messages Nov 22, 2012 – May 6, 2013 6 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 23 lines · * Woegerer, Paul ([email protected]) wrote:

I recently stumbled over the following lttng-ust limitation:

I tried to emit events from a shared objects constructor function:

__attribute__((constructor))
void func_constructor()

{
   tracepoint( foo, bar ); 
}

Unfortunately this doesn't work because the constructor functions of LTTng itself (__tracepoints__init and the ones from ust-tracepoint-event.h) are not yet run.

Trying to solve this problem with explicit constructor priority on my side alone doesn't work:

__attribute__((constructor (1111)))
void func_constructor()

{
   tracepoint( foo, bar ); 
}
Quoted 5 lines

That's because whatever I specify as priority is irrelevant. LTTng's constructors without priority will always come last.

After also adding priority to the lttng constructor I was able to make it work (see patch below):

Hi Paul,

I notice that you probably embed your probes along with your code within the same shared object/executable, right ?

Unfortunately, this approach does not work if the probes are placed in a different shared object than the program code. Here is an example:

a.c:

#include <stdio.h>

__attribute__((constructor(998)))
void fcta_prio1(void)

{
	fprintf(stderr, "fcta prio 998\n");
}

__attribute__((constructor))
void fcta(void)

{
	fprintf(stderr, "fcta\n");
}

__attribute__((constructor(1000)))
void fcta_prio2(void)

{
	fprintf(stderr, "fcta prio 1000\n");
}

b.c:
#include <stdio.h>

__attribute__((constructor(999)))
void fctb_prio1(void)

{
	fprintf(stderr, "fctb prio 999\n");
}

__attribute__((constructor))
void fctb(void)

{
	fprintf(stderr, "fctb\n");
}

__attribute__((constructor(1001)))
void fctb_prio2(void)

{
	fprintf(stderr, "fctb prio 1001\n");
}

main.c:

int main()

{
	return 0;
}

Makefile:
all:
gcc -fPIC -shared -Wl,-soname,liba.so -o liba.so a.c gcc -fPIC -shared -Wl,-soname,libb.so -o libb.so b.c gcc -o test main.c -L./ -la -lb

exec with:

LD_LIBRARY_PATH=. ./test

result:

fctb prio 999
fctb prio 1001
fctb
fcta prio 998
fcta prio 1000
fcta

If we build test2 all within the same executable, then the order is fine:

gcc -o test2 main.c a.c b.c
./test2
result:
fcta prio 998
fctb prio 999
fcta prio 1000
fctb prio 1001
fcta
fctb

As we can notice, the priority is effective within a shared object, but has no ordering effect across .so. And having probes in separate .so is a very typical use case.

One way we could envision allowing tracing of constructors in applications is to ensure __tracepoints__init, which always resides within the same object as the traced application code, runs with a high constructor priorita -- low value -- (for code compiled by compilers supporting this feature), and allow the application to invoke the probe constructor early. So the proposed changes would be:

1 - add a constructor priority for
__tracepoints__init/__tracepoints_destroy in tracepoint.h. Use macros to test for gcc version which allow this attribute.
2 - add some API that could let user code call
__lttng_events_init__PROVIDER_NAME explicitly.
Add an "already executed" flag to guard __lttng_events_init__PROVIDER_NAME.
This API should still allow to LD_PRELOAD the probes (no dependency on the probe symbols, possibly by using dlsym ?)
3 - make lttng_probe_register call lttng_ust_init needed.
Add an "already executed" flag to guard lttng_ust_init.

Thoughts ?

Thanks,

Mathieu

Quoted 62 lines
include/lttng/tracepoint.h +2 -2
index 8b08914..532f1d2 100644
@@ -261,7 +261,7 @@ int __tracepoint_registered
 struct tracepoint_dlopen tracepoint_dlopen
        __attribute__((weak, visibility("hidden")));
 
-static void lttng_ust_notrace __attribute__((constructor))
+static void lttng_ust_notrace __attribute__((constructor (1000)))
 __tracepoints__init(void);
 static void
 __tracepoints__init(void)
@@ -300,7 +300,7 @@ __tracepoints__init(void)
                                __start___tracepoints_ptrs);
 }
 
-static void lttng_ust_notrace __attribute__((destructor))
+static void lttng_ust_notrace __attribute__((destructor (1000)))
 __tracepoints__destroy(void);
 static void
 __tracepoints__destroy(void)
include/lttng/ust-tracepoint-event.h +2 -2
index 74d06e1..7ad3144 100644
@@ -638,7 +638,7 @@ static struct lttng_probe_desc _TP_COMBINE_TOKENS(__probe_desc___, TRACEPOINT_PR
 
 /* Reset all macros within TRACEPOINT_EVENT */
 #include <lttng/ust-tracepoint-event-reset.h>
-static void lttng_ust_notrace __attribute__((constructor))
+static void lttng_ust_notrace __attribute__((constructor (1000)))
 _TP_COMBINE_TOKENS(__lttng_events_init__, TRACEPOINT_PROVIDER)(void);
 static void
 _TP_COMBINE_TOKENS(__lttng_events_init__, TRACEPOINT_PROVIDER)(void)
@@ -649,7 +649,7 @@ _TP_COMBINE_TOKENS(__lttng_events_init__, TRACEPOINT_PROVIDER)(void)
        assert(!ret);
 }
 
-static void lttng_ust_notrace __attribute__((destructor))
+static void lttng_ust_notrace __attribute__((destructor (1000)))
 _TP_COMBINE_TOKENS(__lttng_events_exit__, TRACEPOINT_PROVIDER)(void);
 static void
 _TP_COMBINE_TOKENS(__lttng_events_exit__, TRACEPOINT_PROVIDER)(void)
Signature · 14 lines
-- 
Paul Woegerer | SW Development Engineer
http://go.mentor.com/sourceryanalyzer

Mentor Embedded(tm) | Prinz Eugen Straße 72/2/4, Vienna, 1040 Austria
Nucleus® | Linux® | Android(tm) | Services | UI | Multi-OS

Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions.
Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

A message that is not in the archive (sent off-list or not yet received): 20121122153901.GA14685@Krystal

#3 lore raw
Quoted 30 lines · Mathieu Desnoyers
Quoted 23 lines · * Woegerer, Paul ([email protected]) wrote:

I recently stumbled over the following lttng-ust limitation:

I tried to emit events from a shared objects constructor function:

__attribute__((constructor))
void func_constructor()

{
   tracepoint( foo, bar ); 
}

Unfortunately this doesn't work because the constructor functions of LTTng itself (__tracepoints__init and the ones from ust-tracepoint-event.h) are not yet run.

Trying to solve this problem with explicit constructor priority on my side alone doesn't work:

__attribute__((constructor (1111)))
void func_constructor()

{
   tracepoint( foo, bar ); 
}
Quoted 5 lines

That's because whatever I specify as priority is irrelevant. LTTng's constructors without priority will always come last.

After also adding priority to the lttng constructor I was able to make it work (see patch below):

Quoted 7 lines

Hi Paul,

I notice that you probably embed your probes along with your code within the same shared object/executable, right ?

Unfortunately, this approach does not work if the probes are placed in a different shared object than the program code. Here is an example:

Hi Mathieu,

In case the probes would be defined in different shared object from the one where they are used we wouldn't have a problem at all because the dynamic linker would invoke all the constructor functions for the probe shared object before the constructor functions of the shared object that has a dependency to the probe shared object, right ?

And even if this is not the case I could explicitly dlopen the probe shared object from within my constructor function and thus make sure that the constructor function of the probe shared object gets called before I emit an event from my constructor function.

--
Paul

Signature · 10 lines
Paul Woegerer | SW Development Engineer
http://go.mentor.com/sourceryanalyzer

Mentor Embedded(tm) | Prinz Eugen Straße 72/2/4, Vienna, 1040 Austria
Nucleus® | Linux® | Android(tm) | Services | UI | Multi-OS

Android is a trademark of Google Inc. Use of this trademark is subject
to Google Permissions.
Linux is the registered trademark of Linus Torvalds in the U.S. and
other countries.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#5 lore raw
Quoted 39 lines · * Woegerer, Paul ([email protected]) wrote:
Quoted 30 lines · Mathieu Desnoyers
Quoted 23 lines · * Woegerer, Paul ([email protected]) wrote:

I recently stumbled over the following lttng-ust limitation:

I tried to emit events from a shared objects constructor function:

__attribute__((constructor))
void func_constructor()

{
   tracepoint( foo, bar ); 
}

Unfortunately this doesn't work because the constructor functions of LTTng itself (__tracepoints__init and the ones from ust-tracepoint-event.h) are not yet run.

Trying to solve this problem with explicit constructor priority on my side alone doesn't work:

__attribute__((constructor (1111)))
void func_constructor()

{
   tracepoint( foo, bar ); 
}
Quoted 5 lines

That's because whatever I specify as priority is irrelevant. LTTng's constructors without priority will always come last.

After also adding priority to the lttng constructor I was able to make it work (see patch below):

Quoted 7 lines

Hi Paul,

I notice that you probably embed your probes along with your code within the same shared object/executable, right ?

Unfortunately, this approach does not work if the probes are placed in a different shared object than the program code. Here is an example:

Quoted 7 lines

Hi Mathieu,

In case the probes would be defined in different shared object from the one where they are used we wouldn't have a problem at all because the dynamic linker would invoke all the constructor functions for the probe shared object before the constructor functions of the shared object that has a dependency to the probe shared object, right ?

The instrumented code does not have dependency on the probe shared object, so we can ship applications separately from their probe .so. So unfortunately, the linker is not doing this for us.

Quoted 5 lines

And even if this is not the case I could explicitly dlopen the probe shared object from within my constructor function and thus make sure that the constructor function of the probe shared object gets called before I emit an event from my constructor function.

The idea would be to provide a nice API to facilitate this use-case, and document it in the instrumentation guide-lines (lttng-ust(3)).

Thanks,

Mathieu

Quoted 15 lines

--
Paul

Signature · 10 lines
Paul Woegerer | SW Development Engineer
http://go.mentor.com/sourceryanalyzer

Mentor Embedded(tm) | Prinz Eugen Straße 72/2/4, Vienna, 1040 Austria
Nucleus® | Linux® | Android(tm) | Services | UI | Multi-OS

Android is a trademark of Google Inc. Use of this trademark is subject
to Google Permissions.
Linux is the registered trademark of Linus Torvalds in the U.S. and
other countries.
Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

A message that is not in the archive (sent off-list or not yet received): 20121122164512.GA16119@Krystal

#7 lore raw
Quoted 10 lines · Mathieu Desnoyers
Quoted 5 lines · * Woegerer, Paul ([email protected]) wrote:

In case the probes would be defined in different shared object from the one where they are used we wouldn't have a problem at all because the dynamic linker would invoke all the constructor functions for the probe shared object before the constructor functions of the shared object that has a dependency to the probe shared object, right ?

The instrumented code does not have dependency on the probe shared object, so we can ship applications separately from their probe .so. So unfortunately, the linker is not doing this for us.

But still, to come back to my original request, whenever the tracepoint provider is statically linked to the tracepoint user (which is a valid lttng-ust use case, right ?) it is not possible to emit tracepoints in object constructors as long as object constructors defined in lttng/tracepoint.h and lttng/ust-tracepoint-event.h are simply using __attribute__((constructor)) instead of e.g. __attribute__((constructor (1000))). Without this change the user simply cannot make sure its own constructor gets invoked after the trace provider constructors.

Quoted 7 lines
Quoted 4 lines

And even if this is not the case I could explicitly dlopen the probe shared object from within my constructor function and thus make sure that the constructor function of the probe shared object gets called before I emit an event from my constructor function.

The idea would be to provide a nice API to facilitate this use-case, and document it in the instrumentation guide-lines (lttng-ust(3)).

Therefore we would need to reliably deduce the tracepoint provider shared library name from the header file that defines it. Since C/C++ has no package concept and gives you the freedom to name your shared library whatever you like it seems to be hard to get there.

One way to deal with it would be to extend the lttng-gen-tp tool directly build the tracepoint provider shared library instead of just outputting the generated .c .h files. This way the shared library name would be controlled by lttng-gen-tp tool and an autogenerated macro (e.g. ensure_tracepoints_init()) in the generated .h file could provide the dlopen code to explicitly load the tracepoint provider shared library if needed.

Thanks,
Paul

Signature · 10 lines
Paul Woegerer | SW Development Engineer
http://go.mentor.com/sourceryanalyzer

Mentor Embedded(tm) | Prinz Eugen Straße 72/2/4, Vienna, 1040 Austria
Nucleus® | Linux® | Android(tm) | Services | UI | Multi-OS

Android is a trademark of Google Inc. Use of this trademark is subject
to Google Permissions.
Linux is the registered trademark of Linus Torvalds in the U.S. and
other countries.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#9 lore raw
Quoted 8 lines · * Woegerer, Paul ([email protected]) wrote:
Quoted 7 lines · Mathieu Desnoyers
Quoted 5 lines · * Woegerer, Paul ([email protected]) wrote:

In case the probes would be defined in different shared object from the one where they are used we wouldn't have a problem at all because the dynamic linker would invoke all the constructor functions for the probe shared object before the constructor functions of the shared object that has a dependency to the probe shared object, right ?

Quoted 7 lines
Quoted 3 lines

The instrumented code does not have dependency on the probe shared object, so we can ship applications separately from their probe .so. So unfortunately, the linker is not doing this for us.

But still, to come back to my original request, whenever the tracepoint provider is statically linked to the tracepoint user (which is a valid lttng-ust use case, right ?)

yes,

Quoted 5 lines

it is not possible to emit tracepoints in
object constructors as long as object constructors defined in lttng/tracepoint.h and lttng/ust-tracepoint-event.h are simply using __attribute__((constructor)) instead of e.g. __attribute__((constructor (1000))).

Those can be emitted, but indeed they might not be traced.

Quoted 2 lines

Without this change the user simply cannot make sure its own constructor gets invoked after the trace provider constructors.

If we try to support tracing constructors, I'm concerned that with this change (using priority):
1 - we lose in terms of portability (not all gcc versions support the priority argument,
2 - there can be corner-cases that won't honor the priority.

We should thoroughly test and document the effect of priority in cases of:

- multiple .o linked into an executable,
- multiple .o linked into a .a, statically linked (does not honor constructors at all),
- multiple .o linked into a .so, many .so linked,

And figure out if the ordering of constructor is OK with each of those cases.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

And even if this is not the case I could explicitly dlopen the probe shared object from within my constructor function and thus make sure that the constructor function of the probe shared object gets called before I emit an event from my constructor function.

Quoted 7 lines
Quoted 2 lines

The idea would be to provide a nice API to facilitate this use-case, and document it in the instrumentation guide-lines (lttng-ust(3)).

Therefore we would need to reliably deduce the tracepoint provider shared library name from the header file that defines it. Since C/C++ has no package concept and gives you the freedom to name your shared library whatever you like it seems to be hard to get there.

indeed,

Quoted 4 lines

One way to deal with it would be to extend the lttng-gen-tp tool directly build the tracepoint provider shared library instead of just outputting the generated .c .h files.

But then we'd _require_ that people use .so for providers. They might want to just pull the probe provider into their executable.

Quoted 5 lines

This way the shared library name
would be controlled by lttng-gen-tp tool and an autogenerated macro (e.g. ensure_tracepoints_init()) in the generated .h file could provide the dlopen code to explicitly load the tracepoint provider shared library if needed.

Would this work fine with your constructor instrumentation use-case ? Is the order of constructors across .so files guaranteed in any way ?

Thanks,

Mathieu

Quoted 16 lines

Thanks,
Paul

Signature · 10 lines
Paul Woegerer | SW Development Engineer
http://go.mentor.com/sourceryanalyzer

Mentor Embedded(tm) | Prinz Eugen Straße 72/2/4, Vienna, 1040 Austria
Nucleus® | Linux® | Android(tm) | Services | UI | Multi-OS

Android is a trademark of Google Inc. Use of this trademark is subject
to Google Permissions.
Linux is the registered trademark of Linus Torvalds in the U.S. and
other countries.
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

A message that is not in the archive (sent off-list or not yet received): 20130503183500.GB7288@Krystal

#11 lore raw
Quoted 13 lines · Mathieu Desnoyers
Quoted 2 lines · * Woegerer, Paul ([email protected]) wrote:

Without this change the user simply cannot make sure its own constructor gets invoked after the trace provider constructors.

If we try to support tracing constructors, I'm concerned that with this change (using priority):
1 - we lose in terms of portability (not all gcc versions support the priority argument,
2 - there can be corner-cases that won't honor the priority.

We should thoroughly test and document the effect of priority in cases of:

Quoted 7 lines

- multiple .o linked into an executable,
- multiple .o linked into a .a, statically linked (does not honor constructors at all),
- multiple .o linked into a .so, many .so linked,

And figure out if the ordering of constructor is OK with each of those cases.

You are right. Using the priority construct might cause more problems than it solves.

A pragmatic solution to the above problem would be to provide a special documented function (as part of man:lttng-ust) that allows the user to ensure that the static constructors (withing the same "dynamic linker unit" (.so or exe)) used by lttng-ust have been executed successfully. Something like:

void ensure_lttng_ust_init_completed();

If the trace provider constructors have been invoked already the function simply returns otherwise it invokes them explicitly and makes sure that a later invocation by the dynamic linker will not have any effect.

Quoted 25 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

And even if this is not the case I could explicitly dlopen the probe shared object from within my constructor function and thus make sure that the constructor function of the probe shared object gets called before I emit an event from my constructor function.

Quoted 7 lines
Quoted 2 lines

The idea would be to provide a nice API to facilitate this use-case, and document it in the instrumentation guide-lines (lttng-ust(3)).

Therefore we would need to reliably deduce the tracepoint provider shared library name from the header file that defines it. Since C/C++ has no package concept and gives you the freedom to name your shared library whatever you like it seems to be hard to get there.

indeed,

Quoted 4 lines

One way to deal with it would be to extend the lttng-gen-tp tool directly build the tracepoint provider shared library instead of just outputting the generated .c .h files.

But then we'd _require_ that people use .so for providers. They might want to just pull the probe provider into their executable.

Quoted 8 lines
Quoted 5 lines

This way the shared library name
would be controlled by lttng-gen-tp tool and an autogenerated macro (e.g. ensure_tracepoints_init()) in the generated .h file could provide the dlopen code to explicitly load the tracepoint provider shared library if needed.

Would this work fine with your constructor instrumentation use-case ? Is the order of constructors across .so files guaranteed in any way ?

Only if we use dlopen *exclusively* and do not rely on any linker induced shared object loading (-lfoo -lbar) anymore we would have full control over the order of execution.

--Paul

Quoted 22 lines

Thanks,

Mathieu

Quoted 16 lines

Thanks,
Paul

Signature · 10 lines
Paul Woegerer | SW Development Engineer
http://go.mentor.com/sourceryanalyzer

Mentor Embedded(tm) | Prinz Eugen Straße 72/2/4, Vienna, 1040 Austria
Nucleus® | Linux® | Android(tm) | Services | UI | Multi-OS

Android is a trademark of Google Inc. Use of this trademark is subject
to Google Permissions.
Linux is the registered trademark of Linus Torvalds in the U.S. and
other countries.
Signature · 10 lines
Paul Woegerer | SW Development Engineer
http://go.mentor.com/sourceryanalyzer

Mentor Embedded(tm) | Prinz Eugen Straße 72/2/4, Vienna, 1040 Austria
Nucleus® | Linux® | Android(tm) | Services | UI | Multi-OS

Android is a trademark of Google Inc. Use of this trademark is subject
to Google Permissions.
Linux is the registered trademark of Linus Torvalds in the U.S. and
other countries.