Emitting events from shared object constructors is (currently) not possible
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 23 lines · * Woegerer, Paul ([email protected]) wrote:
I recently stumbled over the following lttng-ust limitation:
I tried to emit events from a shared objects constructor function:
__attribute__((constructor))
voidfunc_constructor(){ tracepoint( foo, bar ); }Unfortunately this doesn't work because the constructor functions of LTTng itself (
__tracepoints__initand the ones fromust-tracepoint-event.h) are not yet run.Trying to solve this problem with explicit constructor priority on my side alone doesn't work:
__attribute__((constructor (1111)))
voidfunc_constructor(){ tracepoint( foo, bar ); }
Quoted 5 lines
That's because whatever I specify as priority is irrelevant. LTTng's constructors without priority will always come last.
After also adding priority to the lttng constructor I was able to make it work (see patch below):
Hi Paul,
I notice that you probably embed your probes along with your code within the same shared object/executable, right ?
Unfortunately, this approach does not work if the probes are placed in a different shared object than the program code. Here is an example:
a.c:
#include <stdio.h>
__attribute__((constructor(998)))
void fcta_prio1(void)
{
fprintf(stderr, "fcta prio 998\n");
}__attribute__((constructor))
void fcta(void)
{
fprintf(stderr, "fcta\n");
}__attribute__((constructor(1000)))
void fcta_prio2(void)
{
fprintf(stderr, "fcta prio 1000\n");
}b.c:
#include <stdio.h>
__attribute__((constructor(999)))
void fctb_prio1(void)
{
fprintf(stderr, "fctb prio 999\n");
}__attribute__((constructor))
void fctb(void)
{
fprintf(stderr, "fctb\n");
}__attribute__((constructor(1001)))
void fctb_prio2(void)
{
fprintf(stderr, "fctb prio 1001\n");
}main.c:
int main()
{
return 0;
}Makefile:
all:
gcc -fPIC -shared -Wl,-soname,liba.so -o liba.so a.c gcc -fPIC -shared -Wl,-soname,libb.so -o libb.so b.c gcc -o test main.c -L./ -la -lb
exec with:
LD_LIBRARY_PATH=. ./test
result:
fctb prio 999
fctb prio 1001
fctb
fcta prio 998
fcta prio 1000
fcta
If we build test2 all within the same executable, then the order is fine:
gcc -o test2 main.c a.c b.c
./test2
result:
fcta prio 998
fctb prio 999
fcta prio 1000
fctb prio 1001
fcta
fctb
As we can notice, the priority is effective within a shared object, but has no ordering effect across .so. And having probes in separate .so is a very typical use case.
One way we could envision allowing tracing of constructors in applications is to ensure __tracepoints__init, which always resides within the same object as the traced application code, runs with a high constructor priorita -- low value -- (for code compiled by compilers supporting this feature), and allow the application to invoke the probe constructor early. So the proposed changes would be:
1 - add a constructor priority for
__tracepoints__init/__tracepoints_destroy in tracepoint.h. Use macros to test for gcc version which allow this attribute.
2 - add some API that could let user code call
__lttng_events_init__PROVIDER_NAME explicitly.
Add an "already executed" flag to guard __lttng_events_init__PROVIDER_NAME.
This API should still allow to LD_PRELOAD the probes (no dependency on the probe symbols, possibly by using dlsym ?)
3 - make lttng_probe_register call lttng_ust_init needed.
Add an "already executed" flag to guard lttng_ust_init.
Thoughts ?
Thanks,
Mathieu
Quoted 62 lines
include/lttng/tracepoint.h +2
-2@@ -261,7 +261,7 @@ int __tracepoint_registered struct tracepoint_dlopen tracepoint_dlopen __attribute__((weak, visibility("hidden"))); -static void lttng_ust_notrace __attribute__((constructor)) +static void lttng_ust_notrace __attribute__((constructor (1000)))__tracepoints__init (void);static void __tracepoints__init (void )@@ -300,7 +300,7 @@ __tracepoints__init(void) __start___tracepoints_ptrs); } -static void lttng_ust_notrace __attribute__((destructor)) +static void lttng_ust_notrace __attribute__((destructor (1000)))__tracepoints__destroy (void);static void __tracepoints__destroy (void )include/lttng/ust-tracepoint-event.h +2
-2@@ -638,7 +638,7 @@ static struct lttng_probe_desc _TP_COMBINE_TOKENS(__probe_desc___, TRACEPOINT_PR/* Reset all macros within TRACEPOINT_EVENT */ #include <lttng/ust-tracepoint-event-reset.h> -static void lttng_ust_notrace __attribute__((constructor)) +static void lttng_ust_notrace __attribute__((constructor (1000))) _TP_COMBINE_TOKENS(__lttng_events_init__, TRACEPOINT_PROVIDER)(void);static void _TP_COMBINE_TOKENS (__lttng_events_init__ ,TRACEPOINT_PROVIDER )(void )@@ -649,7 +649,7 @@ _TP_COMBINE_TOKENS(__lttng_events_init__, TRACEPOINT_PROVIDER)(void)assert (!ret); } -static void lttng_ust_notrace __attribute__((destructor)) +static void lttng_ust_notrace __attribute__((destructor (1000))) _TP_COMBINE_TOKENS(__lttng_events_exit__, TRACEPOINT_PROVIDER)(void);static void _TP_COMBINE_TOKENS (__lttng_events_exit__ ,TRACEPOINT_PROVIDER )(void )Signature · 14 lines
-- Paul Woegerer | SW Development Engineer http://go.mentor.com/sourceryanalyzer Mentor Embedded(tm) | Prinz Eugen Straße 72/2/4, Vienna, 1040 Austria Nucleus® | Linux® | Android(tm) | Services | UI | Multi-OS Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries. _______________________________________________ lttng-dev mailing list [email protected] http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Signature · 4 lines
Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com
A message that is not in the archive (sent off-list or not yet received): 20121122153901.GA14685@Krystal
Quoted 30 lines · Mathieu Desnoyers
Quoted 23 lines · * Woegerer, Paul ([email protected]) wrote:
I recently stumbled over the following lttng-ust limitation:
I tried to emit events from a shared objects constructor function:
__attribute__((constructor))
voidfunc_constructor(){ tracepoint( foo, bar ); }Unfortunately this doesn't work because the constructor functions of LTTng itself (
__tracepoints__initand the ones fromust-tracepoint-event.h) are not yet run.Trying to solve this problem with explicit constructor priority on my side alone doesn't work:
__attribute__((constructor (1111)))
voidfunc_constructor(){ tracepoint( foo, bar ); }Quoted 5 lines
That's because whatever I specify as priority is irrelevant. LTTng's constructors without priority will always come last.
After also adding priority to the lttng constructor I was able to make it work (see patch below):
Quoted 7 lines
Hi Paul,
I notice that you probably embed your probes along with your code within the same shared object/executable, right ?
Unfortunately, this approach does not work if the probes are placed in a different shared object than the program code. Here is an example:
Hi Mathieu,
In case the probes would be defined in different shared object from the one where they are used we wouldn't have a problem at all because the dynamic linker would invoke all the constructor functions for the probe shared object before the constructor functions of the shared object that has a dependency to the probe shared object, right ?
And even if this is not the case I could explicitly dlopen the probe shared object from within my constructor function and thus make sure that the constructor function of the probe shared object gets called before I emit an event from my constructor function.
--
Paul
Signature · 10 lines
Paul Woegerer | SW Development Engineer http://go.mentor.com/sourceryanalyzer Mentor Embedded(tm) | Prinz Eugen Straße 72/2/4, Vienna, 1040 Austria Nucleus® | Linux® | Android(tm) | Services | UI | Multi-OS Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 39 lines · * Woegerer, Paul ([email protected]) wrote:
Quoted 30 lines · Mathieu Desnoyers
Quoted 23 lines · * Woegerer, Paul ([email protected]) wrote:
I recently stumbled over the following lttng-ust limitation:
I tried to emit events from a shared objects constructor function:
__attribute__((constructor))
voidfunc_constructor(){ tracepoint( foo, bar ); }Unfortunately this doesn't work because the constructor functions of LTTng itself (
__tracepoints__initand the ones fromust-tracepoint-event.h) are not yet run.Trying to solve this problem with explicit constructor priority on my side alone doesn't work:
__attribute__((constructor (1111)))
voidfunc_constructor(){ tracepoint( foo, bar ); }Quoted 5 lines
That's because whatever I specify as priority is irrelevant. LTTng's constructors without priority will always come last.
After also adding priority to the lttng constructor I was able to make it work (see patch below):
Quoted 7 lines
Hi Paul,
I notice that you probably embed your probes along with your code within the same shared object/executable, right ?
Unfortunately, this approach does not work if the probes are placed in a different shared object than the program code. Here is an example:
Quoted 7 lines
Hi Mathieu,
In case the probes would be defined in different shared object from the one where they are used we wouldn't have a problem at all because the dynamic linker would invoke all the constructor functions for the probe shared object before the constructor functions of the shared object that has a dependency to the probe shared object, right ?
The instrumented code does not have dependency on the probe shared object, so we can ship applications separately from their probe .so. So unfortunately, the linker is not doing this for us.
Quoted 5 lines
And even if this is not the case I could explicitly dlopen the probe shared object from within my constructor function and thus make sure that the constructor function of the probe shared object gets called before I emit an event from my constructor function.
The idea would be to provide a nice API to facilitate this use-case, and document it in the instrumentation guide-lines (lttng-ust(3)).
Thanks,
Mathieu
Quoted 15 lines
--
PaulSignature · 10 lines
Paul Woegerer | SW Development Engineer http://go.mentor.com/sourceryanalyzer Mentor Embedded(tm) | Prinz Eugen Straße 72/2/4, Vienna, 1040 Austria Nucleus® | Linux® | Android(tm) | Services | UI | Multi-OS Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.
Signature · 4 lines
Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com
A message that is not in the archive (sent off-list or not yet received): 20121122164512.GA16119@Krystal
Quoted 10 lines · Mathieu Desnoyers
Quoted 5 lines · * Woegerer, Paul ([email protected]) wrote:
In case the probes would be defined in different shared object from the one where they are used we wouldn't have a problem at all because the dynamic linker would invoke all the constructor functions for the probe shared object before the constructor functions of the shared object that has a dependency to the probe shared object, right ?
The instrumented code does not have dependency on the probe shared object, so we can ship applications separately from their probe .so. So unfortunately, the linker is not doing this for us.
But still, to come back to my original request, whenever the tracepoint provider is statically linked to the tracepoint user (which is a valid lttng-ust use case, right ?) it is not possible to emit tracepoints in object constructors as long as object constructors defined in lttng/tracepoint.h and lttng/ust-tracepoint-event.h are simply using __attribute__((constructor)) instead of e.g. __attribute__((constructor (1000))). Without this change the user simply cannot make sure its own constructor gets invoked after the trace provider constructors.
Quoted 7 lines
Quoted 4 lines
And even if this is not the case I could explicitly dlopen the probe shared object from within my constructor function and thus make sure that the constructor function of the probe shared object gets called before I emit an event from my constructor function.
The idea would be to provide a nice API to facilitate this use-case, and document it in the instrumentation guide-lines (lttng-ust(3)).
Therefore we would need to reliably deduce the tracepoint provider shared library name from the header file that defines it. Since C/C++ has no package concept and gives you the freedom to name your shared library whatever you like it seems to be hard to get there.
One way to deal with it would be to extend the lttng-gen-tp tool directly build the tracepoint provider shared library instead of just outputting the generated .c .h files. This way the shared library name would be controlled by lttng-gen-tp tool and an autogenerated macro (e.g. ensure_tracepoints_init()) in the generated .h file could provide the dlopen code to explicitly load the tracepoint provider shared library if needed.
Thanks,
Paul
Signature · 10 lines
Paul Woegerer | SW Development Engineer http://go.mentor.com/sourceryanalyzer Mentor Embedded(tm) | Prinz Eugen Straße 72/2/4, Vienna, 1040 Austria Nucleus® | Linux® | Android(tm) | Services | UI | Multi-OS Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 8 lines · * Woegerer, Paul ([email protected]) wrote:
Quoted 7 lines · Mathieu Desnoyers
Quoted 5 lines · * Woegerer, Paul ([email protected]) wrote:
In case the probes would be defined in different shared object from the one where they are used we wouldn't have a problem at all because the dynamic linker would invoke all the constructor functions for the probe shared object before the constructor functions of the shared object that has a dependency to the probe shared object, right ?
Quoted 7 lines
Quoted 3 lines
The instrumented code does not have dependency on the probe shared object, so we can ship applications separately from their probe .so. So unfortunately, the linker is not doing this for us.
But still, to come back to my original request, whenever the tracepoint provider is statically linked to the tracepoint user (which is a valid lttng-ust use case, right ?)
yes,
Quoted 5 lines
it is not possible to emit tracepoints in
object constructors as long as object constructors defined inlttng/tracepoint.handlttng/ust-tracepoint-event.hare simply using __attribute__((constructor)) instead of e.g.__attribute__((constructor (1000))).
Those can be emitted, but indeed they might not be traced.
Quoted 2 lines
Without this change the user simply cannot make sure its own constructor gets invoked after the trace provider constructors.
If we try to support tracing constructors, I'm concerned that with this change (using priority):
1 - we lose in terms of portability (not all gcc versions support the priority argument,
2 - there can be corner-cases that won't honor the priority.
We should thoroughly test and document the effect of priority in cases of:
- multiple .o linked into an executable,
- multiple .o linked into a .a, statically linked (does not honor constructors at all),
- multiple .o linked into a .so, many .so linked,
And figure out if the ordering of constructor is OK with each of those cases.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
And even if this is not the case I could explicitly dlopen the probe shared object from within my constructor function and thus make sure that the constructor function of the probe shared object gets called before I emit an event from my constructor function.
Quoted 7 lines
Quoted 2 lines
The idea would be to provide a nice API to facilitate this use-case, and document it in the instrumentation guide-lines (lttng-ust(3)).
Therefore we would need to reliably deduce the tracepoint provider shared library name from the header file that defines it. Since C/C++ has no package concept and gives you the freedom to name your shared library whatever you like it seems to be hard to get there.
indeed,
Quoted 4 lines
One way to deal with it would be to extend the lttng-gen-tp tool directly build the tracepoint provider shared library instead of just outputting the generated .c .h files.
But then we'd _require_ that people use .so for providers. They might want to just pull the probe provider into their executable.
Quoted 5 lines
This way the shared library name
would be controlled by lttng-gen-tp tool and an autogenerated macro (e.g.ensure_tracepoints_init()) in the generated .h file could provide the dlopen code to explicitly load the tracepoint provider shared library if needed.
Would this work fine with your constructor instrumentation use-case ? Is the order of constructors across .so files guaranteed in any way ?
Thanks,
Mathieu
Quoted 16 lines
Thanks,
PaulSignature · 10 lines
Paul Woegerer | SW Development Engineer http://go.mentor.com/sourceryanalyzer Mentor Embedded(tm) | Prinz Eugen Straße 72/2/4, Vienna, 1040 Austria Nucleus® | Linux® | Android(tm) | Services | UI | Multi-OS Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
A message that is not in the archive (sent off-list or not yet received): 20130503183500.GB7288@Krystal
Quoted 13 lines · Mathieu Desnoyers
Quoted 2 lines · * Woegerer, Paul ([email protected]) wrote:
Without this change the user simply cannot make sure its own constructor gets invoked after the trace provider constructors.
If we try to support tracing constructors, I'm concerned that with this change (using priority):
1 - we lose in terms of portability (not all gcc versions support the priority argument,
2 - there can be corner-cases that won't honor the priority.We should thoroughly test and document the effect of priority in cases of:
Quoted 7 lines
- multiple .o linked into an executable,
- multiple .o linked into a .a, statically linked (does not honor constructors at all),
- multiple .o linked into a .so, many .so linked,And figure out if the ordering of constructor is OK with each of those cases.
You are right. Using the priority construct might cause more problems than it solves.
A pragmatic solution to the above problem would be to provide a special documented function (as part of man:lttng-ust) that allows the user to ensure that the static constructors (withing the same "dynamic linker unit" (.so or exe)) used by lttng-ust have been executed successfully. Something like:
void ensure_lttng_ust_init_completed();
If the trace provider constructors have been invoked already the function simply returns otherwise it invokes them explicitly and makes sure that a later invocation by the dynamic linker will not have any effect.
Quoted 25 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
And even if this is not the case I could explicitly dlopen the probe shared object from within my constructor function and thus make sure that the constructor function of the probe shared object gets called before I emit an event from my constructor function.
Quoted 7 lines
Quoted 2 lines
The idea would be to provide a nice API to facilitate this use-case, and document it in the instrumentation guide-lines (lttng-ust(3)).
Therefore we would need to reliably deduce the tracepoint provider shared library name from the header file that defines it. Since C/C++ has no package concept and gives you the freedom to name your shared library whatever you like it seems to be hard to get there.
indeed,
Quoted 4 lines
One way to deal with it would be to extend the lttng-gen-tp tool directly build the tracepoint provider shared library instead of just outputting the generated .c .h files.
But then we'd _require_ that people use .so for providers. They might want to just pull the probe provider into their executable.
Quoted 8 lines
Quoted 5 lines
This way the shared library name
would be controlled by lttng-gen-tp tool and an autogenerated macro (e.g.ensure_tracepoints_init()) in the generated .h file could provide the dlopen code to explicitly load the tracepoint provider shared library if needed.Would this work fine with your constructor instrumentation use-case ? Is the order of constructors across .so files guaranteed in any way ?
Only if we use dlopen *exclusively* and do not rely on any linker induced shared object loading (-lfoo -lbar) anymore we would have full control over the order of execution.
--Paul
Quoted 22 lines
Thanks,
Mathieu
Quoted 16 lines
Thanks,
PaulSignature · 10 lines
Paul Woegerer | SW Development Engineer http://go.mentor.com/sourceryanalyzer Mentor Embedded(tm) | Prinz Eugen Straße 72/2/4, Vienna, 1040 Austria Nucleus® | Linux® | Android(tm) | Services | UI | Multi-OS Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.
Signature · 10 lines
Paul Woegerer | SW Development Engineer http://go.mentor.com/sourceryanalyzer Mentor Embedded(tm) | Prinz Eugen Straße 72/2/4, Vienna, 1040 Austria Nucleus® | Linux® | Android(tm) | Services | UI | Multi-OS Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.