Bug in LTTng-UST
Actually, I think the problem comes from UST not handling when a program dlcloses a lib. There is a simpler way to show the problem. Just have an app dlopen a .so that has a tracepoint and dlclose it. When the app quits, UST will try to unregister the tracepoint and read/write in the memory previously mapped to the library, causing a segfault.
I have made a small example. You just need to click on "Download gist" in [1] to download a tarball, and a then simple make should be enough. Start a UST session with -u -a and run ./app. Segfault should ensue.
[1] https://gist.github.com/simark/9fc0e2395899fcd8d10d
Quoted 25 lines · Matthew Khouzam
Hi, it appears as though if a tracepoint is in a so, loading it, unloading it and reloading it makes it no longer visible.
To confirm this you can try out this repo :
https://github.com/TheMatthew/yquake2
(I know, it's not a serious program... but the bug is serious, I think) You can get the demo files here
http://www.fileplanet.com/6584/0/fileinfo/Quake-2-Demo-v3.14
make the program (make from the git dir)
copy the contents of the baseq2 in the demo into the release/baseq2 directorystart a ust session
run the program.
Ignore the baddr deadlocks :)
The tracepoint is not visible, it is located in the game.so file.Chances are though that the problem is on my side, but the evidence (single registration ) seems to indicate that this is an actual problem.
_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Quoted 7 lines · Simon Marchi
Actually, I think the problem comes from UST not handling when a program dlcloses a lib. There is a simpler way to show the problem. Just have an app dlopen a .so that has a tracepoint and dlclose it. When the app quits, UST will try to unregister the tracepoint and read/write in the memory previously mapped to the library, causing a segfault.
Quoted 5 lines
I have made a small example. You just need to click on "Download gist" in [1] to download a tarball, and a then simple make should be enough. Start a UST session with -u -a and run ./app. Segfault should ensue.
Don't you have to _preload_ the tracepoint provider in this context? Either by dlopen()ing the tracepoint provider shared object manually, or by using LD_PRELOAD?
Try this instead <https://gist.github.com/eepp/d280a27f5e7d0a64abaa>.
Look at the changes to Makefile, tracepoints.c and truc.c. Make, then launch the app while tracing is started:
./app
Nothing is recorded: the tracepoint provider shared object is not loaded. Now try preloading the tracepoint provider and launching the app:
LD_PRELOAD=./tracepoints.so ./app
I get this:
lttng stop
lttng view
...
[23:55:36.393480541] (+0.000074994) eeppdesksample_tracepoint:message: { cpu_id = 0 }, { intfield = 4 }
and no segfault.
See: <http://lttng.org/docs/#doc-building-tracepoint-providers-and-user-application>,
"Dynamic linking" section.
Phil
Quoted 35 lines
Quoted 17 lines · Matthew Khouzam
Hi, it appears as though if a tracepoint is in a so, loading it, unloading it and reloading it makes it no longer visible.
To confirm this you can try out this repo :
https://github.com/TheMatthew/yquake2
(I know, it's not a serious program... but the bug is serious, I think) You can get the demo files here
http://www.fileplanet.com/6584/0/fileinfo/Quake-2-Demo-v3.14
make the program (make from the git dir)
copy the contents of the baseq2 in the demo into the release/baseq2 directorystart a ust session
run the program.
Ignore the baddr deadlocks :)
The tracepoint is not visible, it is located in the game.so file.Quoted 8 lines
Chances are though that the problem is on my side, but the evidence (single registration ) seems to indicate that this is an actual problem.
_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Quoted 35 lines · Philippe Proulx
Quoted 7 lines · Simon Marchi
Actually, I think the problem comes from UST not handling when a program dlcloses a lib. There is a simpler way to show the problem. Just have an app dlopen a .so that has a tracepoint and dlclose it. When the app quits, UST will try to unregister the tracepoint and read/write in the memory previously mapped to the library, causing a segfault.
Quoted 5 lines
I have made a small example. You just need to click on "Download gist" in [1] to download a tarball, and a then simple make should be enough. Start a UST session with -u -a and run ./app. Segfault should ensue.
Don't you have to _preload_ the tracepoint provider in this context? Either by dlopen()ing the tracepoint provider shared object manually, or by using
LD_PRELOAD?Try this instead <https://gist.github.com/eepp/d280a27f5e7d0a64abaa>.
Look at the changes toMakefile,tracepoints.candtruc.c. Make, then launch the app while tracing is started:./appNothing is recorded: the tracepoint provider shared object is not loaded. Now try preloading the tracepoint provider and launching the app:
LD_PRELOAD=./tracepoints.so ./app
I get this:
lttng stop lttng view ... [23:55:36.393480541] (+0.000074994) eeppdesk
Quoted 8 lines
sample_tracepoint:message: {
cpu_id= 0 }, { intfield = 4 }and no segfault.
See: <http://lttng.org/docs/#doc-building-tracepoint-providers-and-user-application>,
"Dynamic linking" section.Phil
Ok so as Philippe noticed, the man page is quite clear about this.
- Note about dlclose() usage: it is not safe to use dlclose on a provider shared object that is being actively used for tracing due to a lack of reference counting from lttng-ust to the used shared object.
Given that game.so *is* the trace provider, we can't dlclose it. I suppose the solution would be to put the tracepoints in a separate library (game-tracepoints.so) and LD_PRELOAD that library whenever you want to trace. I'll work on that.