Skip to contentSkip to conversation

running lttng 2.10 on Docker 1.13.1

DISCUSSION whalajam--- via lttng-dev 2 participants 3 messages Apr 27, 2019 – Apr 29, 2019 3 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

Hi,I have one docker container on which I installed LTTng version 2.10.I use the same Docker container to build and run the application. The problem I have is that there are no traces in the path specified when doing "lttng create". "lttng view"fails complaining about files not found, the trace directories are sometimes added to the path but the /root/lttng../../../channel0_* files are empty.=C2=A0I've followed the installation instructions from=C2=A0http://frederic-wou.net/lttng/and docs on=C2=A0https://lttng.org/docs/v2.10/=C2=A0These are the commands: "lttng enable -u a:b", "lttng start", <excercise-the-tracepoints>,"lttng stop" and "lttng view". lttng-consumerd is running. Running 'lttng-sessiond --vvv' returns DEBUG info and an error about server already runningWhat could be the reason for the failure, how can I debug it?John

_______________________________________________
lttng-dev mailing list
[email protected]
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): [email protected]

#3 lore raw

I attach the lttng-sessiond -vvv output,John

    On Friday, April 26, 2019, 9:38:36 p.m. EDT, whalajam--- via lttng-dev <[email protected]> wrote:  

Hi,I have one docker container on which I installed LTTng version 2.10.I use the same Docker container to build and run the application. The problem I have is that there are no traces in the path specified when doing "lttng create". "lttng view"fails complaining about files not found, the trace directories are sometimes added to the path but the /root/lttng../../../channel0_* files are empty.=C2=A0I've followed the installation instructions from=C2=A0http://frederic-wou.net/lttng/and docs on=C2=A0https://lttng.org/docs/v2.10/=C2=A0These are the commands: "lttng enable -u a:b", "lttng start", <excercise-the-tracepoints>,"lttng stop" and "lttng view". lttng-consumerd is running. Running 'lttng-sessiond --vvv' returns DEBUG info and an error about server already runningWhat could be the reason for the failure, how can I debug it?John

_______________________________________________
lttng-dev mailing list
[email protected]
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

# lttng-sessiond -vvv
DEBUG1 - 22:36:39.908428600 [23511/23511]: [sessiond configuration]
DEBUG1 - 22:36:39.908544400 [23511/23511]: 	verbose:                       3
DEBUG1 - 22:36:39.908553000 [23511/23511]: 	verbose consumer:              0
DEBUG1 - 22:36:39.908560800 [23511/23511]: 	quiet mode:                    False
DEBUG1 - 22:36:39.908568000 [23511/23511]: 	agent_tcp_port:                [5345, 5354]
DEBUG1 - 22:36:39.908575400 [23511/23511]: 	application socket timeout:    5
DEBUG1 - 22:36:39.908583000 [23511/23511]: 	no-kernel:                     False
DEBUG1 - 22:36:39.908591600 [23511/23511]: 	background:                    False
DEBUG1 - 22:36:39.908599200 [23511/23511]: 	daemonize:                     False
DEBUG1 - 22:36:39.908606700 [23511/23511]: 	signal parent on start:        False
DEBUG1 - 22:36:39.908613700 [23511/23511]: 	tracing group name:            tracing
DEBUG1 - 22:36:39.908621200 [23511/23511]: 	kmod_probe_list:               None
DEBUG1 - 22:36:39.908628700 [23511/23511]: 	kmod_extra_probe_list:         None
DEBUG1 - 22:36:39.908635900 [23511/23511]: 	rundir:                        /var/run/lttng
DEBUG1 - 22:36:39.908643300 [23511/23511]: 	application socket path:       /var/run/lttng/lttng-ust-sock-7
DEBUG1 - 22:36:39.908651000 [23511/23511]: 	client socket path:            /var/run/lttng/client-lttng-sessiond
DEBUG1 - 22:36:39.908658400 [23511/23511]: 	wait shm path:                 /lttng-ust-wait-7
DEBUG1 - 22:36:39.908665400 [23511/23511]: 	health socket path:            /var/run/lttng/sessiond-health
DEBUG1 - 22:36:39.908672900 [23511/23511]: 	LTTNG_UST_CLOCK_PLUGIN:        None
DEBUG1 - 22:36:39.908680000 [23511/23511]: 	pid file path:                 /var/run/lttng/lttng-sessiond.pid
DEBUG1 - 22:36:39.908687300 [23511/23511]: 	lock file path:                /var/run/lttng/lttng-sessiond.lck
DEBUG1 - 22:36:39.908694400 [23511/23511]: 	session load path:             None
DEBUG1 - 22:36:39.908701700 [23511/23511]: 	agent port file path:          /var/run/lttng/agent.port
DEBUG1 - 22:36:39.908712100 [23511/23511]: 	consumerd32 path:              /var/run/lttng/ustconsumerd32
DEBUG1 - 22:36:39.908719700 [23511/23511]: 	consumerd32 bin path:          Unknown
DEBUG1 - 22:36:39.908726700 [23511/23511]: 	consumerd32 lib dir:           Unknown
DEBUG1 - 22:36:39.908733600 [23511/23511]: 	consumerd32 err unix sock path:/var/run/lttng/ustconsumerd32/error
DEBUG1 - 22:36:39.908740900 [23511/23511]: 	consumerd32 cmd unix sock path:/var/run/lttng/ustconsumerd32/command
DEBUG1 - 22:36:39.908748200 [23511/23511]: 	consumerd64 path:              /var/run/lttng/ustconsumerd64
DEBUG1 - 22:36:39.908755300 [23511/23511]: 	consumerd64 bin path:          /usr/lib64/lttng/libexec/lttng-consumerd
DEBUG1 - 22:36:39.908762700 [23511/23511]: 	consumerd64 lib dir:           /usr/lib64
DEBUG1 - 22:36:39.908772500 [23511/23511]: 	consumerd64 err unix sock path:/var/run/lttng/ustconsumerd64/error
DEBUG1 - 22:36:39.908784500 [23511/23511]: 	consumerd64 cmd unix sock path:/var/run/lttng/ustconsumerd64/command
DEBUG1 - 22:36:39.908791900 [23511/23511]: 	kconsumerd path:               /var/run/lttng/kconsumerd
DEBUG1 - 22:36:39.908799300 [23511/23511]: 	kconsumerd err unix sock path: /var/run/lttng/kconsumerd/error
DEBUG1 - 22:36:39.908807200 [23511/23511]: 	kconsumerd cmd unix sock path: /var/run/lttng/kconsumerd/command
DEBUG3 - 22:36:39.908815000 [23511/23511]: Creating LTTng run directory: /var/run/lttng (in create_lttng_rundir() at main.c:5301)
Error: A session daemon is already running.
DEBUG1 - 22:36:39.908960400 [23511/23511]: Cleaning up options (in sessiond_cleanup_options() at main.c:642)
DEBUG1 - 22:36:39.908973600 [23511/23511]: Destroying run_as worker (in run_as_destroy_worker() at runas.c:710)

_______________________________________________
lttng-dev mailing list
[email protected]
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): [email protected]

#5 lore raw

Hi John,

Quoted 15 lines · whalajam--- via lttng-dev

I attach the lttng-sessiond -vvv output,John

    On Friday, April 26, 2019, 9:38:36 p.m. EDT, whalajam--- via lttng-dev <[email protected]> wrote:  

Hi,I have one docker container on which I installed LTTng version 2.10.I use the same Docker container to build and run the application. The problem I have is that there are no traces in the path specified when doing "lttng create". "lttng view"fails complaining about files not found, the trace directories are sometimes added to the path but the /root/lttng../../../channel0_* files are empty.=C2=A0I've followed the installation instructions from=C2=A0http://frederic-wou.net/lttng/and docs on=C2=A0https://lttng.org/docs/v2.10/=C2=A0These are the commands: "lttng enable -u a:b", "lttng start", <excercise-the-tracepoints>,"lttng stop" and "lttng view". lttng-consumerd is running. Running 'lttng-sessiond --vvv' returns DEBUG info and an error about server already runningWhat could be the reason for the failure, how can I debug it?John

Multiple things going on here.

First, I gather from your explanation and log from sessiond that you most probably installed lttng using a PPA or an equivalent. Note that some packages will install a service and launch a root lttng-sessiond on start.

To get a verbose lttng-sessiond, you can disable this service, and then run lttng-sessiond manually with the verbose option (-vvv --verbose-consumer).

   systemctl stop lttng-sessiond.serive
   systemctl disable lttng-sessiond.service
   lttng-sessiond -vvv --verbose-consumer

Or you can modify the service file. You will have to add the necessary options and systemd output redirection.

If this is not the case, make sure to kill the other instance of lttng-sessiond.

Quoted 2 lines

DEBUG3 - 22:36:39.908815000 [23511/23511]: Creating LTTng run directory: /var/run/lttng (in create_lttng_rundir() at main.c:5301)
Error: A session daemon is already running.

As for your actual "problem", the likely culprit is that either no instrumented application is tracing and no file are generated or you forgot to issue a "lttng stop" or "lttng destroy" (on destroy an implicit stop is done since 2.9 if I recall correctly) command.

When launching an app that you know is instrumented you can launch it with LTTNG_UST_DEBUG=y env variable defined to get more information e.g if the app is able to actually register to a lttng-sessiond instances.

Could you give us more information on the Docker image used
and if an open-source version of it is available so we can try and recreate the problem? I guess, most probably, a Centos image on the tutorial link you provided. We would still need the version and as much information you can provide.

Cheers