crosstap: a systemtap usability script
cover crosstap: a systemtap usability script
From: Tom Zanussi <[email protected]>
This patch implements a script called 'crosstap', which essentially encapsulates and automates the tedious steps until now required to use systemtap on a remote embedded target, outlined here:
https://wiki.yoctoproject.org/wiki/Tracing_and_Profiling
There's actually a Yocto bug tracking this work, see for a little more background:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=1551
I've tested the below scripts on a couple qemu targets (qemuarm and qemux86) as well as on real hardware (emenlow). I'll be doing a lot more testing of this script in the coming weeks as I address another enhancement (tracing and profiling HOWTOs):
https://bugzilla.yoctoproject.org/show_bug.cgi?id=1640
The examples tested were:
1) The simplest possible probe, which just prints 'hello world':
probe begin
{
print ("hello world\n")
exit ()
}[trz@empanada tutorial]$ crosstap [email protected] helloworld.stp ~/tracing/systemtap/scripts/tutorial ~/tracing/systemtap/scripts/tutorial ~/tracing/systemtap/scripts/tutorial
[email protected]'s password:
hello world
2) The next simplest possible probe, which prints 'hello world', but also demonstrates that any number of arguements can be passed to the systemtap script:
probe begin
{
printf("hello world with cmdline args, numeric arg: %d, string arg: %s\n", $1, @2)
exit ()
}[trz@empanada tutorial]$ crosstap [email protected] helloworld_args.stp 99 ninetynine -v ~/tracing/systemtap/scripts/tutorial ~/tracing/systemtap/scripts/tutorial
~/tracing/systemtap/scripts/tutorial
[email protected]'s password:
Pass 1: parsed user script and 83 library script(s) using 150500virt/22236res/2148shr/20612data kb, in 80usr/0sys/81real ms. Pass 2: analyzed script: 1 probe(s), 1 function(s), 0 embed(s), 0 global(s) using 150896virt/22760res/2224shr/21008data kb, in 0usr/0sys/2real ms. Pass 3: translated to C into "/tmp/stapJsYotR/helloworld_args_src.c" using 151020virt/23208res/2616shr/21132data kb, in 0usr/0sys/0real ms. Pass 4: compiled C into "helloworld_args.ko" in 1030usr/450sys/1568real ms.
Pass 5: starting run.
hello world with cmdline args, numeric arg: 99, string arg: ninetynine
Pass 5: run completed in 10usr/0sys/426real ms.
3) A more interesting, marginally useful test, instrumenting and printing out information about every 'open' syscall that happens on the system:
probe syscall.open
{
printf ("%s(%d) open (%s)\n", execname(), pid(), argstr)
}probe timer.ms(9000) # after 9 seconds
{
exit ()
}On the target system:
# cat /etc/*
[trz@empanada tutorial]$ crosstap [email protected] trace_open.stp ~/tracing/systemtap/scripts/tutorial ~/tracing/systemtap/scripts/tutorial ~/tracing/systemtap/scripts/tutorial
[email protected]'s password:
syslogd(811) open ("/var/log/messages", O_WRONLY|O_APPEND|O_CREAT|O_LARGEFILE|O_NOCTTY|O_NONBLOCK, 0666) cat(963) open ("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC|O_CLOEXEC)
cat(963) open ("/lib/libc.so.6", O_RDONLY|O_CLOEXEC|O_CLOEXEC)
cat(963) open ("X11", O_RDONLY|O_LARGEFILE)
cat(963) open ("apm", O_RDONLY|O_LARGEFILE)
cat(963) open ("asound.conf", O_RDONLY|O_LARGEFILE)
.
.
.
cat(963) open ("init.d", O_RDONLY|O_LARGEFILE)
cat(963) open ("inittab", O_RDONLY|O_LARGEFILE)
cat(963) open ("inputrc", O_RDONLY|O_LARGEFILE)
cat(963) open ("issue", O_RDONLY|O_LARGEFILE)
cat(963) open ("issue.net", O_RDONLY|O_LARGEFILE)
cat(963) open ("ld.so.cache", O_RDONLY|O_LARGEFILE)
matchbox-termin(891) open ("/tmp/vteNQTCKW", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600)
matchbox-termin(891) open ("/tmp/vteQOVCKW", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600)
matchbox-termin(891) open ("/tmp/vte0QKDKW", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600)
matchbox-termin(891) open ("/tmp/vteUBAFKW", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600)cat(963) open ("ld.so.conf", O_RDONLY|O_LARGEFILE) cat(963) open ("libnl", O_RDONLY|O_LARGEFILE) cat(963) open ("limits", O_RDONLY|O_LARGEFILE) cat(963) open ("login.access", O_RDONLY|O_LARGEFILE) cat(963) open ("login.defs", O_RDONLY|O_LARGEFILE) cat(963) open ("logrotate.d", O_RDONLY|O_LARGEFILE) cat(963) open ("matchbox", O_RDONLY|O_LARGEFILE) .
.
.
The following changes since commit 9ba1e33e2d14362971d6441ee6142bcb0857df1a:
sstate: Ensure master.list exists if it doesn't already (2012-08-30 22:45:56 -0700)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib.git tzanussi/crosstap.v1
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=tzanussi/crosstap.v1
Tom Zanussi (1):
crosstap: new script
scripts/crosstap | 148 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 148 insertions(+) create mode 100755 scripts/crosstap
Signature · 1 line
1.7.11.4
PATCH 1/1 crosstap: new script
From: Tom Zanussi <[email protected]>
This script makes it easy to run a systemtap script on a remote target from a build host.
The script itself contains documentation on basic setup and usage - see that for details. In a nutshell, if you have a systemtap script on the build host, this script allows that systemtap script to be compiled and run on the target host and for the output appear on the host terminal.
The crosstap script requires to an sdk build of the target (or a build with 'tools-profile' added to EXTRA_IMAGE_FEATURES) because it needs to be able to run the native systemtap built by the systemtap recipe, which in turn needs access to the kernel built for the target in order to build the kernel modules that implement the systemtap probe specified by the systemtap script and which ultimately get shipped to the target and insmod'ed there.
The crosstap script also needs to be able to ssh to the target host in order insert the modules on the target, so the target system needs to support incoming ssh connections.
--- scripts/crosstap | 148 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 148 insertions(+) create mode 100755 scripts/crosstap
scripts/crosstap new file +148 -0
@@ -0,0 +1,148 @@ +#!/bin/bash +# +# Run a systemtap script on remote target +# +# Examples (run on build host, target is 192.168.1.xxx): +# $ source oe-init-build-env" +# $ cd ~/my/systemtap/scripts" +# +# $ crosstap [email protected] myscript.stp" +# $ crosstap [email protected] myscript-with-args.stp 99 ninetynine" +# +# Copyright (c) 2012, Intel Corporation. +# All rights reserved. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. +# See the GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + +function usage() { + echo "Usage: $0 <user@hostname> <sytemtap-script> [additional systemtap-script args]" +} + +function setup_usage() { + echo "" + echo "'crosstap' requires a local sdk build of the target system" + echo "(or a build that includes 'tools-profile') in order to build" + echo "kernel modules that can probe the target system." + echo "" + echo "Practically speaking, that means you need to do the following:" + echo " - If you're running a pre-built image, download the release" + echo " and/or BSP tarballs used to build the image." + echo " - If you're working from git sources, just clone the metadata" + echo " and BSP layers needed to build the image you'll be booting." + echo " - Make sure you're properly set up to build a new image (see" + echo " the BSP README and/or the widely available basic documentation" + echo " that discusses how to build images)." + echo " - Build an -sdk version of the image e.g.:" + echo " $ bitbake core-image-sato-sdk" + echo " OR" + echo " - Build a non-sdk image but include the profiling tools:" + echo " [ edit local.conf and add 'tools-profile' to the end of" + echo " the EXTRA_IMAGE_FEATURES variable ]" + echo " $ bitbake core-image-sato" + echo "" + echo " [ NOTE that 'crosstap' needs to be able to ssh into the target" + echo " system, which isn't enabled by default in -minimal images. ]" + echo "" + echo "Once you've build the image on the host system, you're ready to" + echo "boot it (or the equivalent pre-built image) and use 'crosstap'" + echo "to probe it (you need to source the environment as usual first):" + echo "" + echo " $ source oe-init-build-env" + echo " $ cd ~/my/systemtap/scripts" + echo " $ crosstap [email protected] myscript.stp" + echo "" +} + +function systemtap_target_arch() { + SYSTEMTAP_TARGET_ARCH=$1 + case $SYSTEMTAP_TARGET_ARCH in + i?86) + SYSTEMTAP_TARGET_ARCH="i386" + ;; + x86_64*) + SYSTEMTAP_TARGET_ARCH="x86_64" + ;; + arm*) + SYSTEMTAP_TARGET_ARCH="arm" + ;; + powerpc*) + SYSTEMTAP_TARGET_ARCH="powerpc" + ;; + *) + ;; + esac +} + +if [ $# -lt 2 ]; then + usage + exit 1 +fi + +if [ -z "$BUILDDIR" ]; then + echo "Error: Unable to find the BUILDDIR environment variable." + echo "Did you forget to source your build system environment setup script?" + exit 1 +fi + +pushd $PWD +cd $BUILDDIR +BITBAKE_VARS=`bitbake -e virtual/kernel` +popd + +STAGING_BINDIR_TOOLCHAIN=$(echo "$BITBAKE_VARS" | grep ^STAGING_BINDIR_TOOLCHAIN \ + | cut -d '=' -f2 | cut -d '"' -f2) +STAGING_BINDIR_TOOLPREFIX=$(echo "$BITBAKE_VARS" | grep ^TARGET_PREFIX \ + | cut -d '=' -f2 | cut -d '"' -f2) +SYSTEMTAP_HOST_INSTALLDIR=$(echo "$BITBAKE_VARS" | grep ^STAGING_DIR_NATIVE \ + | cut -d '=' -f2 | cut -d '"' -f2) +TARGET_ARCH=$(echo "$BITBAKE_VARS" | grep ^TRANSLATED_TARGET_ARCH \ + | cut -d '=' -f2 | cut -d '"' -f2) +TARGET_KERNEL_BUILDDIR=$(echo "$BITBAKE_VARS" | grep ^B= \ + | cut -d '=' -f2 | cut -d '"' -f2) + +systemtap_target_arch "$TARGET_ARCH" + +if [ ! -d $TARGET_KERNEL_BUILDDIR ] || + [ ! -f $TARGET_KERNEL_BUILDDIR/vmlinux ]; then + echo -e "\nError: No target kernel build found." + echo -e "Did you forget to create a local build of your image?" + setup_usage + exit 1 +fi + +if [ ! -f $SYSTEMTAP_HOST_INSTALLDIR/usr/bin/stap ]; then + echo -e "\nError: Native (host) systemtap not found." + echo -e "Did you accidentally build a local non-sdk image? (or forget to" + echo -e "add 'tools-profile' to EXTRA_IMAGE_FEATURES in your local.conf)?" + setup_usage + exit 1 +fi + +target_user_hostname="$1" +full_script_name="$2" +script_name=$(basename "$2") +script_base=${script_name%.*} +shift 2 + +${SYSTEMTAP_HOST_INSTALLDIR}/usr/bin/stap \ + -a ${SYSTEMTAP_TARGET_ARCH} \ + -B CROSS_COMPILE="${STAGING_BINDIR_TOOLCHAIN}/${STAGING_BINDIR_TOOLPREFIX}" \ + -r ${TARGET_KERNEL_BUILDDIR} \ + -I ${SYSTEMTAP_HOST_INSTALLDIR}/usr/share/systemtap/tapset \ + -R ${SYSTEMTAP_HOST_INSTALLDIR}/usr/share/systemtap/runtime \ + --remote=$target_user_hostname \ + -m $script_base \ + $full_script_name "$@" + +exit 0
Signature · 1 line
1.7.11.4