systemtap 3.0 release
The SystemTap team announces release 3.0!
Experimental monitor and interactive modes, optimized associative arrays, two types of function overloading, probe point brace-pattern expansion, security band-aid samples, improved string quoting and escaping, pretty printing array aggregates, private scoping for variables / functions, oxford commas, and much more!
= Where to get it
https://sourceware.org/systemtap/ - our project page
https://sourceware.org/systemtap/ftp/releases/systemtap-3.0.tar.gz
https://koji.fedoraproject.org/koji/packageinfo?packageID=615
git tag release-3.0 (commit 616ec7a0b916df)
There have been over 450.32 commits since the last release. There have been between 57 and 60 bugs fixed / features added since the
last release.
= How to build it
See the README and NEWS files at
https://sourceware.org/git/?p=systemtap.git;a=tree
Further information at https://sourceware.org/systemtap/wiki/
= SystemTap frontend (stap) changes
- The new experimental "interactive" mode, specified by "stap -i", drops you into a command-line prompt where you can build up a script, run it, edit it, run it again, etc. Type "help" for a list of commands.
- New experimental --monitor[=INTERVAL] option similar to unix "top". This allows users to see statistics about the running module(uptime, module name, invoker uid, memory sizes, global variables, and the current probe list along with their statistics).
An interface is also provided to allow control over the running module (resetting global variables, sorting the list of probes, deactivating and reactivating probes).
- Add macros @prints to print a scalar aggregate variable, @prints[1-9] to print an array aggregate (of given index-arity), formatted similarly to the automatic printing of written-only global variables.
global a, b
probe oneshot { a <<< 1; b[tid()] <<< 2 }
probe end { @prints(a); @prints1(b) }
- Add Czech version of key manual pages.
- The stap compile server will log the stap client's options that are passed to the server. The options that get logged on the server will include the script name or the -e script, depending on which is used by the client.
= SystemTap script language changes
- The performance of associative arrays have been dramatically improved, especially for densely filled tables and for multiple indexes. The hash tables behind these arrays is now sized as a function of the array maximum size with an optional MAPHASHBIAS space/time tradeoff knob.
- Functions may now be overloaded during module runtime using the "next" statement in script functions and STAP_NEXT macro for embedded-C functions. They may also be overloaded by number of parameters during compile time. For example,
Runtime overloading:
function f() { if (condition) next; print("first function") }
function f() %{ STAP_NEXT; print("second function") %}
function f() { print("third function") } For the given functions above, a functioncall f(), will execute the body of the third function if condition evaluates to true and print "third function". Note that the second function is unconditionally nexted.
Parameter overloading:
function g() { print("first function") }
function g(x) { print("second function") }
g() -> "first function"
g(1) -> "second function"Note that runtime overloading does not occur in the above example as the number of parameters of the functions differ. The use of a next statement inside a function while no more overloads remain will trigger a runtime exception. The function candidates are selected at compile time and is determined by the number of arguments provided for the functioncall.
- Embedded-C functions and blocks may now access script level global variables using the STAP_GLOBAL_GET_* and STAP_GLOBAL_SET_* macros.
To read or write the script global var, the /* pragma:read:var */ or /* pragma:write:var */ marker must be placed in the embedded-C function or block. The written type must match the type inferred at script level.
Scalars:
STAP_GLOBAL_SET_var(STAP_GLOBAL_GET_var()+1) -> increments script global var by 1
STAP_GLOBAL_SET_var("hello")Associative arrays:
STAP_GLOBAL_GET_var(index-1, ..., index-n)
STAP_GLOBAL_SET_var(index-1, ..., index-n, new value)- Probe point brace expansion is now supported to improve brevity in specifying probe points. For example,
process.{function("a"), function("b").{call,return}}
=> process.function("a"), process.function("b").call, process.function("b").return
process.{function("*").{callees,plt}?
=> process.function("*").callees?, process.plt?
{kernel,module("nfs")}.function("nfs*")!
=> kernel.function("nfs*")!, module("nfs").function("nfs*")!
- Profiling timers at arbitrary frequencies are now provided and perf probes now support a frequency field as an alternative to sampling counts.
probe timer.profile.freq.hz(N)
probe perf.type(N).config(M).hz(X)
The specified frequency is only accurate up to around 100hz. You may need to provide a higher value to achieve the desired rate.
- Added support for private global variables and private functions. The 'private' keyword limits these to the tapset file they are defined in.
= SystemTap tapset changes
ansi.stp Functions ansi_set_color{2,3} are replaced by
overloaded ansi_set_color
linux/[arm/]aux_syscalls.stp Support for arm kernels less than 3.7.
linux/arm/[nd_]syscalls.stp Support for [nd_]syscall.execve for arm
kernels less than 3.7.
linux/aux_syscalls.stp New _stp_mlock2_str function to convert mlock2
syscall flags to a string.
linux/context.stp New module_size() function.
linux/conversions.stp - New kernel_string_quoted_utf[16|32] functions
combines @string_quoted and @kernel_string_utf*
- kernel_string* functions with alternative error strings are replaced by overloaded variants
linux/nd_syscalls.stp Add nd_syscall.mlock2 kprobe based probe point
linux/perf.stp Update recent uapi/linux/perf_event.h bits
linux/proc_mem.stp proc_mem_*_pid functions are replaced by overloaded
proc_mem_*
linux/syscalls.stp Add syscall.mlock2 kernel function probe point
linux/task.stp New task_cwd_path and task_exe_file functions
linux/task_time.stp task_{s,u}time_tid functions are replaced by
overloaded task_{s,u}time
linux/uconversions.stp user_string functions with alternative warning strings
are replaced by overloaded variants
logging.stp New overloaded variant of assert, assert(expression)
print_stats.stpm @prints* macros for printing stats
timers.stp Probe point timer.profile.freq for profiling
try_assign.stpm New @try_assign macro
uconversions.stp Add user_string_quoted_utf[16|32] function thatquotes a given UTF-[16|32] string from a given user address
- Internal tapset functions and global variables are marked as private, where possible.
- Some tapsets have been modified to make use of the new function overloading feature. Instead of having new function names with suffixes such as "2" or "pid" to indicate extra arguments, the functions now seem to have optional arguments.
- New tapset function string_quoted() to quote and \-escape general strings. String $context variables that are pretty-printed are now processed with such a quotation engine, falling back to a 0x%x (hex pointer) on errors
- Functions get_mmap_args() and get_32mmap_args() got deprecated.
= SystemTap sample scripts - now at 156 samples!
who_sent_it.stp Trace outgoing network packets using the netfilter probes,
printing the source thread name/id and destination host:port
- New to collection is a selection of security band aids for specific CVEs. They are historical emergency security band-aids, and are for reference/education only. The scripts can be found under the security-band-aids folder in the examples directory.
- A number of samples were tweaked for portability and demonstration of newer language/tapset facilities.
= Examples of tested kernel versions
2.6.18 (RHEL 5 x86 and x86_64)
2.6.32 (RHEL 6 x86 and x86_64)
3.10.0 (RHEL 7 x86_64)
4.1.6 (Fedora 22 x86_64)
4.3.4 (Fedora 22 x86_64)
4.6.0-rc0 (Fedora rawhide x86_64)
= Known issues with this release
- Some kernel crashes continue to be reported when a script probes broad kernel function wildcards. (PR2725)
- The dyninst backend is still very much a prototype, with a number of issues, limitations, and general teething woes. See dyninst/README and the systemtap/dyninst Bugzilla component (
http://tinyurl. com/stapdyn-PR-list ) if you want all the gory details about the state of the feature.
- An upstream kernel commit #2062afb4f804a put "-fno-var-tracking-assignments" into KCFLAGS, reducing debuginfo quality which can cause debuginfo failures. A proposed workaround to this issue exists in:
https://lkml.org/lkml/2014/11/21/505 . Fedora kernels are not affected by this issue.
= Contributors for this release
Abegail Jakop, David Smith, Felix Lu, Frank Ch. Eigler, Jesper Brouer*, Jonathan Lebon, Josh Stone, Ken Raeburn*, Lukas Berk, Mark Wielaard, Martin Cermak, Nan Xiao*, Nikolay Borisov*, Orivej Desh*, Stan Cox, Torsten Polle, William Cohen, Zhou Wenjian*
Special thanks to new contributors, marked with '*' above. Special thanks to Abegail for compiling these notes.
= Bugs fixed for this release <https://sourceware.org/PR#####>
6525 need utrace task-finder-based pid->execname, pid->cwd-path-name tables
6977 Incorrect resolution of variables in function scope
9968 Share global variable between embedded C and script
10607 need way to protect sensitive tapset globals
11347 intra-tapset visibility scoping for embedded-c functions
12414 Missing return value is not reported
13514 more robust -rt support
13608 Limit the size of the request from the compile server client
13643 log stap options
13721 local variable name collision
13820 timer.profile(NNN) variant using perf probes
14203 Embedded-C functions in various tapsets are not stress tested
14431 char * always being printed as a, possibly "<unknown>", without the address
15775 relay host/guest functionality is broken on RHEL6
16672 Allow brace expansion on probe points
17393 java.exp: ERROR: Cannot attach to module control channel
17587 too sensitive to duplicate globals across tapset files
17741 try-catch statement as a tapset function
18075 Add a return-probe variant of .callee
18431 runtime function overloading facility
18848 new --monitor option
18876 need to assert
18884 const-folded literals get unnecessary tmp storage
18889 lost ability to probe kernel module initializers
19120 kbuild warnings on RHEL5
19135 testsuite results hard to compare
19136 "private" keyword for tapsets
19216 kprobe.module("MOD").function("FOO") doesn't work on not loaded modules
19234 user_string_n2_quoted() vs. user_string_n_quoted.exp
19236 NULL global variable breakage
19241 systemtap fails to start when CPU0 is offline
19265 finding out-of-tree kernel modules works with full paths, not relative paths
19275 the stap_run testsuite function doesn't handle unexpected output
19345 RHEL 7.0 s390x crash in check.exp
19346 escape xml documentation
19358 netfilter probes are broken on rawhide
19362 translator puts two probes for each probe point in .../library.exp tests
19396 systemtap can't find certain kernel tracepoints
19470 process_by_pid.exp always fails on arches that don't support .plt probes
19497 linux 4.4 commit 7523e4dc50
19502 extend stap.1 with documentation overview
19510 the "private" keyword support has made -p1 output less useful
19521 the "private" keyword support has made error messages less useful
19525 script doesn't finish in bulk mode
19536 kretprobe data-pouch type issue
19537 the parseok/semko.stp test case always fails
19560 flight recorder mode intermittently fails
19639 parallel_exec.exp fails in beaker-project.org
19644 linux 4.5-rc4 commit 8244062ef1
19700 Support dash as /bin/sh in generated Makefile
19756 linux 4.5-rc1 commit 8463833590
19764 stap --tmpdir leaves useless tempdirs behind
19799 deleting from array of aggregate unreliable
19802 bad hash value distribution and horrible performance for large arrays
19812 _stp_target misdirected for pid_namespace containers
19833 monitor mode causes kernel crash when exited immediately
19847 Entering ^C in interactive mode prompt prevents script from running
19873 staprun -o /NO/SUCH/FILE -c CMD imperfect cleanup