scripts: Dump /proc and /sys
PATCH v2 1/1 scripts: Dump /proc and /sys
When a test fails during automated testing, the state of the system may be lost due to storage constraints or hardware availability. For example, if the system under test is virtualized, then the VM image may no longer be available due to storage capacity limits. Or if one is testing a bare metal system then the hardware may no longer be accessible for whatever reason.
The serial console output may be available, but lacking some information. So in order to retain more information (about the kernel at least), we can systematically copy the contents of /proc and /sys.
Not all sysfs or proc files are meant to be read or block indefinitely or are infinitely large etc. So the script only tries to copy ~256K from each file and times out after a second. Files which appear to be OK to read from are saved to a white list which can be used during future invocations to save some time.
---
V2:
* Fixed numerous style problems
* Instead of spawning X subshells per iteration I now only spawn $(nproc) subshells and pass files to them to through named pipes. This is much faster.
* Add some extra options
scripts/proc_sys_dump.sh | 145 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 145 insertions(+) create mode 100755 scripts/proc_sys_dump.sh
scripts/proc_sys_dump.sh new file +145 -0
@@ -0,0 +1,145 @@ +#!/bin/sh + +my_name=${0##*/} +dump_dir=/tmp/dump +dump_tar= +white_list=/tmp/dump/white.list +use_white_list= +timeout=1 +proc_exclude="/proc/\(self\|thread-self\|[0-9]+\)/.*" +page_sz=$(getconf PAGESIZE) +nproc=$(nproc) + +echo "Dump /proc and /sys script, written by [email protected]" + +while getopts d:c:w:u:t:b:h arg; do + case $arg in + d) dump_dir="$OPTARG" ;; + c) dump_tar="$OPTARG" ;; + w) white_list="$OPTARG" ;; + u) use_white_list="$OPTARG" ;; + t) timeout="$OPTARG" ;; + b) batch_size="$OPTARG" ;; + h) cat <<-EOF >&1 + +This script partially dumps the contents of /proc and /sys to a folder. It can +attempt to dump all the contents and then write out which files were +successful to a white list. Or use a previously generated white list. + +usage: +$my_name [ -d dump_dir ] + [ -w white_list ] [ -u use_white_list ] + [ -t timeout ] + [ -b batch_size ] +$my_name -h + +-d dump_dir The output directory, note that this will be recreated. + Set to $dump_dir + +-c dump_tar Compress the output directory contents into a tar file + specified by dump_tar. + +-w white_list The white list to output after dumping the files. + Set to $white_list + +-u use_white_list The list of files to dump, if this is unset then 'find' is + used to create a list. + +-t timeout How long, in seconds, we should wait for operations to finish. + Set to $timeout. + +-h Print this help message and exit. + +Files are added to the white list if they are small, but not empty and can be +copied quickly. The idea is that when running the script with -w white_list +the majority of useful information can be quickly saved during automated +testing. + +It is best to run this script as root otherwise sensitive information will be +filtered out by the kernel e.g. memory layout. Which happens to be the most +useful information in most situations. + +EOF + exit 0 ;; + esac +done + +files= +if [ ! $use_white_list ]; then + echo "Finding files in /proc and /sys" + proc_files=$(find -L /proc -maxdepth 4 -perm /u=r,g=r -not -regex "$proc_exclude" -not -type d 2> /dev/null) + sys_files=$(find -L /sys -maxdepth 4 -perm /u=r,g=r -not -type d 2> /dev/null) + files="$proc_files $sys_files" +else + echo "Using file list $use_white_list" + files=$(cat $use_white_list) +fi + +dump_dir=${dump_dir%/} +echo "Dumping to $dump_dir" +rm -rf $dump_dir +mkdir $dump_dir + +# This is run inside background subshells with stdin redirected to a named pipe. +spawn_dd () { + local f= + + read -r f + until [ "$f" = "EXIT" ]; do + mkdir -p $dump_dir$(dirname $f) + timeout $timeout dd if=$f of=$dump_dir$f bs=$page_sz count=63 status=none + if [ $? -eq 124 ]; then + echo "dd $f took too long" + rm $dump_dir$f + fi + read -r f + done +} + +for i in $(seq $nproc); do + mkfifo "$dump_dir/fifo$i" + ( spawn_dd ) < "$dump_dir/fifo$i" & +done + +count=$nproc +for f in $files; do + if [ $count -gt 0 ]; then + echo "$f" > "$dump_dir/fifo$count" + count=$((count - 1)) + else + count=$nproc + fi +done + +for i in $(seq $nproc); do + echo "EXIT" > "$dump_dir/fifo$i" +done + +wait + +for i in $(seq $nproc); do + rm "$dump_dir/fifo$i" +done + +if [ $white_list ]; then + echo "Finished copying files, now generating $white_list" + for f in $(find $dump_dir -not -type d); do + size=$(stat -c '%s' $f) + + if [ ${size:=0} -gt 0 -a $size -lt 256000 ]; then + echo "${f#$dump_dir}" >> $white_list + else + echo "$f has invalid file size: ${size:=0}" + fi + done +else + echo "Finished copying files." +fi + +if [ $dump_tar ]; then + echo "Compressing dumped files into $dump_tar" + oldpwd=${pwd} + cd $dump_dir + tar -cJf $dump_tar * + cd $oldpwd +fi
Signature · 1 line
2.15.0
PATCH v2 1/1 scripts: Dump /proc and /sys
Hi!
Looks very good in general a few comments below.
Also have you tried to run it with dash or some alternative shell? I haven't spotted any bashism but testing for compatibility never hurts :-)
Quoted 49 lines
Signed-off-by Richard Palethorpe <[email protected]>---
V2:
* Fixed numerous style problems
* Instead of spawning X subshells per iteration I now only spawn $(nproc) subshells and pass files to them to through named pipes. This is much faster.
* Add some extra optionsscripts/proc_sys_dump.sh | 145 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 145 insertions(+) create mode 100755 scripts/proc_sys_dump.shscripts/proc_sys_dump.sh new file +30
-0@@ -0,0 +1,145 @@ +#!/bin/sh + +my_name=${0##*/} +dump_dir=/tmp/dump +dump_tar= +white_list=/tmp/dump/white.list +use_white_list= +timeout=1 +proc_exclude="/proc/\(self\|thread-self\|[0-9]+\)/.*" +page_sz=$(getconf PAGESIZE) +nproc=$(nproc) + +echo "Dump /proc and /sys script, written by [email protected]" + +while getopts d:c:w:u:t:b:h arg; do + case $arg in + d) dump_dir="$OPTARG" ;; + c) dump_tar="$OPTARG" ;; + w) white_list="$OPTARG" ;; + u) use_white_list="$OPTARG" ;; + t) timeout="$OPTARG" ;; + b) batch_size="$OPTARG" ;; + h) cat <<-EOF >&1 + +This script partially dumps the contents of /proc and /sys to a folder. It can +attempt to dump all the contents and then write out which files were +successful to a white list. Or use a previously generated white list. + +usage: +$my_name [ -d dump_dir ]
scripts/proc_sys_dump.sh line 40
Quoted 10 lines
scripts/proc_sys_dump.sh new file +10
-0+ [ -w white_list ] [ -u use_white_list ] + [ -t timeout ] + [ -b batch_size ] +$my_name -h + +-d dump_dir The output directory, note that this will be recreated. + Set to $dump_dir + +-c dump_tar Compress the output directory contents into a tar file + specified by dump_tar.
This should say that we are compressing to tar.xz, "compressing to tar" does not make any sense since tar just puts several files into a single one.
Moreover xz is not supported on slightly older distros so we may as well be more conservative and use bz2 that is more widely supported instead. Or eventually we may add fallback to bz2 if xz compression fails.
scripts/proc_sys_dump.sh line 46
Quoted 5 lines
scripts/proc_sys_dump.sh new file +5
-0+-w white_list The white list to output after dumping the files. + Set to $white_list + +-u use_white_list The list of files to dump, if this is unset then 'find' is + used to create a list.
I would have dropped the use_ from the use_white_list since IMHO that is more confusing than helping here.
Quoted 54 lines
scripts/proc_sys_dump.sh new file +54
-0+-t timeout How long, in seconds, we should wait for operations to finish. + Set to $timeout. + +-h Print this help message and exit. + +Files are added to the white list if they are small, but not empty and can be +copied quickly. The idea is that when running the script with -w white_list +the majority of useful information can be quickly saved during automated +testing. + +It is best to run this script as root otherwise sensitive information will be +filtered out by the kernel e.g. memory layout. Which happens to be the most +useful information in most situations. + +EOF + exit 0 ;; + esac +done + +files= +if [ ! $use_white_list ]; then + echo "Finding files in /proc and /sys" + proc_files=$(find -L /proc -maxdepth 4 -perm /u=r,g=r -not -regex "$proc_exclude" -not -type d 2> /dev/null) + sys_files=$(find -L /sys -maxdepth 4 -perm /u=r,g=r -not -type d 2> /dev/null) + files="$proc_files $sys_files" +else + echo "Using file list $use_white_list" + files=$(cat $use_white_list) +fi + +dump_dir=${dump_dir%/} +echo "Dumping to $dump_dir" +rm -rf $dump_dir +mkdir $dump_dir + +# This is run inside background subshells with stdin redirected to a named pipe. +spawn_dd () { + local f= + + read -r f + until [ "$f" = "EXIT" ]; do + mkdir -p $dump_dir$(dirname $f) + timeout $timeout dd if=$f of=$dump_dir$f bs=$page_sz count=63 status=none + if [ $? -eq 124 ]; then + echo "dd $f took too long" + rm $dump_dir$f + fi + read -r f + done +} + +for i in $(seq $nproc); do + mkfifo "$dump_dir/fifo$i" + ( spawn_dd ) < "$dump_dir/fifo$i" &
scripts/proc_sys_dump.sh line 111
Quoted 10 lines
scripts/proc_sys_dump.sh new file +10
-0+done + +count=$nproc +for f in $files; do + if [ $count -gt 0 ]; then + echo "$f" > "$dump_dir/fifo$count" + count=$((count - 1)) + else + count=$nproc + fi
Uh, shouldn't the be line with echo "$f" here? Since otherwise we will skipp each $nproc+1 file in $files.
Quoted 5 lines
scripts/proc_sys_dump.sh new file +5
-0+done + +for i in $(seq $nproc); do + echo "EXIT" > "$dump_dir/fifo$i" +done
Signature · 2 lines
Cyril Hrubis [email protected]