Allow easy re-running of grub-shell from failed tests
cover Allow easy re-running of grub-shell from failed tests
When a non-native (eg. grub-shell/qemu) test fails it can be useful to be able to re-run the grub-shell invocation in at least two circumstances.
1. The tester wants to re-run the grub-shell invocation with extra args so that QEMU starts in GDB attach mode for using GDB to debug the test case
2. For a tester who wants to send a failed test case to another developer for additional debugging. It would be nice to be able to just zip/tar just the failed test case test output directory and allow the extracted directory to easily run the failed test case again. The results in this case still might not be the same if the sender and reciever do not have the same QEMU. Though when the QMEU versions are different, I expect the emulated execution on sender and reciever to be close enough to produce the same execution path (maybe not with super old QEMUs, but I don't think its something to be concerned about).
The first patch is needed to make the other two useful. Without it, I know of no easy way to enable grub-shell's debug mode when run via make check. And if grub-shell is not in debug mode then it will remove most/all of the generated files, which would prevent debugging.
Glenn
Glenn Washburn (3):
tests: Allow turning on shell tracing from environment variables
grub-shell: Create run.sh in working directory for easily running test
again
grub-shell: Add $GRUB_QEMU_OPTS to run.sh to easily see unofficial
QEMU arguments
tests/util/grub-fs-tester.in | 2 ++ tests/util/grub-shell.in | 37 +++++++++++++++++++++++++++++------- 2 files changed, 32 insertions(+), 7 deletions(-)
Signature · 1 line
2.34.1
PATCH 1/3 tests: Allow turning on shell tracing from environment variables
This allows turning on shell tracing for grub-shell and grub-fs-tester when its not practical or not possible to use commandline arguments (eg. from make check). Turn on tracing when the envvar is an integer greater than 1, since these can generate a lot of output. Since this change uses the environment variables to set the default value for debug in grub-shell, this allows enabling grub-shell's debug mode which will preserve various generated output files that are helpful for debugging tests.
--- tests/util/grub-fs-tester.in | 2 ++ tests/util/grub-shell.in | 5 ++++- 2 files changed, 6 insertions(+), 1 deletion(-)
tests/util/grub-fs-tester.in +2 -0
@@ -6,6 +6,8 @@ export BLKID_FILE=/dev/null # We can't have this set, or filesystem tests will fail. unset SOURCE_DATE_EPOCH +[ "${GRUB_TEST_DEFAULT_DEBUG:-0}" -gt 1 ] && set -x + fs="$1" GRUBFSTEST="@builddir@/grub-fstest"
tests/util/grub-shell.in +4 -1
@@ -216,6 +216,7 @@ esac timeout=60 mkimage_extra_arg= +debug=${GRUB_SHELL_DEFAULT_DEBUG:-$GRUB_TEST_DEFAULT_DEBUG} # Check the arguments. for option in "$@"; do
@@ -235,7 +236,7 @@ for option in "$@"; do --no-trim) trim=0 ;; --debug) - debug=1 ;; + debug=$((debug+1)) ;; --modules=*) ms=`echo "$option" | sed -e 's/--modules=//' -e 's/,/ /g'` modules="$modules $ms" ;;
@@ -320,6 +321,8 @@ for option in "$@"; do esac done +[ "${debug:-0}" -gt 1 ] && set -x + if [ "x${source}" = x ] ; then tmpfile="$work_directory/testcase.cfg" while read REPLY; do
Signature · 1 line
2.34.1
PATCH 2/3 grub-shell: Create run.sh in working directory for easily running test again
In reply to Glenn WashburnNow it becomes trivial to re-run a test from the output in its working directory. This also makes it easy to send a reproducible failing test to the mailing list. This has allowed a refactor so that the duplicated code to call QEMU has be condensed (eg. the use of timeout and file descriptor redirection). The run.sh script will pass any arguments given to QEMU. This allows QEMU to be easily started in a state ready for GDB to be attached.
--- tests/util/grub-shell.in | 28 +++++++++++++++++++++++----- 1 file changed, 23 insertions(+), 5 deletions(-)
tests/util/grub-shell.in +23 -5
@@ -521,8 +521,11 @@ if [ x$boot = xnet ]; then cp "${cfgfile}" "$netdir/boot/grub/grub.cfg" cp "${source}" "$netdir/boot/grub/testcase.cfg" [ -z "$files" ] || copy_extra_files "$netdir" $files - setup_qemu_logger - timeout -s KILL $timeout "${qemu}" ${qemuopts} ${serial_null} -serial file:/dev/stdout -boot n -net "user,tftp=$netdir,bootfile=/boot/grub/${grub_modinfo_target_cpu}-${grub_modinfo_platform}/core.$netbootext" -net nic > "$work_directory/qemu-pipe" || ret=$? + cat >"$work_directory/run.sh" <<EOF +#! @BUILD_SHEBANG@ +qemuopts="${qemuopts}" +exec "${qemu}" \${qemuopts} ${serial_null} -serial file:/dev/stdout -boot n -net "user,tftp=$netdir,bootfile=/boot/grub/${grub_modinfo_target_cpu}-${grub_modinfo_platform}/core.$netbootext" -net nic "\$@" +EOF elif [ x$boot = xemu ]; then rootdir="$work_directory/rootdir" grubdir="$rootdir/boot/grub"
@@ -541,11 +544,25 @@ elif [ x$boot = xemu ]; then [ -z "$files" ] || copy_extra_files "$rootdir" $files roottar="$work_directory/root.tar" (cd "$rootdir"; tar cf "$roottar" .) - setup_qemu_logger - "${builddir}/grub-core/grub-emu" -m "$device_map" --memdisk "$roottar" -r memdisk -d "/boot/grub" > "$work_directory/qemu-pipe" || ret=$? + cat >"$work_directory/run.sh" <<EOF +#! @BUILD_SHEBANG@ +SDIR=\$(realpath -e \${0%/*}) +exec "$(realpath -e "${builddir}")/grub-core/grub-emu" -m "\$SDIR/${device_map##*/}" --memdisk "\$SDIR/${roottar##*/}" -r memdisk -d "/boot/grub" +EOF else + cat >"$work_directory/run.sh" <<EOF +#! @BUILD_SHEBANG@ +SDIR=\$(realpath -e \${0%/*}) +qemuopts="${qemuopts}" +cd "\$SDIR" +exec "${qemu}" \${qemuopts} ${serial_null} -serial file:/dev/stdout -${device}"\${SDIR}/${isofile##*/}" ${bootdev} "\$@" +EOF +fi + +if [ -f "$work_directory/run.sh" ]; then setup_qemu_logger - timeout -s KILL $timeout "${qemu}" ${qemuopts} ${serial_null} -serial file:/dev/stdout -${device}"${isofile}" ${bootdev} > "$work_directory/qemu-pipe" || ret=$? + chmod +x "$work_directory/run.sh" + timeout -s KILL $timeout "$work_directory/run.sh" > "$work_directory/qemu-pipe" || ret=$? fi wait
@@ -565,6 +582,7 @@ fi test -n "$debug" || rm -f "${isofile}" test -n "$debug" || rm -rf "${rom_directory}" test -n "$debug" || rm -f "${tmpfile}" "${cfgfile}" "${goutfile}" +test -n "$debug" || rm -f "$work_directory/run.sh" exit $ret
Signature · 1 line
2.34.1
PATCH 3/3 grub-shell: Add $GRUB_QEMU_OPTS to run.sh to easily see unofficial QEMU arguments
In reply to Glenn WashburnWhen re-running a failed test, even the non-standard grub-shell QEMU arguments should be preserved in the run.sh to more precisely replay the failed test run.
--- tests/util/grub-shell.in | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
tests/util/grub-shell.in +5 -3
@@ -74,7 +74,7 @@ exec_show_error () { work_directory=${WORKDIR:-`mktemp -d "${TMPDIR:-/tmp}/grub-shell.XXXXXXXXXX"`} || exit 1 . "${builddir}/grub-core/modinfo.sh" -qemuopts="${GRUB_QEMU_OPTS}" +qemuopts= serial_port=com0 serial_null= halt_cmd=halt
@@ -523,8 +523,9 @@ if [ x$boot = xnet ]; then [ -z "$files" ] || copy_extra_files "$netdir" $files cat >"$work_directory/run.sh" <<EOF #! @BUILD_SHEBANG@ +GRUB_QEMU_OPTS=\${GRUB_QEMU_OPTS:-"$GRUB_QEMU_OPTS"} qemuopts="${qemuopts}" -exec "${qemu}" \${qemuopts} ${serial_null} -serial file:/dev/stdout -boot n -net "user,tftp=$netdir,bootfile=/boot/grub/${grub_modinfo_target_cpu}-${grub_modinfo_platform}/core.$netbootext" -net nic "\$@" +exec "${qemu}" \${qemuopts} \${GRUB_QEMU_OPTS} ${serial_null} -serial file:/dev/stdout -boot n -net "user,tftp=$netdir,bootfile=/boot/grub/${grub_modinfo_target_cpu}-${grub_modinfo_platform}/core.$netbootext" -net nic "\$@" EOF elif [ x$boot = xemu ]; then rootdir="$work_directory/rootdir"
@@ -553,9 +554,10 @@ else cat >"$work_directory/run.sh" <<EOF #! @BUILD_SHEBANG@ SDIR=\$(realpath -e \${0%/*}) +GRUB_QEMU_OPTS=\${GRUB_QEMU_OPTS:-"$GRUB_QEMU_OPTS"} qemuopts="${qemuopts}" cd "\$SDIR" -exec "${qemu}" \${qemuopts} ${serial_null} -serial file:/dev/stdout -${device}"\${SDIR}/${isofile##*/}" ${bootdev} "\$@" +exec "${qemu}" \${qemuopts} \${GRUB_QEMU_OPTS} ${serial_null} -serial file:/dev/stdout -${device}"\${SDIR}/${isofile##*/}" ${bootdev} "\$@" EOF fi
Signature · 1 line
2.34.1
Quoted 20 lines · Glenn Washburn
When a non-native (eg. grub-shell/qemu) test fails it can be useful to be able to re-run the grub-shell invocation in at least two circumstances.
1. The tester wants to re-run the grub-shell invocation with extra args so that QEMU starts in GDB attach mode for using GDB to debug the test case
2. For a tester who wants to send a failed test case to another developer for additional debugging. It would be nice to be able to just zip/tar just the failed test case test output directory and allow the extracted directory to easily run the failed test case again. The results in this case still might not be the same if the sender and reciever do not have the same QEMU. Though when the QMEU versions are different, I expect the emulated execution on sender and reciever to be close enough to produce the same execution path (maybe not with super old QEMUs, but I don't think its something to be concerned about).The first patch is needed to make the other two useful. Without it, I know of no easy way to enable grub-shell's debug mode when run via
make check. And if grub-shell is not in debug mode then it will remove most/all of the generated files, which would prevent debugging.
Quoted 8 lines
Glenn
Glenn Washburn (3):
tests: Allow turning on shell tracing from environment variables
grub-shell: Createrun.shin working directory for easily running test
again
grub-shell: Add $GRUB_QEMU_OPTS torun.shto easily see unofficial
QEMU arguments
For all the patches Reviewed-by: Daniel Kiper <[email protected]>...
Daniel