Debugging aids
Subsystems (3)
v7 cover Debugging aids
Paolo, please apply :-)
Some aids for debugging: stack dumping, register dumping, and printing failing assertion conditions.
Compiled on ppc64, arm64, i386, x86_64.
v7:
* Rebased.
v6:
* Don't print dump_stack's call to backtrace() in the stack trace rather than having backtrace() skip over the first return address in the trace.
* Actually removed lib/asm-generic/stack.c.
v5:
* Removed unused "b" flag from run_tests.sh
* Removed misleading comment in mkstandalone.sh.
v4:
* Moved around arch and asm files per Linux style (hopefully).
* Added --[enable|disable]-pretty-print-stacks to ./configure. This has obviated the -p command-line option in ./run_tests.sh that I introduced in v1.
* Fixed arch-neutral backtrace so backtrace itself isn't included.
v3:
* Renamed functions to match standard backtrace functions
* Added asm-generic stack walker that uses __builtin_return_address.
* Moved offset out of libcflat and into pretty printer
* Made pretty printer more robust w.r.t. errors: now it just continues printing.
v2:
* Fixed a bunch tests on x86 that broke with -fno-omit-frame-pointer.
* Only use -fno-omit-frame-pointer on x86. I suspect arm & ppc will have similar problems that I fixed for x86.
Peter Feiner (5):
lib: backtrace printing
x86: lib: debug dump on unhandled exceptions
lib: dump stack on failed assert()
scripts: pretty print stack traces
scripts: automatically pretty print stacks
Makefile | 7 +-- configure | 10 +++++ lib/arm/asm/stack.h | 0 lib/arm64/asm/stack.h | 0 lib/libcflat.h | 9 +++- lib/powerpc/asm/stack.h | 0 lib/ppc64/asm/stack.h | 0 lib/stack.c | 96 ++++++++++++++++++++++++++++++++++++++++++ lib/stack.h | 20 +++++++++ lib/x86/asm/stack.h | 14 ++++++ lib/x86/desc.c | 69 +++++++++++++++++++++++++++--- lib/x86/stack.c | 31 ++++++++++++++ run_tests.sh | 8 +++- scripts/mkstandalone.sh | 2 + scripts/pretty_print_stacks.py | 89 +++++++++++++++++++++++++++++++++++++++ x86/Makefile.common | 4 ++ 16 files changed, 347 insertions(+), 12 deletions(-) create mode 100644 lib/arm/asm/stack.h create mode 100644 lib/arm64/asm/stack.h create mode 100644 lib/powerpc/asm/stack.h create mode 100644 lib/ppc64/asm/stack.h create mode 100644 lib/stack.c create mode 100644 lib/stack.h create mode 100644 lib/x86/asm/stack.h create mode 100644 lib/x86/stack.c create mode 100755 scripts/pretty_print_stacks.py
Signature · 1 line
2.8.0.rc3.226.g39d4020
PATCH v7 1/5 lib: backtrace printing
Functions to walk stack and print backtrace. The stack's unadorned as
STACK: [@]addr addr addr ...
where the optional @ indicates that addr isn't a return address.
A follow-up patch post-processes the output to pretty-print the stack.
Frame stack walker is just a stub on arm and ppc.
--- Makefile | 7 ++-- lib/arm/asm/stack.h | 0 lib/arm64/asm/stack.h | 0 lib/libcflat.h | 3 ++ lib/powerpc/asm/stack.h | 0 lib/ppc64/asm/stack.h | 0 lib/stack.c | 96 +++++++++++++++++++++++++++++++++++++++++++++++++ lib/stack.h | 20 +++++++++++ lib/x86/asm/stack.h | 14 ++++++++ lib/x86/stack.c | 31 ++++++++++++++++ x86/Makefile.common | 4 +++ 11 files changed, 172 insertions(+), 3 deletions(-) create mode 100644 lib/arm/asm/stack.h create mode 100644 lib/arm64/asm/stack.h create mode 100644 lib/powerpc/asm/stack.h create mode 100644 lib/ppc64/asm/stack.h create mode 100644 lib/stack.c create mode 100644 lib/stack.h create mode 100644 lib/x86/asm/stack.h create mode 100644 lib/x86/stack.c
Makefile +4 -3
@@ -22,7 +22,8 @@ cflatobjs := \ lib/printf.o \ lib/string.o \ lib/abort.o \ - lib/report.o + lib/report.o \ + lib/stack.o # libfdt paths LIBFDT_objdir = lib/libfdt
@@ -42,8 +43,8 @@ cc-option = $(shell if $(CC) $(1) -S -o /dev/null -xc /dev/null \ CFLAGS += -g CFLAGS += $(autodepend-flags) -Wall -Werror - -fomit_frame_pointer := $(call cc-option, -fomit-frame-pointer, "") +frame-pointer-flag=-f$(if $(KEEP_FRAME_POINTER),no-,)omit-frame-pointer +fomit_frame_pointer := $(call cc-option, $(frame-pointer-flag), "") fnostack_protector := $(call cc-option, -fno-stack-protector, "") fnostack_protector_all := $(call cc-option, -fno-stack-protector-all, "") CFLAGS += $(fomit_frame_pointer)
lib/arm/asm/stack.h new file +0 -0
lib/arm64/asm/stack.h new file +0 -0
lib/libcflat.h +3 -0
@@ -81,6 +81,9 @@ extern void report_xfail(const char *msg_fmt, bool xfail, bool pass, ...);extern void report_abort (const char *msg_fmt, ...);extern int report_summary (void ); +extern void dump_stack (void ); +extern void dump_frame_stack (const void *instruction,const void *frame); +#define ARRAY_SIZE (_a) (sizeof(_a)/sizeof((_a)[0]))#define container_of (ptr, type, member) ({ \
lib/powerpc/asm/stack.h new file +0 -0
lib/ppc64/asm/stack.h new file +0 -0
lib/stack.c new file +96 -0
96 lines not on this page · raw message
lib/stack.h new file +20 -0
@@ -0,0 +1,20 @@ +#ifndef _STACK_H_ +#define _STACK_H_ + +#include <libcflat.h> +#include <asm/stack.h> + +#ifndef HAVE_ARCH_BACKTRACE_FRAME +static inline int +backtrace_frame(const void *frame __unused, const void **return_addrs __unused, + int max_depth __unused) +{ +return 0 ; +} +#endif + +#ifndef HAVE_ARCH_BACKTRACE +int backtrace (const void **return_addrs,int max_depth); +#endif + +#endif
lib/x86/asm/stack.h new file +14 -0
@@ -0,0 +1,14 @@ +#ifndef _X86ASM_STACK_H_ +#define _X86ASM_STACK_H_ + +#ifndef _STACK_H_ +#error Do not directly include <asm/stack.h>. Just use <stack.h>. +#endif + +#define HAVE_ARCH_BACKTRACE_FRAME +int backtrace_frame (const void *frame,const void **return_addrs,int max_depth); + +#define HAVE_ARCH_BACKTRACE +int backtrace (const void **return_addrs,int max_depth); + +#endif
lib/x86/stack.c new file +31 -0
@@ -0,0 +1,31 @@ +#include <libcflat.h> +#include <stack.h> + +int backtrace_frame (const void *frame,const void **return_addrs,int max_depth) +{ +static int walking; +int depth =0 ; +const unsigned long *bp = (unsigned long *) frame; + +if (walking) { +printf ("RECURSIVE STACK WALK!!!\n" ); +return 0 ; + } + walking =1 ; + +for (depth =0 ; depth < max_depth; depth++) { + return_addrs[depth] = (void *) bp[1 ]; +if (return_addrs[depth] ==0 ) +break ; + bp = (unsigned long *) bp[0 ]; + } + + walking =0 ; +return depth; +} + +int backtrace (const void **return_addrs,int max_depth) +{ +return backtrace_frame (__builtin_frame_address (0 ), return_addrs, + max_depth); +}
x86/Makefile.common +4 -0
@@ -12,6 +12,7 @@ cflatobjs += lib/x86/atomic.o cflatobjs += lib/x86/desc.o cflatobjs += lib/x86/isr.o cflatobjs += lib/x86/acpi.o +cflatobjs += lib/x86/stack.o $(libcflat): LDFLAGS += -nostdlib $(libcflat): CFLAGS += -ffreestanding -I lib
@@ -19,6 +20,9 @@ $(libcflat): CFLAGS += -ffreestanding -I lib CFLAGS += -m$(bits) CFLAGS += -O1 +# dump_stack.o relies on frame pointers. +KEEP_FRAME_POINTER := y + libgcc := $(shell $(CC) -m$(bits) --print-libgcc-file-name) FLATLIBS = lib/libcflat.a $(libgcc)
Signature · 1 line
2.8.0.rc3.226.g39d4020
PATCH v7 2/5 x86: lib: debug dump on unhandled exceptions
In reply to Peter FeinerPrints the stack trace and registers.
--- lib/x86/desc.c | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 63 insertions(+), 6 deletions(-)
lib/x86/desc.c +63 -6
@@ -38,6 +38,67 @@ struct ex_record {extern struct ex_record exception_table_start, exception_table_end; +static const char *exception_mnemonic (int vector) +{ +switch (vector) { +case 0 :return "#DE" ; +case 1 :return "#DB" ; +case 2 :return "#NMI" ; +case 3 :return "#BP" ; +case 4 :return "#OF" ; +case 5 :return "#BR" ; +case 6 :return "#UD" ; +case 7 :return "#NM" ; +case 8 :return "#DF" ; +case 10 :return "#TS" ; +case 11 :return "#NP" ; +case 12 :return "#SS" ; +case 13 :return "#GP" ; +case 14 :return "#PF" ; +case 16 :return "#MF" ; +case 17 :return "#AC" ; +case 18 :return "#MC" ; +case 19 :return "#XM" ; +default :return "#??" ; + } +} + +static void unhandled_exception (struct ex_regs *regs,bool cpu) +{ +printf ("Unhandled %sexception %ld %s at ip %016lx\n" , + cpu ?"cpu " :"" , regs->vector, +exception_mnemonic (regs->vector), regs->rip); +if (regs->vector ==14 ) +printf ("PF at 0x%lx addr 0x%lx\n" , regs->rip,read_cr2 ()); + + printf("error_code=%04lx rflags=%08lx cs=%08lx\n" + "rax=%016lx rcx=%016lx rdx=%016lx rbx=%016lx\n" + "rbp=%016lx rsi=%016lx rdi=%016lx\n" +#ifdef __x86_64__ + " r8=%016lx r9=%016lx r10=%016lx r11=%016lx\n" + "r12=%016lx r13=%016lx r14=%016lx r15=%016lx\n" +#endif +"cr0=%016lx cr2=%016lx cr3=%016lx cr4=%016lx\n" +#ifdef __x86_64__ +"cr8=%016lx\n" +#endif + , + regs->error_code, regs->rflags, regs->cs, + regs->rax, regs->rcx, regs->rdx, regs->rbx, + regs->rbp, regs->rsi, regs->rdi, +#ifdef __x86_64__ + regs->r8, regs->r9, regs->r10, regs->r11, + regs->r12, regs->r13, regs->r14, regs->r15, +#endif +read_cr0 (),read_cr2 (),read_cr3 (),read_cr4 () +#ifdef __x86_64__ + ,read_cr8 () +#endif + ); +dump_frame_stack ((void *) regs->rip, (void *) regs->rbp); +exit (7 ); +} +static void check_exception_table (struct ex_regs *regs) {struct ex_record *ex;
@@ -53,8 +114,7 @@ static void check_exception_table(struct ex_regs *regs)return ; } } -printf ("unhandled exception %lu\n" , regs->vector); -abort (); +unhandled_exception (regs, false); }static void (*exception_handlers[32 ])(struct ex_regs *regs);
@@ -75,10 +135,7 @@ void do_handle_exception(struct ex_regs *regs) exception_handlers[regs->vector](regs);return ; } -printf ("unhandled cpu exception %lu\n" , regs->vector); -if (regs->vector ==14 ) -printf ("PF at 0x%lx addr 0x%lx\n" , regs->rip,read_cr2 ()); -abort (); +unhandled_exception (regs, true); }#define EX (NAME ,N ) extern char NAME##_fault; \
Signature · 1 line
2.8.0.rc3.226.g39d4020
PATCH v7 3/5 lib: dump stack on failed assert()
In reply to Peter Feiner--- lib/libcflat.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
lib/libcflat.h +4 -2
@@ -92,10 +92,12 @@ extern void dump_frame_stack(const void *instruction, const void *frame);#define assert (cond) \ do { \ - if (!(cond)) \ + if (!(cond)) { \ printf("%s:%d: assert failed: %s\n", \ - __FILE__, __LINE__, #cond), \ + __FILE__, __LINE__, #cond); \ + dump_stack(); \ abort(); \ + } \ } while (0)#endif
Signature · 1 line
2.8.0.rc3.226.g39d4020
PATCH v7 4/5 scripts: pretty print stack traces
In reply to Peter FeinerThe script runs addresses through addr2line to get file names and line numbers. Also prints out source code lines if files are available. For example
0x004013f5: ac_test_exec at x86/access.c:818
}
> ac_test_setup_pte(at, pool);
r = ac_test_do_access(at);
0x004014dd: ac_test_run at x86/access.c:852
++tests;
> successes += ac_test_exec(&at, &pool);
} while (ac_test_bump(&at));
0x0040157d: main at x86/access.c:873
printf("starting test\n\n");
> r = ac_test_run();
return r ? 0 : 1;0x0040028e: start64 at x86/cstart64.S:206
lea __argv(%rip), %rsi
> call main
mov %eax, %edi
--- scripts/pretty_print_stacks.py | 89 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) create mode 100755 scripts/pretty_print_stacks.py
scripts/pretty_print_stacks.py new file +89 -0
@@ -0,0 +1,89 @@ +#!/usr/bin/env python + +import re +import subprocess +import sys +import traceback + +# Subvert output buffering. +def puts(string): + sys.stdout.write(string) + sys.stdout.flush() + +def pretty_print_stack(binary, line): + addrs = line.split()[1:] + # Addresses are return addresses unless preceded by a '@'. We want the + # caller address so line numbers are more intuitive. Thus we subtract 1 + # from the address to get the call code. + for i in range(len(addrs)): + addr = addrs[i] + if addr.startswith('@'): + addrs[i] = addr[1:] + else: + addrs[i] = '%lx' % (int(addrs[i], 16) - 1) + + # Output like this: + # 0x004002be: start64 at path/to/kvm-unit-tests/x86/cstart64.S:208 + # (inlined by) test_ept_violation at path/to/kvm-unit-tests/x86/vmx_tests.c:1719 (discriminator 1) + cmd = ['addr2line', '-e', binary, '-i', '-f', '--pretty', '--address'] + cmd.extend(addrs) + + p = subprocess.Popen(cmd, stdout=subprocess.PIPE) + out, err = p.communicate() + if p.returncode != 0: + puts(line) + return + + for line in out.splitlines(): + m = re.match('(.*) at [^ ]*/kvm-unit-tests/([^ ]*):([0-9]+)(.*)', line) + if m is None: + puts('%s\n' % line) + return + + head, path, line, tail = m.groups() + line = int(line) + puts('%s at %s:%d%s\n' % (head, path, line, tail)) + try: + lines = open(path).readlines() + except IOError: + continue + if line > 1: + puts(' %s\n' % lines[line - 2].rstrip()) + puts(' > %s\n' % lines[line - 1].rstrip()) + if line < len(lines): + puts(' %s\n' % lines[line].rstrip()) + +def main(): + if len(sys.argv) != 2: + sys.stderr.write('usage: %s <kernel>\n' % sys.argv[0]) + sys.exit(1) + + binary = sys.argv[1] + + try: + while True: + # Subvert input buffering. + line = sys.stdin.readline() + if line == '': + break + + if not line.strip().startswith('STACK:'): + puts(line) + continue + + try: + pretty_print_stack(binary, line) + except Exception: + puts('Error pretty printing stack:\n') + puts(traceback.format_exc()) + puts('Continuing without pretty printing...\n') + while True: + puts(line) + line = sys.stdin.readline() + if line == '': + break + except: + sys.exit(1) + +if __name__ == '__main__': + main()
Signature · 1 line
2.8.0.rc3.226.g39d4020
PATCH v7 5/5 scripts: automatically pretty print stacks
In reply to Peter FeinerNow run_tests.sh automatically pipes test output through scripts/pretty_print_stacks.py. Can be disabled with ./configure --disable-pretty-print-stacks.
Pretty printing disabled on standalone tests because there's no binary.
--- configure | 10 ++++++++++ run_tests.sh | 8 +++++++- scripts/mkstandalone.sh | 2 ++ 3 files changed, 19 insertions(+), 1 deletion(-)
configure +10 -0
@@ -11,6 +11,7 @@ arch=`uname -m | sed -e 's/i.86/i386/;s/arm.*/arm/;s/ppc64.*/ppc64/'` host=$arch cross_prefix= endian="" +pretty_print_stacks=yes usage() { cat <<-EOF
@@ -25,6 +26,8 @@ usage() { --prefix=PREFIX where to install things ($prefix) --kerneldir=DIR kernel build directory for kvm.h ($kerneldir) --endian=ENDIAN endianness to compile for (little or big, ppc64 only) + --[enable|disable]-pretty-print-stacks + enable or disable pretty stack printing (enabled by default) EOF exit 1 }
@@ -61,6 +64,12 @@ while [[ "$1" = -* ]]; do --ld) ld="$arg" ;; + --enable-pretty-print-stacks) + pretty_print_stacks=yes + ;; + --disable-pretty-print-stacks) + pretty_print_stacks=no + ;; --help) usage ;;
@@ -149,4 +158,5 @@ API=$api TEST_DIR=$testdir FIRMWARE=$firmware ENDIAN=$endian +PRETTY_PRINT_STACKS=$pretty_print_stacks EOF
run_tests.sh +7 -1
@@ -46,7 +46,13 @@ while getopts "g:hv" opt; do esac done -RUNTIME_arch_run="./$TEST_DIR/run >> test.log" +if [ "$PRETTY_PRINT_STACKS" = "yes" ]; then + log_redir="> >(./scripts/pretty_print_stacks.py \$kernel >> test.log)" +else + log_redir=">> test.log" +fi + +RUNTIME_arch_run="./$TEST_DIR/run $log_redir" config=$TEST_DIR/unittests.cfg rm -f test.log printf "BUILD_HEAD=$(cat build-head)\n\n" > test.log
scripts/mkstandalone.sh +2 -0
@@ -41,6 +41,8 @@ generate_test () echo "#!/bin/bash" echo "export STANDALONE=yes" echo "export HOST=\$(uname -m | sed -e s/i.86/i386/ | sed -e 's/arm.*/arm/')" + echo "export PRETTY_PRINT_STACKS=no" + config_export ARCH config_export ARCH_NAME config_export PROCESSOR
Signature · 1 line
2.8.0.rc3.226.g39d4020
Paolo Bonzini
maintainerRe: [PATCH kvm-unit-tests v7 5/5] scripts: automatically pretty print stacks
run_tests.sh while getopts "g:hv" opt; do · line 50
Quoted 2 lines · Peter Feiner
run_tests.sh +2
-0+if [ "$PRETTY_PRINT_STACKS" = "yes" ]; then + log_redir="> >(./scripts/pretty_print_stacks.py \$kernel >> test.log)"
Isn't this the same as "| ./scripts/pretty_print_stack.py \$kernel >> test.log"?
If so, perhaps
if [ "$PRETTY_PRINT_STACKS" = "yes" ]; then
stack_filter="./scripts/pretty_print_stack.py \$kernel"
else
stack_filter=cat
RUNTIME_arch_run="./$TEST_DIR/run | $stack_filter >> test.log"
... would be a bit easier to read.
Paolo
Paolo Bonzini
maintainerQuoted 6 lines · Peter Feiner
Paolo, please apply :-)
Some aids for debugging: stack dumping, register dumping, and printing failing assertion conditions.
Compiled on ppc64, arm64, i386,
x86_64.
Applied patches 1-4.
Paolo
Quoted 68 lines
v7:
* Rebased.v6:
* Don't print dump_stack's call tobacktrace()in the stack trace rather than havingbacktrace()skip over the first return address in the trace.
* Actually removedlib/asm-generic/stack.c.v5:
* Removed unused "b" flag fromrun_tests.sh
* Removed misleading comment inmkstandalone.sh.v4:
* Moved around arch and asm files per Linux style (hopefully).
* Added --[enable|disable]-pretty-print-stacks to ./configure. This has obviated the -p command-line option in ./run_tests.sh that I introduced in v1.
* Fixed arch-neutral backtrace so backtrace itself isn't included.v3:
* Renamed functions to match standard backtrace functions
* Added asm-generic stack walker that uses__builtin_return_address.
* Moved offset out of libcflat and into pretty printer
* Made pretty printer more robust w.r.t. errors: now it just continues printing.v2:
* Fixed a bunch tests on x86 that broke with -fno-omit-frame-pointer.
* Only use -fno-omit-frame-pointer on x86. I suspect arm & ppc will have similar problems that I fixed for x86.Peter Feiner (5):
lib: backtrace printing
x86: lib: debug dump on unhandled exceptions
lib: dump stack on failedassert()
scripts: pretty print stack traces
scripts: automatically pretty print stacksMakefile | 7 +-- configure | 10 +++++ lib/arm/asm/stack.h | 0 lib/arm64/asm/stack.h | 0 lib/libcflat.h | 9 +++- lib/powerpc/asm/stack.h | 0 lib/ppc64/asm/stack.h | 0 lib/stack.c | 96 ++++++++++++++++++++++++++++++++++++++++++ lib/stack.h | 20 +++++++++ lib/x86/asm/stack.h | 14 ++++++ lib/x86/desc.c | 69 +++++++++++++++++++++++++++--- lib/x86/stack.c | 31 ++++++++++++++ run_tests.sh | 8 +++- scripts/mkstandalone.sh | 2 + scripts/pretty_print_stacks.py | 89 +++++++++++++++++++++++++++++++++++++++ x86/Makefile.common | 4 ++ 16 files changed, 347 insertions(+), 12 deletions(-) create mode 100644 lib/arm/asm/stack.h create mode 100644 lib/arm64/asm/stack.h create mode 100644 lib/powerpc/asm/stack.h create mode 100644 lib/ppc64/asm/stack.h create mode 100644 lib/stack.c create mode 100644 lib/stack.h create mode 100644 lib/x86/asm/stack.h create mode 100644 lib/x86/stack.c create mode 100755 scripts/pretty_print_stacks.py
Re: [PATCH kvm-unit-tests v7 5/5] scripts: automatically pretty print stacks
In reply to Paolo BonziniQuoted 8 lines · Paolo Bonzini
run_tests.sh while getopts "g:hv" opt; do · line 50
Quoted 2 lines · Peter Feiner
run_tests.sh +2
-0+if [ "$PRETTY_PRINT_STACKS" = "yes" ]; then + log_redir="> >(./scripts/pretty_print_stacks.py \$kernel >> test.log)"Isn't this the same as "|
./scripts/pretty_print_stack.py\$kernel >> test.log"?
Not the same. Consider scripts/runtime.bash's usage:
cmdline="...$RUNTIME_arch_run $kernel -smp $smp $opts"
With your alternative, the command-line arguments after $RUNTIME_arch_run are be passed to pretty_print_stacks.py instead of Qemu.
Quoted 10 lines
If so, perhaps
if [ "$PRETTY_PRINT_STACKS" = "yes" ]; then
stack_filter="./scripts/pretty_print_stack.py \$kernel"
else
stack_filter=catRUNTIME_arch_run="./$TEST_DIR/run | $stack_filter >> test.log"
... would be a bit easier to read.
Yes, it's super ugly, but it works!
Peter
Paolo Bonzini
maintainerRe: [PATCH kvm-unit-tests v7 5/5] scripts: automatically pretty print stacks
Quoted 7 lines · Peter Feiner
Quoted 7 lines
Quoted 7 lines · Peter Feiner
Quoted 2 lines
run_tests.sh while getopts "g:hv" opt; do · line 50
Quoted 2 lines
run_tests.sh +2
-0+if [ "$PRETTY_PRINT_STACKS" = "yes" ]; then + log_redir="> >(./scripts/pretty_print_stacks.py \$kernel >> test.log)"Isn't this the same as "|
./scripts/pretty_print_stack.py\$kernel >> test.log"?
Quoted 6 lines
Not the same. Consider scripts/runtime.bash's usage:
cmdline="...$RUNTIME_arch_run $kernel -smp $smp $opts"
With your alternative, the command-line arguments after $RUNTIME_arch_run are be passed to
pretty_print_stacks.pyinstead of Qemu.
Ok, I'll add a comment then.
Paolo
Quoted 6 lines · Peter Feiner
Paolo, please apply :-)
Some aids for debugging: stack dumping, register dumping, and printing failing assertion conditions.
Compiled on ppc64, arm64, i386,
x86_64.
Applied, thanks.
Paolo