Received: from eggs.gnu.org ([2001:4830:134:3::10]:40462) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZYqNL-0005ej-VA for qemu-devel@nongnu.org; Mon, 07 Sep 2015 02:58:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZYqNI-00028i-0v for qemu-devel@nongnu.org; Mon, 07 Sep 2015 02:57:59 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57260) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZYqNH-00028e-Ro for qemu-devel@nongnu.org; Mon, 07 Sep 2015 02:57:55 -0400 From: Markus Armbruster References: <878u8mb62t.fsf@blackfin.pond.sub.org> <1441410668-25265-1-git-send-email-eblake@redhat.com> <55EAEBDF.5070704@redhat.com> <55EAED45.3010301@redhat.com> <55EAF1EE.7050106@redhat.com> Date: Mon, 07 Sep 2015 08:57:53 +0200 In-Reply-To: <55EAF1EE.7050106@redhat.com> (Eric Blake's message of "Sat, 5 Sep 2015 07:45:18 -0600") Message-ID: <87k2s2vhqm.fsf@blackfin.pond.sub.org> MIME-Version: 1.0 Content-Type: text/plain Subject: Re: [Qemu-devel] [PATCH RFC v4 02.5/32] qapi: Hide internal data members of schema objects. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Eric Blake Cc: qemu-devel@nongnu.org, mdroth@linux.vnet.ibm.com Eric Blake writes: > On 09/05/2015 07:25 AM, Eric Blake wrote: >>> >>> Oh, and tests/qapi-schema/test-qapi.py is annoying - even when it exits >>> non-zero, it does NOT cause a failure in 'make check-qapi-schema'. >> >> Rather, it does not leave an obvious stack trace or immediate non-zero >> status; and 'make check-qapi-schema' only fails as a side effect if >> later output happens to differ from expectations. I would have had a >> much easier time chasing the problem if I'd had an obvious python stack >> trace. > > Oh, I see what happened. It did capture a trace in .err; but since the > code checks for differences in .out before .err, I didn't even realize > there was a trace to look at. Yes, and it has annoyed me, too. Perhaps we should do something like this (entirely untested): diff --git a/tests/Makefile b/tests/Makefile index 92d682f..d622356 100644 --- a/tests/Makefile +++ b/tests/Makefile @@ -519,13 +519,12 @@ check-tests/test-qapi.py: tests/test-qapi.py $(patsubst %, check-%, $(check-qapi-schema-y)): check-%.json: $(SRC_PATH)/%.json $(call quiet-command, PYTHONPATH=$(SRC_PATH)/scripts \ $(PYTHON) $(SRC_PATH)/tests/qapi-schema/test-qapi.py \ - $^ >$*.test.out 2>$*.test.err; \ + $^ >$*.test.out 2>$*.test.raw-err; \ echo $$? >$*.test.exit, \ " TEST $*.out") - @diff -q $(SRC_PATH)/$*.out $*.test.out @# Sanitize error messages (make them independent of build directory) - @perl -p -e 's|\Q$(SRC_PATH)\E/||g' $*.test.err | diff -q $(SRC_PATH)/$*.err - - @diff -q $(SRC_PATH)/$*.exit $*.test.exit + @perl -p -e 's|\Q$(SRC_PATH)\E/||g' $*.test.raw-err >$*.test.err + @diff -q $(SRC_PATH)/$*.out $*.test.out && diff -q $(SRC_PATH)/$*.err $*.test.err && diff -q $(SRC_PATH)/$*.exit $*.test.exit # Consolidated targets