Modularize generated QAPI code
Subsystems (4)
cover Modularize generated QAPI code
Our qapi-schema.json is composed of modules connected by include directives, but the generated code is monolithic all the same: one qapi-types.h with all the types, one qapi-visit.h with all the visitors, and so forth. These monolithic headers get included all over the place. In my "build everyhing" tree, adding a QAPI type recompiles about 4500 out of 4800 objects.
Nobody would write such monolithic headers by hand. It stands to reason that one shouldn't generate them, either.
This series' basic idea is to split up generated headers to mirror the schema's modular structure: one header per module. That way, you can include just what you need.
The series is RFC for a number of reasons:
* The split is implemented only for qapi-types.h. That one should provide the biggest benefits, though.
* There's a bit of code duplication.
* I haven't re-read my patches, yet.
Even in this incomplete state, the compile-time improvements can be massive. Before this series, any QAPI schema change recompiles some 4500 out of 4800 objects in my "build everything" tree. Afterwards, adding a type to qapi/migration.json recompiles less than 400, adding a QMP event recompiles less than 200, and a documentation change no longer recompiles anything.
Related: Marc-André's 'unit' pragma proposal. That's a different way to split off parts of the generated code, motivated by the desire to use poisoned identifiers such as TARGET_I386. I noted in my review of v3 that I "can either accept it, or come up with a better solution." This is my attempt at a better solution. It's a bit more ambitious, and thus more useful (I hope). The pragma has one theoretical advantage, though: you can modularize the generated output in different ways than the input. The patches using don't do that, however.
Based-on: <[email protected]> [PATCH v3 00/19] Clean up includes to reduce compile time
Markus Armbruster (21):
qapi: Streamline boilerplate comment generation
qapi: Generate up-to-date copyright notice
qapi: New classes QAPIGenC, QAPIGenH, QAPIGenDoc
qapi: Reduce use of global variables in generators some
qapi: Turn generators into modules
qapi-gen: New common driver for code and doc generators
qapi: Move parse_command_line() next to its only use
qapi: Touch generated files only when they change
qapi: Don't absolutize include file name in error messages qapi/common: Eliminate QAPISchema.exprs
qapi: Lift error reporting from QAPISchema.__init__() to callers
qapi: Concentrate QAPISchemaParser.exprs updates in .__init__()
qapi: Record 'include' directives in parse tree
qapi: Generate in source order
qapi: Record 'include' directives in intermediate representation qapi/types qapi/visit: Make visitors use QAPIGen more qapi/types qapi/visit: Generate built-in stuff into separate files qapi/common: Fix guardname() for funny filenames qapi/types: Generate separate .h, .c for each module Include less of qapi-types.h
qapi: Empty out qapi-schema.json
Makefile | 131 +-
Makefile.objs | 20 +-
crypto/cipherpriv.h | 2 +-
hw/block/block.c | 1 +
hw/block/hd-geometry.c | 1 +
hw/net/rocker/rocker_fp.c | 2 +-
include/block/block.h | 2 +-
include/block/dirty-bitmap.h | 2 +-
include/chardev/char.h | 1 +
include/crypto/cipher.h | 2 +-
include/crypto/hash.h | 2 +-
include/crypto/hmac.h | 2 +-
include/crypto/secret.h | 1 +
include/crypto/tlscreds.h | 1 +
include/hw/block/block.h | 2 +-
include/hw/block/fdc.h | 2 +-
include/hw/ppc/spapr_drc.h | 1 +
include/hw/qdev-properties.h | 2 +
include/io/dns-resolver.h | 1 +
include/migration/colo.h | 2 +-
include/migration/failover.h | 2 +-
include/migration/global_state.h | 1 +
include/monitor/monitor.h | 1 +
include/net/filter.h | 1 +
include/net/net.h | 2 +-
include/qapi/error.h | 2 +-
include/qapi/qmp/qobject.h | 2 +-
include/qapi/visitor.h | 2 +-
include/qemu/sockets.h | 2 +-
include/qemu/throttle.h | 2 +-
include/qom/cpu.h | 1 +
include/qom/object.h | 2 +-
include/sysemu/arch_init.h | 2 +-
include/sysemu/balloon.h | 2 +-
include/sysemu/dump.h | 2 +
include/sysemu/hostmem.h | 1 +
include/sysemu/replay.h | 2 +
include/sysemu/sysemu.h | 1 +
include/sysemu/tpm.h | 1 +
include/sysemu/watchdog.h | 2 +-
include/ui/input.h | 2 +-
migration/migration.h | 1 +
migration/ram.h | 2 +-
net/tap_int.h | 2 +-
qapi-schema.json | 3098 +-------------------
qapi/misc.json | 3090 +++++++++++++++++++
qapi/run-state.json | 10 +
replication.h | 1 +
scripts/qapi-gen.py | 95 +
scripts/qapi/__init__.py | 0
scripts/{qapi-commands.py => qapi/commands.py} | 98 +-
scripts/{qapi.py => qapi/common.py} | 290 +-
scripts/{qapi2texi.py => qapi/doc.py} | 28 +-
scripts/{qapi-event.py => qapi/events.py} | 101 +-
scripts/{qapi-introspect.py => qapi/introspect.py} | 87 +-
scripts/{qapi-types.py => qapi/types.py} | 218 +-
scripts/{qapi-visit.py => qapi/visit.py} | 208 +-
tests/Makefile.include | 56 +-
tests/qapi-schema/comments.out | 3 +-
tests/qapi-schema/doc-bad-section.out | 5 +-
tests/qapi-schema/doc-good.out | 33 +-
tests/qapi-schema/doc-good.texi | 3 +-
tests/qapi-schema/empty.out | 2 +-
tests/qapi-schema/event-case.out | 3 +-
tests/qapi-schema/ident-with-escape.out | 7 +-
tests/qapi-schema/include-no-file.err | 2 +-
tests/qapi-schema/include-relpath.out | 7 +-
tests/qapi-schema/include-repetition.out | 12 +-
tests/qapi-schema/include-simple.out | 5 +-
tests/qapi-schema/indented-expr.out | 3 +-
tests/qapi-schema/qapi-schema-test.out | 321 +-
tests/qapi-schema/test-qapi.py | 17 +-
ui/vnc.h | 1 +
73 files changed, 4040 insertions(+), 3984 deletions(-)
create mode 100644 qapi/misc.json
create mode 100755 scripts/qapi-gen.py
create mode 100644 scripts/qapi/__init__.py
rename scripts/{qapi-commands.py => qapi/commands.py} (78%)
rename scripts/{qapi.py => qapi/common.py} (93%)
rename scripts/{qapi2texi.py => qapi/doc.py} (93%)
mode change 100755 => 100644
rename scripts/{qapi-event.py => qapi/events.py} (69%)
rename scripts/{qapi-introspect.py => qapi/introspect.py} (78%)
rename scripts/{qapi-types.py => qapi/types.py} (54%)
rename scripts/{qapi-visit.py => qapi/visit.py} (66%)Signature · 1 line
2.13.6
Quoted 14 lines · Markus Armbruster
Our qapi-schema.json is composed of modules connected by include directives, but the generated code is monolithic all the same: one
qapi-types.hwith all the types, oneqapi-visit.hwith all the visitors, and so forth. These monolithic headers get included all over the place. In my "build everyhing" tree, adding a QAPI type recompiles about 4500 out of 4800 objects.Nobody would write such monolithic headers by hand. It stands to reason that one shouldn't generate them, either.
This series' basic idea is to split up generated headers to mirror the schema's modular structure: one header per module. That way, you can include just what you need.
Quoted 8 lines
The series is RFC for a number of reasons:
* The split is implemented only for
qapi-types.h. That one should provide the biggest benefits, though.* There's a bit of code duplication.
* I haven't re-read my patches, yet.
And of course
* Needs a doc update
[...]
RFC PATCH 1/21 qapi: Streamline boilerplate comment generation
In reply to Markus ArmbrusterEvery generator has separate boilerplate for .h and .c, and their differences are boring. All of them repeat the license note.
Reduce the repetition as follows. Move common text like the license note to common open_output(), next to the existintg common text there. For each generator, replace the two separate descriptions by a single one.
While there, emit an "automatically generated" note into generated documentation, too.
--- scripts/qapi-commands.py | 26 +++----------------------- scripts/qapi-event.py | 26 +++----------------------- scripts/qapi-introspect.py | 21 ++------------------- scripts/qapi-types.py | 26 +++----------------------- scripts/qapi-visit.py | 26 +++----------------------- scripts/qapi.py | 31 ++++++++++++++++++------------- scripts/qapi2texi.py | 3 ++- tests/qapi-schema/doc-good.texi | 3 ++- 8 files changed, 36 insertions(+), 126 deletions(-)
scripts/qapi-commands.py +3 -23
@@ -255,38 +255,18 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): (input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line() -c_comment = ''' -/* - * schema-defined QMP->QAPI command dispatch +blurb = ''' + * Schema-defined QAPI/QMP commands * * Copyright IBM, Corp. 2011 * * Authors: * Anthony Liguori <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ -''' -h_comment = ''' -/* - * schema-defined QAPI function prototypes - * - * Copyright IBM, Corp. 2011 - * - * Authors: - * Anthony Liguori <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qmp-marshal.c', 'qmp-commands.h', - c_comment, h_comment) + blurb) fdef.write(mcgen('''
scripts/qapi-event.py +3 -23
@@ -171,38 +171,18 @@ class QAPISchemaGenEventVisitor(QAPISchemaVisitor): (input_file, output_dir, do_c, do_h, prefix, dummy) = parse_command_line() -c_comment = ''' -/* - * schema-defined QAPI event functions +blurb = ''' + * Schema-defined QAPI/QMP events * * Copyright (c) 2014 Wenchao Xia * * Authors: * Wenchao Xia <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ -''' -h_comment = ''' -/* - * schema-defined QAPI event functions - * - * Copyright (c) 2014 Wenchao Xia - * - * Authors: - * Wenchao Xia <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qapi-event.c', 'qapi-event.h', - c_comment, h_comment) + blurb) fdef.write(mcgen(''' #include "qemu/osdep.h"
scripts/qapi-introspect.py +2 -19
@@ -176,32 +176,15 @@ for o, a in opts: if o in ('-u', '--unmask-non-abi-names'): opt_unmask = True -c_comment = ''' -/* +blurb = ''' * QAPI/QMP schema introspection * * Copyright (C) 2015 Red Hat, Inc. - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ -''' -h_comment = ''' -/* - * QAPI/QMP schema introspection - * - * Copyright (C) 2015 Red Hat, Inc. - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qmp-introspect.c', 'qmp-introspect.h', - c_comment, h_comment) + blurb) fdef.write(mcgen(''' #include "qemu/osdep.h"
scripts/qapi-types.py +3 -23
@@ -250,39 +250,19 @@ for o, a in opts: if o in ('-b', '--builtins'): do_builtins = True -c_comment = ''' -/* - * deallocation functions for schema-defined QAPI types +blurb = ''' + * Schema-defined QAPI types * * Copyright IBM, Corp. 2011 * * Authors: * Anthony Liguori <[email protected]> * Michael Roth <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ -''' -h_comment = ''' -/* - * schema-defined QAPI types - * - * Copyright IBM, Corp. 2011 - * - * Authors: - * Anthony Liguori <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qapi-types.c', 'qapi-types.h', - c_comment, h_comment) + blurb) fdef.write(mcgen(''' #include "qemu/osdep.h"
scripts/qapi-visit.py +3 -23
@@ -334,38 +334,18 @@ for o, a in opts: if o in ('-b', '--builtins'): do_builtins = True -c_comment = ''' -/* - * schema-defined QAPI visitor functions +blurb = ''' + * Schema-defined QAPI visitors * * Copyright IBM, Corp. 2011 * * Authors: * Anthony Liguori <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ -''' -h_comment = ''' -/* - * schema-defined QAPI visitor functions - * - * Copyright IBM, Corp. 2011 - * - * Authors: - * Anthony Liguori <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qapi-visit.c', 'qapi-visit.h', - c_comment, h_comment) + blurb) fdef.write(mcgen(''' #include "qemu/osdep.h"
scripts/qapi.py +18 -13
@@ -1975,11 +1975,21 @@ def parse_command_line(extra_options='', extra_long_options=[]): # -def open_output(output_dir, do_c, do_h, prefix, c_file, h_file, - c_comment, h_comment): +def open_output(output_dir, do_c, do_h, prefix, c_file, h_file, blurb): guard = guardname(prefix + h_file) c_file = output_dir + prefix + c_file h_file = output_dir + prefix + h_file + comment = mcgen('''/* AUTOMATICALLY GENERATED, DO NOT MODIFY */ + +/* +%(blurb)s + * + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. + * See the COPYING.LIB file in the top-level directory. + */ + +''', + blurb=blurb.strip('\n')) if output_dir: try:
@@ -1998,27 +2008,22 @@ def open_output(output_dir, do_c, do_h, prefix, c_file, h_file, fdef = maybe_open(do_c, c_file, 'w') fdecl = maybe_open(do_h, h_file, 'w') - fdef.write(mcgen(''' -/* AUTOMATICALLY GENERATED, DO NOT MODIFY */ -%(comment)s -''', - comment=c_comment)) - + fdef.write(comment) + fdecl.write(comment) fdecl.write(mcgen(''' -/* AUTOMATICALLY GENERATED, DO NOT MODIFY */ -%(comment)s #ifndef %(guard)s #define %(guard)s ''', - comment=h_comment, guard=guard)) + guard=guard)) return (fdef, fdecl) def close_output(fdef, fdecl): - fdecl.write(''' + fdecl.write(mcgen(''' + #endif -''') +''')) fdecl.close() fdef.close()
scripts/qapi2texi.py +2 -1
@@ -282,7 +282,8 @@ def main(argv): print >>sys.stderr, ("%s: need pragma 'doc-required' " "to generate documentation" % argv[0]) sys.exit(1) - print texi_schema(schema) + print '@c AUTOMATICALLY GENERATED, DO NOT MODIFY\n' + print texi_schema(schema), if __name__ == '__main__':
tests/qapi-schema/doc-good.texi +2 -1
@@ -1,3 +1,5 @@ +@c AUTOMATICALLY GENERATED, DO NOT MODIFY + @section Section @subsection Subsection
@@ -231,4 +233,3 @@ If you're bored enough to read this, go see a video of boxed cats @end deftypefn -
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 01/21] qapi: Streamline boilerplate comment generation
Quoted 5 lines · Markus Armbruster
Every generator has separate boilerplate for .h and .c, and their differences are boring. All of them repeat the license note.
Reduce the repetition as follows. Move common text like the license note to common
open_output(), next to the existintg common text there.
s/existintg/existing/
Quoted 11 lines
For each generator, replace the two separate descriptions by a single one.
While there, emit an "automatically generated" note into generated documentation, too.
Signed-off-by Markus Armbruster <[email protected]>---
+++
b/scripts/qapi2texi.py
scripts/qapi2texi.py def main(argv): · line 286
Quoted 6 lines
scripts/qapi2texi.py +2
-1@@ -282,7 +282,8 @@ def main(argv): print >>sys.stderr, ("%s: need pragma 'doc-required' " "to generate documentation" % argv[0]) sys.exit(1) - print texi_schema(schema) + print '@c AUTOMATICALLY GENERATED, DO NOT MODIFY\n' + print texi_schema(schema),
Spurious addition of trailing comma. Otherwise,
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 01/21] qapi: Streamline boilerplate comment generation
Quoted 8 lines · Eric Blake
Quoted 5 lines · Markus Armbruster
Every generator has separate boilerplate for .h and .c, and their differences are boring. All of them repeat the license note.
Reduce the repetition as follows. Move common text like the license note to common
open_output(), next to the existintg common text there.s/existintg/existing/
Fixing...
Quoted 9 lines
Quoted 8 lines
For each generator, replace the two separate descriptions by a single one.
While there, emit an "automatically generated" note into generated documentation, too.
Signed-off-by Markus Armbruster <[email protected]>---
Quoted 10 lines
scripts/qapi2texi.py def main(argv): · line 286
Quoted 8 lines
+++
b/scripts/qapi2texi.pyscripts/qapi2texi.py +2
-1@@ -282,7 +282,8 @@ def main(argv): print >>sys.stderr, ("%s: need pragma 'doc-required' " "to generate documentation" % argv[0]) sys.exit(1) - print texi_schema(schema) + print '@c AUTOMATICALLY GENERATED, DO NOT MODIFY\n' + print texi_schema(schema),Spurious addition of trailing comma. Otherwise,
This is Python's (rather unobvious) way to print without adding a newline.
Quoted 1 line
Reviewed-by Eric Blake <[email protected]>
Thanks!
Re: [Qemu-devel] [PATCH RFC 01/21] qapi: Streamline boilerplate comment generation
In reply to Markus ArmbrusterQuoted 8 lines · Markus Armbruster
Every generator has separate boilerplate for .h and .c, and their differences are boring. All of them repeat the license note.
Reduce the repetition as follows. Move common text like the license note to common
open_output(), next to the existintg common text there. For each generator, replace the two separate descriptions by a single one.
Quoted 4 lines
While there, emit an "automatically generated" note into generated documentation, too.
Signed-off-by Markus Armbruster <[email protected]>
Looks good,
Quoted 329 lines
--- scripts/qapi-commands.py | 26 +++----------------------- scripts/qapi-event.py | 26 +++----------------------- scripts/qapi-introspect.py | 21 ++------------------- scripts/qapi-types.py | 26 +++----------------------- scripts/qapi-visit.py | 26 +++----------------------- scripts/qapi.py | 31 ++++++++++++++++++------------- scripts/qapi2texi.py | 3 ++- tests/qapi-schema/doc-good.texi | 3 ++- 8 files changed, 36 insertions(+), 126 deletions(-)scripts/qapi-commands.py +3
-23@@ -255,38 +255,18 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): (input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line() -c_comment = ''' -/* - * schema-defined QMP->QAPI command dispatch +blurb = ''' + * Schema-defined QAPI/QMP commands * * Copyright IBM, Corp. 2011 * * Authors: * Anthony Liguori <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ -''' -h_comment = ''' -/* - * schema-defined QAPI function prototypes - * - * Copyright IBM, Corp. 2011 - * - * Authors: - * Anthony Liguori <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qmp-marshal.c', 'qmp-commands.h', - c_comment, h_comment) + blurb) fdef.write(mcgen('''scripts/qapi-event.py +3
-23@@ -171,38 +171,18 @@ class QAPISchemaGenEventVisitor(QAPISchemaVisitor): (input_file, output_dir, do_c, do_h, prefix, dummy) = parse_command_line() -c_comment = ''' -/* - * schema-defined QAPI event functions +blurb = ''' + * Schema-defined QAPI/QMP events * * Copyright (c) 2014 Wenchao Xia * * Authors: * Wenchao Xia <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ -''' -h_comment = ''' -/* - * schema-defined QAPI event functions - * - * Copyright (c) 2014 Wenchao Xia - * - * Authors: - * Wenchao Xia <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qapi-event.c', 'qapi-event.h', - c_comment, h_comment) + blurb) fdef.write(mcgen(''' #include "qemu/osdep.h"scripts/qapi-introspect.py +2
-19@@ -176,32 +176,15 @@ for o, a in opts: if o in ('-u', '--unmask-non-abi-names'): opt_unmask = True -c_comment = ''' -/* +blurb = ''' * QAPI/QMP schema introspection * * Copyright (C) 2015 Red Hat, Inc. - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ -''' -h_comment = ''' -/* - * QAPI/QMP schema introspection - * - * Copyright (C) 2015 Red Hat, Inc. - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qmp-introspect.c', 'qmp-introspect.h', - c_comment, h_comment) + blurb) fdef.write(mcgen(''' #include "qemu/osdep.h"scripts/qapi-types.py +3
-23@@ -250,39 +250,19 @@ for o, a in opts: if o in ('-b', '--builtins'): do_builtins = True -c_comment = ''' -/* - * deallocation functions for schema-defined QAPI types +blurb = ''' + * Schema-defined QAPI types * * Copyright IBM, Corp. 2011 * * Authors: * Anthony Liguori <[email protected]> * Michael Roth <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ -''' -h_comment = ''' -/* - * schema-defined QAPI types - * - * Copyright IBM, Corp. 2011 - * - * Authors: - * Anthony Liguori <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qapi-types.c', 'qapi-types.h', - c_comment, h_comment) + blurb) fdef.write(mcgen(''' #include "qemu/osdep.h"scripts/qapi-visit.py +3
-23@@ -334,38 +334,18 @@ for o, a in opts: if o in ('-b', '--builtins'): do_builtins = True -c_comment = ''' -/* - * schema-defined QAPI visitor functions +blurb = ''' + * Schema-defined QAPI visitors * * Copyright IBM, Corp. 2011 * * Authors: * Anthony Liguori <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ -''' -h_comment = ''' -/* - * schema-defined QAPI visitor functions - * - * Copyright IBM, Corp. 2011 - * - * Authors: - * Anthony Liguori <[email protected]> - * - * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. - * See the COPYING.LIB file in the top-level directory. - * - */ ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qapi-visit.c', 'qapi-visit.h', - c_comment, h_comment) + blurb) fdef.write(mcgen(''' #include "qemu/osdep.h"scripts/qapi.py +18
-13@@ -1975,11 +1975,21 @@ def parse_command_line(extra_options='', extra_long_options=[]): # -def open_output(output_dir, do_c, do_h, prefix, c_file, h_file, - c_comment, h_comment): +def open_output(output_dir, do_c, do_h, prefix, c_file, h_file, blurb): guard = guardname(prefix + h_file) c_file = output_dir + prefix + c_file h_file = output_dir + prefix + h_file + comment = mcgen('''/* AUTOMATICALLY GENERATED, DO NOT MODIFY */ + +/* +%(blurb)s + * + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. + * See the COPYING.LIB file in the top-level directory. + */ + +''', + blurb=blurb.strip('\n')) if output_dir: try:@@ -1998,27 +2008,22 @@ def open_output(output_dir, do_c, do_h, prefix, c_file, h_file, fdef = maybe_open(do_c, c_file, 'w') fdecl = maybe_open(do_h, h_file, 'w') - fdef.write(mcgen(''' -/* AUTOMATICALLY GENERATED, DO NOT MODIFY */ -%(comment)s -''', - comment=c_comment)) - + fdef.write(comment) + fdecl.write(comment) fdecl.write(mcgen(''' -/* AUTOMATICALLY GENERATED, DO NOT MODIFY */ -%(comment)s #ifndef %(guard)s #define %(guard)s ''', - comment=h_comment, guard=guard)) + guard=guard)) return (fdef, fdecl) def close_output(fdef, fdecl): - fdecl.write(''' + fdecl.write(mcgen(''' + #endif -''') +''')) fdecl.close() fdef.close()scripts/qapi2texi.py +2
-1@@ -282,7 +282,8 @@ def main(argv): print >>sys.stderr, ("%s: need pragma 'doc-required' " "to generate documentation" % argv[0]) sys.exit(1) - print texi_schema(schema) + print '@c AUTOMATICALLY GENERATED, DO NOT MODIFY\n' + print texi_schema(schema), if __name__ == '__main__':tests/qapi-schema/doc-good.texi +2
-2@@ -1,3 +1,5 @@ +@c AUTOMATICALLY GENERATED, DO NOT MODIFY + @section Section @subsection Subsection@@ -231,4 +233,3 @@ If you're bored enough to read this, go see a video of boxed cats @end deftypefn - --2.13.6
RFC PATCH 2/21 qapi: Generate up-to-date copyright notice
In reply to Markus ArmbrusterEach generator carries a copyright notice for the generator itself, and another one for the files it generates. Only the former have been updated along the way, the latter have not, and are all out of date.
Fix by copying the generator's copyright notice to the generated files instead.
--- scripts/qapi-commands.py | 34 +++++++++++++++------------------- scripts/qapi-event.py | 32 ++++++++++++++------------------ scripts/qapi-introspect.py | 25 ++++++++++++------------- scripts/qapi-types.py | 32 ++++++++++++++------------------ scripts/qapi-visit.py | 34 +++++++++++++++------------------- scripts/qapi.py | 7 +++++-- 6 files changed, 75 insertions(+), 89 deletions(-)
scripts/qapi-commands.py +15 -19
@@ -1,16 +1,17 @@ -# -# QAPI command marshaller generator -# -# Copyright IBM, Corp. 2011 -# Copyright (C) 2014-2016 Red Hat, Inc. -# -# Authors: -# Anthony Liguori <[email protected]> -# Michael Roth <[email protected]> -# Markus Armbruster <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2. -# See the COPYING file in the top-level directory. +""" +QAPI command marshaller generator + +Copyright IBM, Corp. 2011 +Copyright (C) 2014-2018 Red Hat, Inc. + +Authors: + Anthony Liguori <[email protected]> + Michael Roth <[email protected]> + Markus Armbruster <[email protected]> + +This work is licensed under the terms of the GNU GPL, version 2. +See the COPYING file in the top-level directory. +""" from qapi import *
@@ -257,16 +258,11 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): blurb = ''' * Schema-defined QAPI/QMP commands - * - * Copyright IBM, Corp. 2011 - * - * Authors: - * Anthony Liguori <[email protected]> ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qmp-marshal.c', 'qmp-commands.h', - blurb) + blurb, __doc__) fdef.write(mcgen('''
scripts/qapi-event.py +14 -18
@@ -1,15 +1,16 @@ -# -# QAPI event generator -# -# Copyright (c) 2014 Wenchao Xia -# Copyright (c) 2015-2016 Red Hat Inc. -# -# Authors: -# Wenchao Xia <[email protected]> -# Markus Armbruster <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2. -# See the COPYING file in the top-level directory. +""" +QAPI event generator + +Copyright (c) 2014 Wenchao Xia +Copyright (c) 2015-2018 Red Hat Inc. + +Authors: + Wenchao Xia <[email protected]> + Markus Armbruster <[email protected]> + +This work is licensed under the terms of the GNU GPL, version 2. +See the COPYING file in the top-level directory. +""" from qapi import *
@@ -173,16 +174,11 @@ class QAPISchemaGenEventVisitor(QAPISchemaVisitor): blurb = ''' * Schema-defined QAPI/QMP events - * - * Copyright (c) 2014 Wenchao Xia - * - * Authors: - * Wenchao Xia <[email protected]> ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qapi-event.c', 'qapi-event.h', - blurb) + blurb, __doc__) fdef.write(mcgen(''' #include "qemu/osdep.h"
scripts/qapi-introspect.py +12 -13
@@ -1,13 +1,14 @@ -# -# QAPI introspection generator -# -# Copyright (C) 2015-2016 Red Hat, Inc. -# -# Authors: -# Markus Armbruster <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2. -# See the COPYING file in the top-level directory. +""" +QAPI introspection generator + +Copyright (C) 2015-2018 Red Hat, Inc. + +Authors: + Markus Armbruster <[email protected]> + +This work is licensed under the terms of the GNU GPL, version 2. +See the COPYING file in the top-level directory. +""" from qapi import *
@@ -178,13 +179,11 @@ for o, a in opts: blurb = ''' * QAPI/QMP schema introspection - * - * Copyright (C) 2015 Red Hat, Inc. ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qmp-introspect.c', 'qmp-introspect.h', - blurb) + blurb, __doc__) fdef.write(mcgen(''' #include "qemu/osdep.h"
scripts/qapi-types.py +14 -18
@@ -1,15 +1,17 @@ -# -# QAPI types generator -# -# Copyright IBM, Corp. 2011 -# Copyright (c) 2013-2016 Red Hat Inc. -# -# Authors: -# Anthony Liguori <[email protected]> -# Markus Armbruster <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2. +""" +QAPI types generator + +Copyright IBM, Corp. 2011 +Copyright (c) 2013-2018 Red Hat Inc. + +Authors: + Anthony Liguori <[email protected]> + Michael Roth <[email protected]> + Markus Armbruster <[email protected]> + +This work is licensed under the terms of the GNU GPL, version 2. # See the COPYING file in the top-level directory. +""" from qapi import *
@@ -252,17 +254,11 @@ for o, a in opts: blurb = ''' * Schema-defined QAPI types - * - * Copyright IBM, Corp. 2011 - * - * Authors: - * Anthony Liguori <[email protected]> - * Michael Roth <[email protected]> ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qapi-types.c', 'qapi-types.h', - blurb) + blurb, __doc__) fdef.write(mcgen(''' #include "qemu/osdep.h"
scripts/qapi-visit.py +15 -19
@@ -1,16 +1,17 @@ -# -# QAPI visitor generator -# -# Copyright IBM, Corp. 2011 -# Copyright (C) 2014-2016 Red Hat, Inc. -# -# Authors: -# Anthony Liguori <[email protected]> -# Michael Roth <[email protected]> -# Markus Armbruster <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2. -# See the COPYING file in the top-level directory. +""" +QAPI visitor generator + +Copyright IBM, Corp. 2011 +Copyright (C) 2014-2018 Red Hat, Inc. + +Authors: + Anthony Liguori <[email protected]> + Michael Roth <[email protected]> + Markus Armbruster <[email protected]> + +This work is licensed under the terms of the GNU GPL, version 2. +See the COPYING file in the top-level directory. +""" from qapi import *
@@ -336,16 +337,11 @@ for o, a in opts: blurb = ''' * Schema-defined QAPI visitors - * - * Copyright IBM, Corp. 2011 - * - * Authors: - * Anthony Liguori <[email protected]> ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qapi-visit.c', 'qapi-visit.h', - blurb) + blurb, __doc__) fdef.write(mcgen(''' #include "qemu/osdep.h"
scripts/qapi.py +5 -2
@@ -1975,21 +1975,24 @@ def parse_command_line(extra_options='', extra_long_options=[]): # -def open_output(output_dir, do_c, do_h, prefix, c_file, h_file, blurb): +def open_output(output_dir, do_c, do_h, prefix, c_file, h_file, blurb, doc): guard = guardname(prefix + h_file) c_file = output_dir + prefix + c_file h_file = output_dir + prefix + h_file + copyright = '\n * '.join(re.findall(r'^Copyright .*', doc, re.MULTILINE)) comment = mcgen('''/* AUTOMATICALLY GENERATED, DO NOT MODIFY */ /* %(blurb)s * + * %(copyright)s + * * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. * See the COPYING.LIB file in the top-level directory. */ ''', - blurb=blurb.strip('\n')) + blurb=blurb.strip('\n'), copyright=copyright) if output_dir: try:
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 02/21] qapi: Generate up-to-date copyright notice
Quoted 29 lines · Markus Armbruster
Each generator carries a copyright notice for the generator itself, and another one for the files it generates. Only the former have been updated along the way, the latter have not, and are all out of date.
Fix by copying the generator's copyright notice to the generated files instead.
Signed-off-by Markus Armbruster <[email protected]>---
+++
b/scripts/qapi-commands.pyscripts/qapi-commands.py +4
-13@@ -1,16 +1,17 @@ -# -# QAPI command marshaller generator -# -# Copyright IBM, Corp. 2011 -# Copyright (C) 2014-2016 Red Hat, Inc. -# -# Authors: -# Anthony Liguori <[email protected]> -# Michael Roth <[email protected]> -# Markus Armbruster <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2. -# See the COPYING file in the top-level directory. +""" +QAPI command marshaller generator + +Copyright IBM, Corp. 2011
scripts/qapi-commands.py line 14
Quoted 10 lines
scripts/qapi-commands.py +10
-0+Copyright (C) 2014-2018 Red Hat, Inc. + +Authors: + Anthony Liguori <[email protected]> + Michael Roth <[email protected]> + Markus Armbruster <[email protected]> + +This work is licensed under the terms of the GNU GPL, version 2. +See the COPYING file in the top-level directory. +"""
So python lets you start a file with a string constant that is not associated with any variable name?
scripts/qapi-commands.py class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): · line 265
Quoted 5 lines
scripts/qapi-commands.py +1
-1(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qmp-marshal.c', 'qmp-commands.h', - blurb) + blurb, __doc__)
Ah, and there's what I was missing - python auto-assigns such an initial string to a magic automatic variable. Cool language feature I hadn't seen before!
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 02/21] qapi: Generate up-to-date copyright notice
Quoted 35 lines · Eric Blake
Quoted 34 lines · Markus Armbruster
Each generator carries a copyright notice for the generator itself, and another one for the files it generates. Only the former have been updated along the way, the latter have not, and are all out of date.
Fix by copying the generator's copyright notice to the generated files instead.
Signed-off-by Markus Armbruster <[email protected]>---
+++
b/scripts/qapi-commands.pyscripts/qapi-commands.py +9
-13@@ -1,16 +1,17 @@ -# -# QAPI command marshaller generator -# -# Copyright IBM, Corp. 2011 -# Copyright (C) 2014-2016 Red Hat, Inc. -# -# Authors: -# Anthony Liguori <[email protected]> -# Michael Roth <[email protected]> -# Markus Armbruster <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2. -# See the COPYING file in the top-level directory. +""" +QAPI command marshaller generator + +Copyright IBM, Corp. 2011 +Copyright (C) 2014-2018 Red Hat, Inc. + +Authors: + Anthony Liguori <[email protected]> + Michael Roth <[email protected]>
Quoted 8 lines
scripts/qapi-commands.py line 14
Quoted 5 lines
scripts/qapi-commands.py +5
-0+ Markus Armbruster <[email protected]> + +This work is licensed under the terms of the GNU GPL, version 2. +See the COPYING file in the top-level directory. +"""So python lets you start a file with a string constant that is not associated with any variable name?
It's the module's doc string, done the same way as function doc strings.
https://www.python.org/dev/peps/pep-0257/#what-is-a-docstring
Quoted 10 lines
scripts/qapi-commands.py class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): · line 265
Quoted 4 lines
scripts/qapi-commands.py +1
-1(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qmp-marshal.c', 'qmp-commands.h', - blurb) + blurb, __doc__)Ah, and there's what I was missing - python auto-assigns such an initial string to a magic automatic variable. Cool language feature I hadn't seen before!
Reviewed-by Eric Blake <[email protected]>
Thanks!
Re: [Qemu-devel] [PATCH RFC 02/21] qapi: Generate up-to-date copyright notice
In reply to Markus ArmbrusterHi
Quoted 6 lines · Markus Armbruster
Each generator carries a copyright notice for the generator itself, and another one for the files it generates. Only the former have been updated along the way, the latter have not, and are all out of date.
Fix by copying the generator's copyright notice to the generated files instead.
That makes sense, but we loose the Author lines in the generated files. Not a big deal I guess, but worth to point out, no?
Quoted 2 lines
Signed-off-by Markus Armbruster <[email protected]>
Other than that,
Quoted 296 lines
--- scripts/qapi-commands.py | 34 +++++++++++++++------------------- scripts/qapi-event.py | 32 ++++++++++++++------------------ scripts/qapi-introspect.py | 25 ++++++++++++------------- scripts/qapi-types.py | 32 ++++++++++++++------------------ scripts/qapi-visit.py | 34 +++++++++++++++------------------- scripts/qapi.py | 7 +++++-- 6 files changed, 75 insertions(+), 89 deletions(-)scripts/qapi-commands.py +15
-19@@ -1,16 +1,17 @@ -# -# QAPI command marshaller generator -# -# Copyright IBM, Corp. 2011 -# Copyright (C) 2014-2016 Red Hat, Inc. -# -# Authors: -# Anthony Liguori <[email protected]> -# Michael Roth <[email protected]> -# Markus Armbruster <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2. -# See the COPYING file in the top-level directory. +""" +QAPI command marshaller generator + +Copyright IBM, Corp. 2011 +Copyright (C) 2014-2018 Red Hat, Inc. + +Authors: + Anthony Liguori <[email protected]> + Michael Roth <[email protected]> + Markus Armbruster <[email protected]> + +This work is licensed under the terms of the GNU GPL, version 2. +See the COPYING file in the top-level directory. +""" from qapi import *@@ -257,16 +258,11 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): blurb = ''' * Schema-defined QAPI/QMP commands - * - * Copyright IBM, Corp. 2011 - * - * Authors: - * Anthony Liguori <[email protected]> ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qmp-marshal.c', 'qmp-commands.h', - blurb) + blurb, __doc__) fdef.write(mcgen('''scripts/qapi-event.py +14
-18@@ -1,15 +1,16 @@ -# -# QAPI event generator -# -# Copyright (c) 2014 Wenchao Xia -# Copyright (c) 2015-2016 Red Hat Inc. -# -# Authors: -# Wenchao Xia <[email protected]> -# Markus Armbruster <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2. -# See the COPYING file in the top-level directory. +""" +QAPI event generator + +Copyright (c) 2014 Wenchao Xia +Copyright (c) 2015-2018 Red Hat Inc. + +Authors: + Wenchao Xia <[email protected]> + Markus Armbruster <[email protected]> + +This work is licensed under the terms of the GNU GPL, version 2. +See the COPYING file in the top-level directory. +""" from qapi import *@@ -173,16 +174,11 @@ class QAPISchemaGenEventVisitor(QAPISchemaVisitor): blurb = ''' * Schema-defined QAPI/QMP events - * - * Copyright (c) 2014 Wenchao Xia - * - * Authors: - * Wenchao Xia <[email protected]> ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qapi-event.c', 'qapi-event.h', - blurb) + blurb, __doc__) fdef.write(mcgen(''' #include "qemu/osdep.h"scripts/qapi-introspect.py +12
-13@@ -1,13 +1,14 @@ -# -# QAPI introspection generator -# -# Copyright (C) 2015-2016 Red Hat, Inc. -# -# Authors: -# Markus Armbruster <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2. -# See the COPYING file in the top-level directory. +""" +QAPI introspection generator + +Copyright (C) 2015-2018 Red Hat, Inc. + +Authors: + Markus Armbruster <[email protected]> + +This work is licensed under the terms of the GNU GPL, version 2. +See the COPYING file in the top-level directory. +""" from qapi import *@@ -178,13 +179,11 @@ for o, a in opts: blurb = ''' * QAPI/QMP schema introspection - * - * Copyright (C) 2015 Red Hat, Inc. ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qmp-introspect.c', 'qmp-introspect.h', - blurb) + blurb, __doc__) fdef.write(mcgen(''' #include "qemu/osdep.h"scripts/qapi-types.py +14
-18@@ -1,15 +1,17 @@ -# -# QAPI types generator -# -# Copyright IBM, Corp. 2011 -# Copyright (c) 2013-2016 Red Hat Inc. -# -# Authors: -# Anthony Liguori <[email protected]> -# Markus Armbruster <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2. +""" +QAPI types generator + +Copyright IBM, Corp. 2011 +Copyright (c) 2013-2018 Red Hat Inc. + +Authors: + Anthony Liguori <[email protected]> + Michael Roth <[email protected]> + Markus Armbruster <[email protected]> + +This work is licensed under the terms of the GNU GPL, version 2. # See the COPYING file in the top-level directory. +""" from qapi import *@@ -252,17 +254,11 @@ for o, a in opts: blurb = ''' * Schema-defined QAPI types - * - * Copyright IBM, Corp. 2011 - * - * Authors: - * Anthony Liguori <[email protected]> - * Michael Roth <[email protected]> ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qapi-types.c', 'qapi-types.h', - blurb) + blurb, __doc__) fdef.write(mcgen(''' #include "qemu/osdep.h"scripts/qapi-visit.py +15
-19@@ -1,16 +1,17 @@ -# -# QAPI visitor generator -# -# Copyright IBM, Corp. 2011 -# Copyright (C) 2014-2016 Red Hat, Inc. -# -# Authors: -# Anthony Liguori <[email protected]> -# Michael Roth <[email protected]> -# Markus Armbruster <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2. -# See the COPYING file in the top-level directory. +""" +QAPI visitor generator + +Copyright IBM, Corp. 2011 +Copyright (C) 2014-2018 Red Hat, Inc. + +Authors: + Anthony Liguori <[email protected]> + Michael Roth <[email protected]> + Markus Armbruster <[email protected]> + +This work is licensed under the terms of the GNU GPL, version 2. +See the COPYING file in the top-level directory. +""" from qapi import *@@ -336,16 +337,11 @@ for o, a in opts: blurb = ''' * Schema-defined QAPI visitors - * - * Copyright IBM, Corp. 2011 - * - * Authors: - * Anthony Liguori <[email protected]> ''' (fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, 'qapi-visit.c', 'qapi-visit.h', - blurb) + blurb, __doc__) fdef.write(mcgen(''' #include "qemu/osdep.h"scripts/qapi.py +5
-3@@ -1975,21 +1975,24 @@ def parse_command_line(extra_options='', extra_long_options=[]): # -def open_output(output_dir, do_c, do_h, prefix, c_file, h_file, blurb): +def open_output(output_dir, do_c, do_h, prefix, c_file, h_file, blurb, doc): guard = guardname(prefix + h_file) c_file = output_dir + prefix + c_file h_file = output_dir + prefix + h_file + copyright = '\n * '.join(re.findall(r'^Copyright .*', doc, re.MULTILINE)) comment = mcgen('''/* AUTOMATICALLY GENERATED, DO NOT MODIFY */ /* %(blurb)s * + * %(copyright)s + * * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. * See the COPYING.LIB file in the top-level directory. */ ''', - blurb=blurb.strip('\n')) + blurb=blurb.strip('\n'), copyright=copyright) if output_dir: try: --2.13.6
Re: [Qemu-devel] [PATCH RFC 02/21] qapi: Generate up-to-date copyright notice
Quoted 7 lines · Marc-Andre Lureau
Hi
Quoted 4 lines · Markus Armbruster
Each generator carries a copyright notice for the generator itself, and another one for the files it generates. Only the former have been updated along the way, the latter have not, and are all out of date.
Quoted 5 lines
Quoted 2 lines
Fix by copying the generator's copyright notice to the generated files instead.
That makes sense, but we loose the Author lines in the generated files. Not a big deal I guess, but worth to point out, no?
We do. They've become just as outdated as the copyright notices. Giving them the same treatment doesn't seem to be worth the trouble. I'm happy to explain this in the commit message.
Quoted 5 lines
Quoted 2 lines
Signed-off-by Markus Armbruster <[email protected]>Other than that,
Reviewed-by Marc-André Lureau <[email protected]>
Thanks!
Re: [Qemu-devel] [PATCH RFC 02/21] qapi: Generate up-to-date copyright notice
In reply to Marc-Andre LureauQuoted 7 lines · Marc-Andre Lureau
Hi
Quoted 4 lines · Markus Armbruster
Each generator carries a copyright notice for the generator itself, and another one for the files it generates. Only the former have been updated along the way, the latter have not, and are all out of date.
Quoted 5 lines
Quoted 2 lines
Fix by copying the generator's copyright notice to the generated files instead.
That makes sense, but we loose the Author lines in the generated files. Not a big deal I guess, but worth to point out, no?
I don't see too much use for author lines as long as you have a 'this file is generated' line; you can use git history to learn the authors of the generator code. But yeah, a commit message blurb can't hurt.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
RFC PATCH 3/21 qapi: New classes QAPIGenC, QAPIGenH, QAPIGenDoc
In reply to Markus ArmbrusterThese classes encapsulate accumulating and writing output.
Convert C code generation to QAPIGenC and QAPIGenH. The conversion is rather shallow: most of the output accumulation is not converted. Left for later.
The indentation machinery uses a single global variable indent_level, even though we generally interleave creation of a .c and its .h. It should become instance variable of QAPIGenC. Also left for later.
Documentation generation isn't converted, and QAPIGenDoc isn't used. This will change shortly.
--- scripts/qapi-commands.py | 27 ++++++------- scripts/qapi-event.py | 26 +++++++------ scripts/qapi-introspect.py | 22 ++++++----- scripts/qapi-types.py | 26 +++++++------ scripts/qapi-visit.py | 26 +++++++------ scripts/qapi.py | 96 ++++++++++++++++++++++++++-------------------- 6 files changed, 122 insertions(+), 101 deletions(-)
scripts/qapi-commands.py +14 -13
@@ -260,12 +260,10 @@ blurb = ''' * Schema-defined QAPI/QMP commands ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qmp-marshal.c', 'qmp-commands.h', - blurb, __doc__) - -fdef.write(mcgen(''' +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) +genc.body(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" #include "qemu/module.h"
@@ -279,21 +277,24 @@ fdef.write(mcgen(''' #include "%(prefix)sqmp-commands.h" ''', - prefix=prefix)) + prefix=prefix)) -fdecl.write(mcgen(''' +genh.body(mcgen(''' #include "%(prefix)sqapi-types.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/dispatch.h" void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds); ''', - prefix=prefix, c_prefix=c_name(prefix, protect=False))) + prefix=prefix, c_prefix=c_name(prefix, protect=False))) schema = QAPISchema(input_file) -gen = QAPISchemaGenCommandVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenCommandVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qmp-marshal.c') +if do_h: + genh.write(output_dir, prefix + 'qmp-commands.h')
scripts/qapi-event.py +14 -12
@@ -176,11 +176,10 @@ blurb = ''' * Schema-defined QAPI/QMP events ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qapi-event.c', 'qapi-event.h', - blurb, __doc__) +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) -fdef.write(mcgen(''' +genc.body(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" #include "%(prefix)sqapi-event.h"
@@ -190,22 +189,25 @@ fdef.write(mcgen(''' #include "qapi/qmp-event.h" ''', - prefix=prefix)) + prefix=prefix)) -fdecl.write(mcgen(''' +genh.body(mcgen(''' #include "qapi/util.h" #include "qapi/qmp/qdict.h" #include "%(prefix)sqapi-types.h" ''', - prefix=prefix)) + prefix=prefix)) event_enum_name = c_name(prefix + 'QAPIEvent', protect=False) schema = QAPISchema(input_file) -gen = QAPISchemaGenEventVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenEventVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qapi-event.c') +if do_h: + genh.write(output_dir, prefix + 'qapi-event.h')
scripts/qapi-introspect.py +12 -10
@@ -181,21 +181,23 @@ blurb = ''' * QAPI/QMP schema introspection ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qmp-introspect.c', 'qmp-introspect.h', - blurb, __doc__) +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) -fdef.write(mcgen(''' +genc.body(mcgen(''' #include "qemu/osdep.h" #include "%(prefix)sqmp-introspect.h" ''', - prefix=prefix)) + prefix=prefix)) schema = QAPISchema(input_file) -gen = QAPISchemaGenIntrospectVisitor(opt_unmask) -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenIntrospectVisitor(opt_unmask) +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qmp-introspect.c') +if do_h: + genh.write(output_dir, prefix + 'qmp-introspect.h')
scripts/qapi-types.py +14 -12
@@ -180,7 +180,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self.decl = '' self.defn = '' self._fwdecl = '' - self._btin = guardstart('QAPI_TYPES_BUILTIN') + self._btin = '\n' + guardstart('QAPI_TYPES_BUILTIN') def visit_end(self): self.decl = self._fwdecl + self.decl
@@ -256,26 +256,28 @@ blurb = ''' * Schema-defined QAPI types ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qapi-types.c', 'qapi-types.h', - blurb, __doc__) +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) -fdef.write(mcgen(''' +genc.body(mcgen(''' #include "qemu/osdep.h" #include "qapi/dealloc-visitor.h" #include "%(prefix)sqapi-types.h" #include "%(prefix)sqapi-visit.h" ''', - prefix=prefix)) + prefix=prefix)) -fdecl.write(mcgen(''' +genh.body(mcgen(''' #include "qapi/util.h" ''')) schema = QAPISchema(input_file) -gen = QAPISchemaGenTypeVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenTypeVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qapi-types.c') +if do_h: + genh.write(output_dir, prefix + 'qapi-types.h')
scripts/qapi-visit.py +14 -12
@@ -339,30 +339,32 @@ blurb = ''' * Schema-defined QAPI visitors ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qapi-visit.c', 'qapi-visit.h', - blurb, __doc__) +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) -fdef.write(mcgen(''' +genc.body(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" #include "qapi/error.h" #include "%(prefix)sqapi-visit.h" ''', - prefix=prefix)) + prefix=prefix)) -fdecl.write(mcgen(''' +genh.body(mcgen(''' #include "qapi/visitor.h" #include "qapi/qmp/qerror.h" #include "%(prefix)sqapi-types.h" ''', - prefix=prefix)) + prefix=prefix)) schema = QAPISchema(input_file) -gen = QAPISchemaGenVisitVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenVisitVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qapi-visit.c') +if do_h: + genh.write(output_dir, prefix + 'qapi-visit.h')
scripts/qapi.py +54 -42
133 lines not on this page · raw message
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 03/21] qapi: New classes QAPIGenC, QAPIGenH, QAPIGenDoc
Quoted 14 lines · Markus Armbruster
These classes encapsulate accumulating and writing output.
Convert C code generation to QAPIGenC and QAPIGenH. The conversion is rather shallow: most of the output accumulation is not converted. Left for later.
The indentation machinery uses a single global variable
indent_level, even though we generally interleave creation of a .c and its .h. It should become instance variable of QAPIGenC. Also left for later.Documentation generation isn't converted, and QAPIGenDoc isn't used. This will change shortly.
Signed-off-by Markus Armbruster <[email protected]>
Quoted 9 lines
--- scripts/qapi-commands.py | 27 ++++++------- scripts/qapi-event.py | 26 +++++++------ scripts/qapi-introspect.py | 22 ++++++----- scripts/qapi-types.py | 26 +++++++------ scripts/qapi-visit.py | 26 +++++++------ scripts/qapi.py | 96 ++++++++++++++++++++++++++-------------------- 6 files changed, 122 insertions(+), 101 deletions(-)
A little bit longer due to more structure, but reasonable diffstat in that it shows the conversion is fairly straightforward and opens the doors for later patches to use the new structures more effectively.
scripts/qapi-event.py fdef.write(mcgen(''' · line 208
Quoted 10 lines
schema =
QAPISchema(input_file)scripts/qapi-event.py +4
-4-gen = QAPISchemaGenEventVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenEventVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl)
I don't know if it is worth a sentence in the commit message that the visitor variable is renamed from 'gen' to 'vis' for less confusion with the new class instances 'genc' and 'genh'.
scripts/qapi-types.py class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): · line 183
Quoted 7 lines
+++
b/scripts/qapi-types.pyscripts/qapi-types.py +1
-1@@ -180,7 +180,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self.decl = '' self.defn = '' self._fwdecl = '' - self._btin = guardstart('QAPI_TYPES_BUILTIN') + self._btin = '\n' + guardstart('QAPI_TYPES_BUILTIN')
Tweaks like this means you were paying attention to still producing identical generated files; always a good sign.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 03/21] qapi: New classes QAPIGenC, QAPIGenH, QAPIGenDoc
Quoted 30 lines · Eric Blake
Quoted 14 lines · Markus Armbruster
These classes encapsulate accumulating and writing output.
Convert C code generation to QAPIGenC and QAPIGenH. The conversion is rather shallow: most of the output accumulation is not converted. Left for later.
The indentation machinery uses a single global variable
indent_level, even though we generally interleave creation of a .c and its .h. It should become instance variable of QAPIGenC. Also left for later.Documentation generation isn't converted, and QAPIGenDoc isn't used. This will change shortly.
Signed-off-by Markus Armbruster <[email protected]>Quoted 9 lines
--- scripts/qapi-commands.py | 27 ++++++------- scripts/qapi-event.py | 26 +++++++------ scripts/qapi-introspect.py | 22 ++++++----- scripts/qapi-types.py | 26 +++++++------ scripts/qapi-visit.py | 26 +++++++------ scripts/qapi.py | 96 ++++++++++++++++++++++++++-------------------- 6 files changed, 122 insertions(+), 101 deletions(-)A little bit longer due to more structure, but reasonable diffstat in that it shows the conversion is fairly straightforward and opens the doors for later patches to use the new structures more effectively.
Quoted 1 line
Quoted 13 lines
scripts/qapi-event.py fdef.write(mcgen(''' · line 208
Quoted 9 lines
schema =
QAPISchema(input_file)scripts/qapi-event.py +4
-4-gen = QAPISchemaGenEventVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenEventVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl)I don't know if it is worth a sentence in the commit message that the visitor variable is renamed from 'gen' to 'vis' for less confusion with the new class instances 'genc' and 'genh'.
Did the rename give you pause when reviewing?
Quoted 11 lines
scripts/qapi-types.py class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): · line 183
Quoted 7 lines
+++
b/scripts/qapi-types.pyscripts/qapi-types.py +1
-1@@ -180,7 +180,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self.decl = '' self.defn = '' self._fwdecl = '' - self._btin = guardstart('QAPI_TYPES_BUILTIN') + self._btin = '\n' + guardstart('QAPI_TYPES_BUILTIN')Tweaks like this means you were paying attention to still producing identical generated files; always a good sign.
Quoted 1 line
Reviewed-by Eric Blake <[email protected]>
Thanks!
Re: [Qemu-devel] [PATCH RFC 03/21] qapi: New classes QAPIGenC, QAPIGenH, QAPIGenDoc
Quoted 31 lines · Markus Armbruster
Quoted 15 lines · Eric Blake
Quoted 13 lines · Markus Armbruster
These classes encapsulate accumulating and writing output.
Convert C code generation to QAPIGenC and QAPIGenH. The conversion is rather shallow: most of the output accumulation is not converted. Left for later.
The indentation machinery uses a single global variable
indent_level, even though we generally interleave creation of a .c and its .h. It should become instance variable of QAPIGenC. Also left for later.Documentation generation isn't converted, and QAPIGenDoc isn't used. This will change shortly.
Quoted 13 lines
scripts/qapi-event.py fdef.write(mcgen(''' · line 208
Quoted 9 lines
schema =
QAPISchema(input_file)scripts/qapi-event.py +4
-4-gen = QAPISchemaGenEventVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenEventVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl)I don't know if it is worth a sentence in the commit message that the visitor variable is renamed from 'gen' to 'vis' for less confusion with the new class instances 'genc' and 'genh'.
Quoted 1 line
Did the rename give you pause when reviewing?
Enough to question whether it was intentional, since it wasn't mentioned in the commit message (I obviously figured out that it was intentional and useful, but the fact that I even pointed it out meant that I did pause during the review).
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 03/21] qapi: New classes QAPIGenC, QAPIGenH, QAPIGenDoc
Quoted 32 lines · Eric Blake
Quoted 31 lines · Markus Armbruster
Quoted 28 lines · Eric Blake
Quoted 15 lines · Markus Armbruster
These classes encapsulate accumulating and writing output.
Convert C code generation to QAPIGenC and QAPIGenH. The conversion is rather shallow: most of the output accumulation is not converted. Left for later.
The indentation machinery uses a single global variable
indent_level, even though we generally interleave creation of a .c and its .h. It should become instance variable of QAPIGenC. Also left for later.Documentation generation isn't converted, and QAPIGenDoc isn't used. This will change shortly.
schema =
QAPISchema(input_file)scripts/qapi-event.py fdef.write(mcgen(''' · line 208
Quoted 8 lines
scripts/qapi-event.py +4
-4-gen = QAPISchemaGenEventVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenEventVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl)I don't know if it is worth a sentence in the commit message that the visitor variable is renamed from 'gen' to 'vis' for less confusion with the new class instances 'genc' and 'genh'.
Quoted 6 lines
Quoted 1 line
Did the rename give you pause when reviewing?
Enough to question whether it was intentional, since it wasn't mentioned in the commit message (I obviously figured out that it was intentional and useful, but the fact that I even pointed it out meant that I did pause during the review).
Okay, I'll try to make it clearer.
Re: [Qemu-devel] [PATCH RFC 03/21] qapi: New classes QAPIGenC, QAPIGenH, QAPIGenDoc
In reply to Markus ArmbrusterHi
Quoted 316 lines · Markus Armbruster
These classes encapsulate accumulating and writing output.
Convert C code generation to QAPIGenC and QAPIGenH. The conversion is rather shallow: most of the output accumulation is not converted. Left for later.
The indentation machinery uses a single global variable
indent_level, even though we generally interleave creation of a .c and its .h. It should become instance variable of QAPIGenC. Also left for later.Documentation generation isn't converted, and QAPIGenDoc isn't used. This will change shortly.
Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi-commands.py | 27 ++++++------- scripts/qapi-event.py | 26 +++++++------ scripts/qapi-introspect.py | 22 ++++++----- scripts/qapi-types.py | 26 +++++++------ scripts/qapi-visit.py | 26 +++++++------ scripts/qapi.py | 96 ++++++++++++++++++++++++++-------------------- 6 files changed, 122 insertions(+), 101 deletions(-)scripts/qapi-commands.py +14
-13@@ -260,12 +260,10 @@ blurb = ''' * Schema-defined QAPI/QMP commands ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qmp-marshal.c', 'qmp-commands.h', - blurb, __doc__) - -fdef.write(mcgen(''' +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) +genc.body(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" #include "qemu/module.h"@@ -279,21 +277,24 @@ fdef.write(mcgen(''' #include "%(prefix)sqmp-commands.h" ''', - prefix=prefix)) + prefix=prefix)) -fdecl.write(mcgen(''' +genh.body(mcgen(''' #include "%(prefix)sqapi-types.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/dispatch.h" void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds); ''', - prefix=prefix, c_prefix=c_name(prefix, protect=False))) + prefix=prefix, c_prefix=c_name(prefix, protect=False))) schema = QAPISchema(input_file) -gen = QAPISchemaGenCommandVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenCommandVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qmp-marshal.c') +if do_h: + genh.write(output_dir, prefix + 'qmp-commands.h')scripts/qapi-event.py +14
-12@@ -176,11 +176,10 @@ blurb = ''' * Schema-defined QAPI/QMP events ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qapi-event.c', 'qapi-event.h', - blurb, __doc__) +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) -fdef.write(mcgen(''' +genc.body(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" #include "%(prefix)sqapi-event.h"@@ -190,22 +189,25 @@ fdef.write(mcgen(''' #include "qapi/qmp-event.h" ''', - prefix=prefix)) + prefix=prefix)) -fdecl.write(mcgen(''' +genh.body(mcgen(''' #include "qapi/util.h" #include "qapi/qmp/qdict.h" #include "%(prefix)sqapi-types.h" ''', - prefix=prefix)) + prefix=prefix)) event_enum_name = c_name(prefix + 'QAPIEvent', protect=False) schema = QAPISchema(input_file) -gen = QAPISchemaGenEventVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenEventVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qapi-event.c') +if do_h: + genh.write(output_dir, prefix + 'qapi-event.h')scripts/qapi-introspect.py +12
-10@@ -181,21 +181,23 @@ blurb = ''' * QAPI/QMP schema introspection ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qmp-introspect.c', 'qmp-introspect.h', - blurb, __doc__) +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) -fdef.write(mcgen(''' +genc.body(mcgen(''' #include "qemu/osdep.h" #include "%(prefix)sqmp-introspect.h" ''', - prefix=prefix)) + prefix=prefix)) schema = QAPISchema(input_file) -gen = QAPISchemaGenIntrospectVisitor(opt_unmask) -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenIntrospectVisitor(opt_unmask) +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qmp-introspect.c') +if do_h: + genh.write(output_dir, prefix + 'qmp-introspect.h')scripts/qapi-types.py +14
-12@@ -180,7 +180,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self.decl = '' self.defn = '' self._fwdecl = '' - self._btin = guardstart('QAPI_TYPES_BUILTIN') + self._btin = '\n' + guardstart('QAPI_TYPES_BUILTIN') def visit_end(self): self.decl = self._fwdecl + self.decl@@ -256,26 +256,28 @@ blurb = ''' * Schema-defined QAPI types ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qapi-types.c', 'qapi-types.h', - blurb, __doc__) +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) -fdef.write(mcgen(''' +genc.body(mcgen(''' #include "qemu/osdep.h" #include "qapi/dealloc-visitor.h" #include "%(prefix)sqapi-types.h" #include "%(prefix)sqapi-visit.h" ''', - prefix=prefix)) + prefix=prefix)) -fdecl.write(mcgen(''' +genh.body(mcgen(''' #include "qapi/util.h" ''')) schema = QAPISchema(input_file) -gen = QAPISchemaGenTypeVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenTypeVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qapi-types.c') +if do_h: + genh.write(output_dir, prefix + 'qapi-types.h')scripts/qapi-visit.py +14
-12@@ -339,30 +339,32 @@ blurb = ''' * Schema-defined QAPI visitors ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qapi-visit.c', 'qapi-visit.h', - blurb, __doc__) +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) -fdef.write(mcgen(''' +genc.body(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" #include "qapi/error.h" #include "%(prefix)sqapi-visit.h" ''', - prefix=prefix)) + prefix=prefix)) -fdecl.write(mcgen(''' +genh.body(mcgen(''' #include "qapi/visitor.h" #include "qapi/qmp/qerror.h" #include "%(prefix)sqapi-types.h" ''', - prefix=prefix)) + prefix=prefix)) schema = QAPISchema(input_file) -gen = QAPISchemaGenVisitVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenVisitVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qapi-visit.c') +if do_h: + genh.write(output_dir, prefix + 'qapi-visit.h')scripts/qapi.py +11
-4@@ -2,7 +2,7 @@ # QAPI helper library # # Copyright IBM, Corp. 2011 -# Copyright (c) 2013-2016 Red Hat Inc. +# Copyright (c) 2013-2018 Red Hat Inc. # # Authors: # Anthony Liguori <[email protected]>@@ -1820,7 +1820,6 @@ def guardname(filename): def guardstart(name): return mcgen(''' - #ifndef %(name)s #define %(name)s@@ -1832,7 +1831,6 @@ def guardend(name): return mcgen(''' #endif /* %(name)s */ - ''', name=guardname(name))@@ -1970,17 +1968,53 @@ def parse_command_line(extra_options='', extra_long_options=[]): return (fname, output_dir, do_c, do_h, prefix, extra_opts) + # -# Generate output files with boilerplate +# Accumulate and write output # +class QAPIGen(object): + + def __init__(self): + self._preamble = '' + self._body = '' + + def preamble(self, text): + self._preamble += text
scripts/qapi.py def parse_command_line(extra_options='', extra_long_options=[]): · line 1993
Quoted 10 lines
scripts/qapi.py +10
-0+ + def body(self, text): + self._body += text + + def top(self, fname): + return '' + + def bottom(self, fname): + return '' +
Some methods appends, other return. That's a bit confusing. Why not name them accordingly? add_preamble, add_body, get_top...?
Quoted 86 lines
scripts/qapi.py +33
-39+ def write(self, output_dir, fname): + if output_dir: + try: + os.makedirs(output_dir) + except os.error as e: + if e.errno != errno.EEXIST: + raise + f = open(os.path.join(output_dir, fname), 'w') + f.write(self.top(fname) + self._preamble + self._body + + self.bottom(fname)) + f.close() + + +class QAPIGenC(QAPIGen): + + def __init__(self, blurb, pydoc): + QAPIGen.__init__(self) + self._blurb = blurb.strip('\n') + self._copyright = '\n * '.join(re.findall(r'^Copyright .*', pydoc, + re.MULTILINE)) -def open_output(output_dir, do_c, do_h, prefix, c_file, h_file, blurb, doc): - guard = guardname(prefix + h_file) - c_file = output_dir + prefix + c_file - h_file = output_dir + prefix + h_file - copyright = '\n * '.join(re.findall(r'^Copyright .*', doc, re.MULTILINE)) - comment = mcgen('''/* AUTOMATICALLY GENERATED, DO NOT MODIFY */ + def top(self, fname): + return mcgen(''' +/* AUTOMATICALLY GENERATED, DO NOT MODIFY */ /* %(blurb)s@@ -1992,41 +2026,19 @@ def open_output(output_dir, do_c, do_h, prefix, c_file, h_file, blurb, doc): */ ''', - blurb=blurb.strip('\n'), copyright=copyright) + blurb=self._blurb, copyright=self._copyright) - if output_dir: - try: - os.makedirs(output_dir) - except os.error as e: - if e.errno != errno.EEXIST: - raise - def maybe_open(really, name, opt): - if really: - return open(name, opt) - else: - import StringIO - return StringIO.StringIO() +class QAPIGenH(QAPIGenC): - fdef = maybe_open(do_c, c_file, 'w') - fdecl = maybe_open(do_h, h_file, 'w') + def top(self, fname): + return QAPIGenC.top(self, fname) + guardstart(fname) - fdef.write(comment) - fdecl.write(comment) - fdecl.write(mcgen(''' -#ifndef %(guard)s -#define %(guard)s + def bottom(self, fname): + return guardend(fname) -''', - guard=guard)) - return (fdef, fdecl) - - -def close_output(fdef, fdecl): - fdecl.write(mcgen(''' - -#endif -''')) - fdecl.close() - fdef.close() +class QAPIGenDoc(QAPIGen): + def top(self, fname): + return (QAPIGen.top(self, fname) + + '@c AUTOMATICALLY GENERATED, DO NOT MODIFY\n\n') --
Quoted 2 lines
2.13.6
Re: [Qemu-devel] [PATCH RFC 03/21] qapi: New classes QAPIGenC, QAPIGenH, QAPIGenDoc
Quoted 324 lines · Marc-Andre Lureau
Hi
Quoted 321 lines · Markus Armbruster
These classes encapsulate accumulating and writing output.
Convert C code generation to QAPIGenC and QAPIGenH. The conversion is rather shallow: most of the output accumulation is not converted. Left for later.
The indentation machinery uses a single global variable
indent_level, even though we generally interleave creation of a .c and its .h. It should become instance variable of QAPIGenC. Also left for later.Documentation generation isn't converted, and QAPIGenDoc isn't used. This will change shortly.
Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi-commands.py | 27 ++++++------- scripts/qapi-event.py | 26 +++++++------ scripts/qapi-introspect.py | 22 ++++++----- scripts/qapi-types.py | 26 +++++++------ scripts/qapi-visit.py | 26 +++++++------ scripts/qapi.py | 96 ++++++++++++++++++++++++++-------------------- 6 files changed, 122 insertions(+), 101 deletions(-)scripts/qapi-commands.py +14
-13@@ -260,12 +260,10 @@ blurb = ''' * Schema-defined QAPI/QMP commands ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qmp-marshal.c', 'qmp-commands.h', - blurb, __doc__) - -fdef.write(mcgen(''' +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) +genc.body(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" #include "qemu/module.h"@@ -279,21 +277,24 @@ fdef.write(mcgen(''' #include "%(prefix)sqmp-commands.h" ''', - prefix=prefix)) + prefix=prefix)) -fdecl.write(mcgen(''' +genh.body(mcgen(''' #include "%(prefix)sqapi-types.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/dispatch.h" void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds); ''', - prefix=prefix, c_prefix=c_name(prefix, protect=False))) + prefix=prefix, c_prefix=c_name(prefix, protect=False))) schema = QAPISchema(input_file) -gen = QAPISchemaGenCommandVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenCommandVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qmp-marshal.c') +if do_h: + genh.write(output_dir, prefix + 'qmp-commands.h')scripts/qapi-event.py +14
-12@@ -176,11 +176,10 @@ blurb = ''' * Schema-defined QAPI/QMP events ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qapi-event.c', 'qapi-event.h', - blurb, __doc__) +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) -fdef.write(mcgen(''' +genc.body(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" #include "%(prefix)sqapi-event.h"@@ -190,22 +189,25 @@ fdef.write(mcgen(''' #include "qapi/qmp-event.h" ''', - prefix=prefix)) + prefix=prefix)) -fdecl.write(mcgen(''' +genh.body(mcgen(''' #include "qapi/util.h" #include "qapi/qmp/qdict.h" #include "%(prefix)sqapi-types.h" ''', - prefix=prefix)) + prefix=prefix)) event_enum_name = c_name(prefix + 'QAPIEvent', protect=False) schema = QAPISchema(input_file) -gen = QAPISchemaGenEventVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenEventVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qapi-event.c') +if do_h: + genh.write(output_dir, prefix + 'qapi-event.h')scripts/qapi-introspect.py +12
-10@@ -181,21 +181,23 @@ blurb = ''' * QAPI/QMP schema introspection ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qmp-introspect.c', 'qmp-introspect.h', - blurb, __doc__) +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) -fdef.write(mcgen(''' +genc.body(mcgen(''' #include "qemu/osdep.h" #include "%(prefix)sqmp-introspect.h" ''', - prefix=prefix)) + prefix=prefix)) schema = QAPISchema(input_file) -gen = QAPISchemaGenIntrospectVisitor(opt_unmask) -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenIntrospectVisitor(opt_unmask) +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qmp-introspect.c') +if do_h: + genh.write(output_dir, prefix + 'qmp-introspect.h')scripts/qapi-types.py +14
-12@@ -180,7 +180,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self.decl = '' self.defn = '' self._fwdecl = '' - self._btin = guardstart('QAPI_TYPES_BUILTIN') + self._btin = '\n' + guardstart('QAPI_TYPES_BUILTIN') def visit_end(self): self.decl = self._fwdecl + self.decl@@ -256,26 +256,28 @@ blurb = ''' * Schema-defined QAPI types ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qapi-types.c', 'qapi-types.h', - blurb, __doc__) +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) -fdef.write(mcgen(''' +genc.body(mcgen(''' #include "qemu/osdep.h" #include "qapi/dealloc-visitor.h" #include "%(prefix)sqapi-types.h" #include "%(prefix)sqapi-visit.h" ''', - prefix=prefix)) + prefix=prefix)) -fdecl.write(mcgen(''' +genh.body(mcgen(''' #include "qapi/util.h" ''')) schema = QAPISchema(input_file) -gen = QAPISchemaGenTypeVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenTypeVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qapi-types.c') +if do_h: + genh.write(output_dir, prefix + 'qapi-types.h')scripts/qapi-visit.py +14
-12@@ -339,30 +339,32 @@ blurb = ''' * Schema-defined QAPI visitors ''' -(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix, - 'qapi-visit.c', 'qapi-visit.h', - blurb, __doc__) +genc = QAPIGenC(blurb, __doc__) +genh = QAPIGenH(blurb, __doc__) -fdef.write(mcgen(''' +genc.body(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" #include "qapi/error.h" #include "%(prefix)sqapi-visit.h" ''', - prefix=prefix)) + prefix=prefix)) -fdecl.write(mcgen(''' +genh.body(mcgen(''' #include "qapi/visitor.h" #include "qapi/qmp/qerror.h" #include "%(prefix)sqapi-types.h" ''', - prefix=prefix)) + prefix=prefix)) schema = QAPISchema(input_file) -gen = QAPISchemaGenVisitVisitor() -schema.visit(gen) -fdef.write(gen.defn) -fdecl.write(gen.decl) +vis = QAPISchemaGenVisitVisitor() +schema.visit(vis) +genc.body(vis.defn) +genh.body(vis.decl) -close_output(fdef, fdecl) +if do_c: + genc.write(output_dir, prefix + 'qapi-visit.c') +if do_h: + genh.write(output_dir, prefix + 'qapi-visit.h')scripts/qapi.py +16
-4@@ -2,7 +2,7 @@ # QAPI helper library # # Copyright IBM, Corp. 2011 -# Copyright (c) 2013-2016 Red Hat Inc. +# Copyright (c) 2013-2018 Red Hat Inc. # # Authors: # Anthony Liguori <[email protected]>@@ -1820,7 +1820,6 @@ def guardname(filename): def guardstart(name): return mcgen(''' - #ifndef %(name)s #define %(name)s@@ -1832,7 +1831,6 @@ def guardend(name): return mcgen(''' #endif /* %(name)s */ - ''', name=guardname(name))@@ -1970,17 +1968,53 @@ def parse_command_line(extra_options='', extra_long_options=[]): return (fname, output_dir, do_c, do_h, prefix, extra_opts) + # -# Generate output files with boilerplate +# Accumulate and write output # +class QAPIGen(object): + + def __init__(self): + self._preamble = '' + self._body = '' + + def preamble(self, text): + self._preamble += text + + def body(self, text): + self._body += text + + def top(self, fname):
Quoted 8 lines
scripts/qapi.py def parse_command_line(extra_options='', extra_long_options=[]): · line 1993
Quoted 5 lines
scripts/qapi.py +5
-0+ return '' + + def bottom(self, fname): + return '' +Some methods appends, other return. That's a bit confusing. Why not name them accordingly?
add_preamble,add_body,get_top...?
You're right, the functions called for side effects have less than obvious names. I think I'll keep the noun names for the pure functions.
Quoted 82 lines
Quoted 82 lines
scripts/qapi.py +30
-38+ def write(self, output_dir, fname): + if output_dir: + try: + os.makedirs(output_dir) + except os.error as e: + if e.errno != errno.EEXIST: + raise + f = open(os.path.join(output_dir, fname), 'w') + f.write(self.top(fname) + self._preamble + self._body + + self.bottom(fname)) + f.close() + + +class QAPIGenC(QAPIGen): + + def __init__(self, blurb, pydoc): + QAPIGen.__init__(self) + self._blurb = blurb.strip('\n') + self._copyright = '\n * '.join(re.findall(r'^Copyright .*', pydoc, + re.MULTILINE)) -def open_output(output_dir, do_c, do_h, prefix, c_file, h_file, blurb, doc): - guard = guardname(prefix + h_file) - c_file = output_dir + prefix + c_file - h_file = output_dir + prefix + h_file - copyright = '\n * '.join(re.findall(r'^Copyright .*', doc, re.MULTILINE)) - comment = mcgen('''/* AUTOMATICALLY GENERATED, DO NOT MODIFY */ + def top(self, fname): + return mcgen(''' +/* AUTOMATICALLY GENERATED, DO NOT MODIFY */ /* %(blurb)s@@ -1992,41 +2026,19 @@ def open_output(output_dir, do_c, do_h, prefix, c_file, h_file, blurb, doc): */ ''', - blurb=blurb.strip('\n'), copyright=copyright) + blurb=self._blurb, copyright=self._copyright) - if output_dir: - try: - os.makedirs(output_dir) - except os.error as e: - if e.errno != errno.EEXIST: - raise - def maybe_open(really, name, opt): - if really: - return open(name, opt) - else: - import StringIO - return StringIO.StringIO() +class QAPIGenH(QAPIGenC): - fdef = maybe_open(do_c, c_file, 'w') - fdecl = maybe_open(do_h, h_file, 'w') + def top(self, fname): + return QAPIGenC.top(self, fname) + guardstart(fname) - fdef.write(comment) - fdecl.write(comment) - fdecl.write(mcgen(''' -#ifndef %(guard)s -#define %(guard)s + def bottom(self, fname): + return guardend(fname) -''', - guard=guard)) - return (fdef, fdecl) - - -def close_output(fdef, fdecl): - fdecl.write(mcgen(''' - -#endif -''')) - fdecl.close() - fdef.close() +class QAPIGenDoc(QAPIGen):
Quoted 8 lines
Quoted 6 lines
scripts/qapi.py +3
-1+ def top(self, fname): + return (QAPIGen.top(self, fname) + + '@c AUTOMATICALLY GENERATED, DO NOT MODIFY\n\n') --2.13.6
Reviewed-by Marc-André Lureau <[email protected]>
Thanks!
RFC PATCH 4/21 qapi: Reduce use of global variables in generators some
In reply to Markus ArmbrusterIn preparation of the next commit, which will turn the generators into modules. These global variables will become local to main() then.
--- scripts/qapi-commands.py | 9 +++++---- scripts/qapi-event.py | 15 +++++++-------- scripts/qapi-introspect.py | 7 ++++--- scripts/qapi-types.py | 17 +++++++++-------- scripts/qapi-visit.py | 17 +++++++++-------- 5 files changed, 34 insertions(+), 31 deletions(-)
scripts/qapi-commands.py +5 -4
@@ -207,7 +207,7 @@ def gen_register_command(name, success_response): return ret -def gen_registry(registry): +def gen_registry(registry, prefix): ret = mcgen(''' void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds)
@@ -224,7 +224,8 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds) class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): - def __init__(self): + def __init__(self, prefix): + self._prefix = prefix self.decl = None self.defn = None self._regy = None
@@ -237,7 +238,7 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): self._visited_ret_types = set() def visit_end(self): - self.defn += gen_registry(self._regy) + self.defn += gen_registry(self._regy, self._prefix) self._regy = None self._visited_ret_types = None
@@ -289,7 +290,7 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds); prefix=prefix, c_prefix=c_name(prefix, protect=False))) schema = QAPISchema(input_file) -vis = QAPISchemaGenCommandVisitor() +vis = QAPISchemaGenCommandVisitor(prefix) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl)
scripts/qapi-event.py +7 -8
@@ -58,7 +58,7 @@ def gen_param_var(typ): return ret -def gen_event_send(name, arg_type, boxed): +def gen_event_send(name, arg_type, boxed, event_enum_name): # FIXME: Our declaration of local variables (and of 'errp' in the # parameter list) can collide with exploded members of the event's # data type passed in as parameters. If this collision ever hits in
@@ -149,7 +149,8 @@ out: class QAPISchemaGenEventVisitor(QAPISchemaVisitor): - def __init__(self): + def __init__(self, prefix): + self._enum_name = c_name(prefix + 'QAPIEvent', protect=False) self.decl = None self.defn = None self._event_names = None
@@ -160,13 +161,13 @@ class QAPISchemaGenEventVisitor(QAPISchemaVisitor): self._event_names = [] def visit_end(self): - self.decl += gen_enum(event_enum_name, self._event_names) - self.defn += gen_enum_lookup(event_enum_name, self._event_names) + self.decl += gen_enum(self._enum_name, self._event_names) + self.defn += gen_enum_lookup(self._enum_name, self._event_names) self._event_names = None def visit_event(self, name, info, arg_type, boxed): self.decl += gen_event_send_decl(name, arg_type, boxed) - self.defn += gen_event_send(name, arg_type, boxed) + self.defn += gen_event_send(name, arg_type, boxed, self._enum_name) self._event_names.append(name)
@@ -199,10 +200,8 @@ genh.body(mcgen(''' ''', prefix=prefix)) -event_enum_name = c_name(prefix + 'QAPIEvent', protect=False) - schema = QAPISchema(input_file) -vis = QAPISchemaGenEventVisitor() +vis = QAPISchemaGenEventVisitor(prefix) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl)
scripts/qapi-introspect.py +4 -3
@@ -41,7 +41,8 @@ def to_c_string(string): class QAPISchemaGenIntrospectVisitor(QAPISchemaVisitor): - def __init__(self, unmask): + def __init__(self, prefix, unmask): + self._prefix = prefix self._unmask = unmask self.defn = None self.decl = None
@@ -65,7 +66,7 @@ class QAPISchemaGenIntrospectVisitor(QAPISchemaVisitor): # generate C # TODO can generate awfully long lines jsons.extend(self._jsons) - name = c_name(prefix, protect=False) + 'qmp_schema_json' + name = c_name(self._prefix, protect=False) + 'qmp_schema_json' self.decl = mcgen(''' extern const char %(c_name)s[]; ''',
@@ -192,7 +193,7 @@ genc.body(mcgen(''' prefix=prefix)) schema = QAPISchema(input_file) -vis = QAPISchemaGenIntrospectVisitor(opt_unmask) +vis = QAPISchemaGenIntrospectVisitor(prefix, opt_unmask) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl)
scripts/qapi-types.py +9 -8
@@ -168,7 +168,8 @@ void qapi_free_%(c_name)s(%(c_name)s *obj) class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): - def __init__(self): + def __init__(self, opt_builtins): + self._opt_builtins = opt_builtins self.decl = None self.defn = None self._fwdecl = None
@@ -187,7 +188,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self._fwdecl = None # To avoid header dependency hell, we always generate # declarations for built-in types in our header files and - # simply guard them. See also do_builtins (command line + # simply guard them. See also opt_builtins (command line # option -b). self._btin += guardend('QAPI_TYPES_BUILTIN') self.decl = self._btin + self.decl
@@ -202,7 +203,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): # TODO use something cleaner than existence of info if not info: self._btin += gen_enum(name, values, prefix) - if do_builtins: + if self._opt_builtins: self.defn += gen_enum_lookup(name, values, prefix) else: self._fwdecl += gen_enum(name, values, prefix)
@@ -213,7 +214,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self._btin += gen_fwd_object_or_array(name) self._btin += gen_array(name, element_type) self._btin += gen_type_cleanup_decl(name) - if do_builtins: + if self._opt_builtins: self.defn += gen_type_cleanup(name) else: self._fwdecl += gen_fwd_object_or_array(name)
@@ -241,16 +242,16 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): # If you link code generated from multiple schemata, you want only one # instance of the code for built-in types. Generate it only when -# do_builtins, enabled by command line option -b. See also +# opt_builtins, enabled by command line option -b. See also # QAPISchemaGenTypeVisitor.visit_end(). -do_builtins = False +opt_builtins = False (input_file, output_dir, do_c, do_h, prefix, opts) = \ parse_command_line('b', ['builtins']) for o, a in opts: if o in ('-b', '--builtins'): - do_builtins = True + opt_builtins = True blurb = ''' * Schema-defined QAPI types
@@ -272,7 +273,7 @@ genh.body(mcgen(''' ''')) schema = QAPISchema(input_file) -vis = QAPISchemaGenTypeVisitor() +vis = QAPISchemaGenTypeVisitor(opt_builtins) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl)
scripts/qapi-visit.py +9 -8
@@ -264,7 +264,8 @@ out: class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): - def __init__(self): + def __init__(self, opt_builtins): + self._opt_builtins = opt_builtins self.decl = None self.defn = None self._btin = None
@@ -277,7 +278,7 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): def visit_end(self): # To avoid header dependency hell, we always generate # declarations for built-in types in our header files and - # simply guard them. See also do_builtins (command line + # simply guard them. See also opt_builtins (command line # option -b). self._btin += guardend('QAPI_VISIT_BUILTIN') self.decl = self._btin + self.decl
@@ -288,7 +289,7 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): # TODO use something cleaner than existence of info if not info: self._btin += gen_visit_decl(name, scalar=True) - if do_builtins: + if self._opt_builtins: self.defn += gen_visit_enum(name) else: self.decl += gen_visit_decl(name, scalar=True)
@@ -299,7 +300,7 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): defn = gen_visit_list(name, element_type) if isinstance(element_type, QAPISchemaBuiltinType): self._btin += decl - if do_builtins: + if self._opt_builtins: self.defn += defn else: self.decl += decl
@@ -324,16 +325,16 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): # If you link code generated from multiple schemata, you want only one # instance of the code for built-in types. Generate it only when -# do_builtins, enabled by command line option -b. See also +# opt_builtins, enabled by command line option -b. See also # QAPISchemaGenVisitVisitor.visit_end(). -do_builtins = False +opt_builtins = False (input_file, output_dir, do_c, do_h, prefix, opts) = \ parse_command_line('b', ['builtins']) for o, a in opts: if o in ('-b', '--builtins'): - do_builtins = True + opt_builtins = True blurb = ''' * Schema-defined QAPI visitors
@@ -359,7 +360,7 @@ genh.body(mcgen(''' prefix=prefix)) schema = QAPISchema(input_file) -vis = QAPISchemaGenVisitVisitor() +vis = QAPISchemaGenVisitVisitor(opt_builtins) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl)
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 04/21] qapi: Reduce use of global variables in generators some
Quoted 5 lines · Markus Armbruster
In preparation of the next commit, which will turn the generators into modules. These global variables will become local to
main()then.Signed-off-by Markus Armbruster <[email protected]>---
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 04/21] qapi: Reduce use of global variables in generators some
In reply to Markus ArmbrusterQuoted 4 lines · Markus Armbruster
In preparation of the next commit, which will turn the generators into modules. These global variables will become local to
main()then.Signed-off-by Markus Armbruster <[email protected]>
Quoted 276 lines
--- scripts/qapi-commands.py | 9 +++++---- scripts/qapi-event.py | 15 +++++++-------- scripts/qapi-introspect.py | 7 ++++--- scripts/qapi-types.py | 17 +++++++++-------- scripts/qapi-visit.py | 17 +++++++++-------- 5 files changed, 34 insertions(+), 31 deletions(-)scripts/qapi-commands.py +5
-4@@ -207,7 +207,7 @@ def gen_register_command(name, success_response): return ret -def gen_registry(registry): +def gen_registry(registry, prefix): ret = mcgen(''' void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds)@@ -224,7 +224,8 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds) class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): - def __init__(self): + def __init__(self, prefix): + self._prefix = prefix self.decl = None self.defn = None self._regy = None@@ -237,7 +238,7 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): self._visited_ret_types = set() def visit_end(self): - self.defn += gen_registry(self._regy) + self.defn += gen_registry(self._regy, self._prefix) self._regy = None self._visited_ret_types = None@@ -289,7 +290,7 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds); prefix=prefix, c_prefix=c_name(prefix, protect=False))) schema = QAPISchema(input_file) -vis = QAPISchemaGenCommandVisitor() +vis = QAPISchemaGenCommandVisitor(prefix) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl)scripts/qapi-event.py +7
-8@@ -58,7 +58,7 @@ def gen_param_var(typ): return ret -def gen_event_send(name, arg_type, boxed): +def gen_event_send(name, arg_type, boxed, event_enum_name): # FIXME: Our declaration of local variables (and of 'errp' in the # parameter list) can collide with exploded members of the event's # data type passed in as parameters. If this collision ever hits in@@ -149,7 +149,8 @@ out: class QAPISchemaGenEventVisitor(QAPISchemaVisitor): - def __init__(self): + def __init__(self, prefix): + self._enum_name = c_name(prefix + 'QAPIEvent', protect=False) self.decl = None self.defn = None self._event_names = None@@ -160,13 +161,13 @@ class QAPISchemaGenEventVisitor(QAPISchemaVisitor): self._event_names = [] def visit_end(self): - self.decl += gen_enum(event_enum_name, self._event_names) - self.defn += gen_enum_lookup(event_enum_name, self._event_names) + self.decl += gen_enum(self._enum_name, self._event_names) + self.defn += gen_enum_lookup(self._enum_name, self._event_names) self._event_names = None def visit_event(self, name, info, arg_type, boxed): self.decl += gen_event_send_decl(name, arg_type, boxed) - self.defn += gen_event_send(name, arg_type, boxed) + self.defn += gen_event_send(name, arg_type, boxed, self._enum_name) self._event_names.append(name)@@ -199,10 +200,8 @@ genh.body(mcgen(''' ''', prefix=prefix)) -event_enum_name = c_name(prefix + 'QAPIEvent', protect=False) - schema = QAPISchema(input_file) -vis = QAPISchemaGenEventVisitor() +vis = QAPISchemaGenEventVisitor(prefix) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl)scripts/qapi-introspect.py +4
-3@@ -41,7 +41,8 @@ def to_c_string(string): class QAPISchemaGenIntrospectVisitor(QAPISchemaVisitor): - def __init__(self, unmask): + def __init__(self, prefix, unmask): + self._prefix = prefix self._unmask = unmask self.defn = None self.decl = None@@ -65,7 +66,7 @@ class QAPISchemaGenIntrospectVisitor(QAPISchemaVisitor): # generate C # TODO can generate awfully long lines jsons.extend(self._jsons) - name = c_name(prefix, protect=False) + 'qmp_schema_json' + name = c_name(self._prefix, protect=False) + 'qmp_schema_json' self.decl = mcgen(''' extern const char %(c_name)s[]; ''',@@ -192,7 +193,7 @@ genc.body(mcgen(''' prefix=prefix)) schema = QAPISchema(input_file) -vis = QAPISchemaGenIntrospectVisitor(opt_unmask) +vis = QAPISchemaGenIntrospectVisitor(prefix, opt_unmask) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl)scripts/qapi-types.py +9
-8@@ -168,7 +168,8 @@ void qapi_free_%(c_name)s(%(c_name)s *obj) class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): - def __init__(self): + def __init__(self, opt_builtins): + self._opt_builtins = opt_builtins self.decl = None self.defn = None self._fwdecl = None@@ -187,7 +188,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self._fwdecl = None # To avoid header dependency hell, we always generate # declarations for built-in types in our header files and - # simply guard them. See also do_builtins (command line + # simply guard them. See also opt_builtins (command line # option -b). self._btin += guardend('QAPI_TYPES_BUILTIN') self.decl = self._btin + self.decl@@ -202,7 +203,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): # TODO use something cleaner than existence of info if not info: self._btin += gen_enum(name, values, prefix) - if do_builtins: + if self._opt_builtins: self.defn += gen_enum_lookup(name, values, prefix) else: self._fwdecl += gen_enum(name, values, prefix)@@ -213,7 +214,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self._btin += gen_fwd_object_or_array(name) self._btin += gen_array(name, element_type) self._btin += gen_type_cleanup_decl(name) - if do_builtins: + if self._opt_builtins: self.defn += gen_type_cleanup(name) else: self._fwdecl += gen_fwd_object_or_array(name)@@ -241,16 +242,16 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): # If you link code generated from multiple schemata, you want only one # instance of the code for built-in types. Generate it only when -# do_builtins, enabled by command line option -b. See also +# opt_builtins, enabled by command line option -b. See also # QAPISchemaGenTypeVisitor.visit_end(). -do_builtins = False +opt_builtins = False (input_file, output_dir, do_c, do_h, prefix, opts) = \ parse_command_line('b', ['builtins']) for o, a in opts: if o in ('-b', '--builtins'): - do_builtins = True + opt_builtins = True blurb = ''' * Schema-defined QAPI types@@ -272,7 +273,7 @@ genh.body(mcgen(''' ''')) schema = QAPISchema(input_file) -vis = QAPISchemaGenTypeVisitor() +vis = QAPISchemaGenTypeVisitor(opt_builtins) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl)scripts/qapi-visit.py +9
-9@@ -264,7 +264,8 @@ out: class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): - def __init__(self): + def __init__(self, opt_builtins): + self._opt_builtins = opt_builtins self.decl = None self.defn = None self._btin = None@@ -277,7 +278,7 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): def visit_end(self): # To avoid header dependency hell, we always generate # declarations for built-in types in our header files and - # simply guard them. See also do_builtins (command line + # simply guard them. See also opt_builtins (command line # option -b). self._btin += guardend('QAPI_VISIT_BUILTIN') self.decl = self._btin + self.decl@@ -288,7 +289,7 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): # TODO use something cleaner than existence of info if not info: self._btin += gen_visit_decl(name, scalar=True) - if do_builtins: + if self._opt_builtins: self.defn += gen_visit_enum(name) else: self.decl += gen_visit_decl(name, scalar=True)@@ -299,7 +300,7 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): defn = gen_visit_list(name, element_type) if isinstance(element_type, QAPISchemaBuiltinType): self._btin += decl - if do_builtins: + if self._opt_builtins: self.defn += defn else: self.decl += decl@@ -324,16 +325,16 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): # If you link code generated from multiple schemata, you want only one # instance of the code for built-in types. Generate it only when -# do_builtins, enabled by command line option -b. See also +# opt_builtins, enabled by command line option -b. See also # QAPISchemaGenVisitVisitor.visit_end(). -do_builtins = False +opt_builtins = False (input_file, output_dir, do_c, do_h, prefix, opts) = \ parse_command_line('b', ['builtins']) for o, a in opts: if o in ('-b', '--builtins'): - do_builtins = True + opt_builtins = True blurb = ''' * Schema-defined QAPI visitors@@ -359,7 +360,7 @@ genh.body(mcgen(''' prefix=prefix)) schema = QAPISchema(input_file) -vis = QAPISchemaGenVisitVisitor() +vis = QAPISchemaGenVisitVisitor(opt_builtins) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) --2.13.6
RFC PATCH 5/21 qapi: Turn generators into modules
In reply to Markus ArmbrusterThe next commit will introduce a common driver program for all generators. The generators need to be modules for that. qapi2texi.py already is. Make the other generators follow suit.
The changes are actually trivial. Obvious in the diffs once you view them with whitespace changes ignored.
--- scripts/qapi-commands.py | 43 ++++++++++++++++++-------------- scripts/qapi-event.py | 43 ++++++++++++++++++-------------- scripts/qapi-introspect.py | 54 ++++++++++++++++++++++------------------ scripts/qapi-types.py | 56 ++++++++++++++++++++++------------------- scripts/qapi-visit.py | 62 +++++++++++++++++++++++++--------------------- 5 files changed, 143 insertions(+), 115 deletions(-)
scripts/qapi-commands.py +24 -19
@@ -255,16 +255,17 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): self._regy += gen_register_command(name, success_response) -(input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line() +def main(argv): + (input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line() -blurb = ''' + blurb = ''' * Schema-defined QAPI/QMP commands ''' -genc = QAPIGenC(blurb, __doc__) -genh = QAPIGenH(blurb, __doc__) + genc = QAPIGenC(blurb, __doc__) + genh = QAPIGenH(blurb, __doc__) -genc.body(mcgen(''' + genc.body(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" #include "qemu/module.h"
@@ -278,24 +279,28 @@ genc.body(mcgen(''' #include "%(prefix)sqmp-commands.h" ''', - prefix=prefix)) + prefix=prefix)) -genh.body(mcgen(''' + genh.body(mcgen(''' #include "%(prefix)sqapi-types.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/dispatch.h" void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds); ''', - prefix=prefix, c_prefix=c_name(prefix, protect=False))) - -schema = QAPISchema(input_file) -vis = QAPISchemaGenCommandVisitor(prefix) -schema.visit(vis) -genc.body(vis.defn) -genh.body(vis.decl) - -if do_c: - genc.write(output_dir, prefix + 'qmp-marshal.c') -if do_h: - genh.write(output_dir, prefix + 'qmp-commands.h') + prefix=prefix, c_prefix=c_name(prefix, protect=False))) + + schema = QAPISchema(input_file) + vis = QAPISchemaGenCommandVisitor(prefix) + schema.visit(vis) + genc.body(vis.defn) + genh.body(vis.decl) + + if do_c: + genc.write(output_dir, prefix + 'qmp-marshal.c') + if do_h: + genh.write(output_dir, prefix + 'qmp-commands.h') + + +if __name__ == '__main__': + main(sys.argv)
scripts/qapi-event.py +24 -19
63 lines not on this page · raw message
scripts/qapi-introspect.py +30 -24
68 lines not on this page · raw message
scripts/qapi-types.py +31 -25
76 lines not on this page · raw message
scripts/qapi-visit.py +34 -28
83 lines not on this page · raw message
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 05/21] qapi: Turn generators into modules
Quoted 7 lines · Markus Armbruster
The next commit will introduce a common driver program for all generators. The generators need to be modules for that.
qapi2texi.pyalready is. Make the other generators follow suit.The changes are actually trivial. Obvious in the diffs once you view them with whitespace changes ignored.
Quoted 9 lines
Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi-commands.py | 43 ++++++++++++++++++-------------- scripts/qapi-event.py | 43 ++++++++++++++++++-------------- scripts/qapi-introspect.py | 54 ++++++++++++++++++++++------------------ scripts/qapi-types.py | 56 ++++++++++++++++++++++------------------- scripts/qapi-visit.py | 62 +++++++++++++++++++++++++--------------------- 5 files changed, 143 insertions(+), 115 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 05/21] qapi: Turn generators into modules
In reply to Markus ArmbrusterQuoted 8 lines · Markus Armbruster
The next commit will introduce a common driver program for all generators. The generators need to be modules for that.
qapi2texi.pyalready is. Make the other generators follow suit.The changes are actually trivial. Obvious in the diffs once you view them with whitespace changes ignored.
Signed-off-by Markus Armbruster <[email protected]>
Quoted 392 lines
--- scripts/qapi-commands.py | 43 ++++++++++++++++++-------------- scripts/qapi-event.py | 43 ++++++++++++++++++-------------- scripts/qapi-introspect.py | 54 ++++++++++++++++++++++------------------ scripts/qapi-types.py | 56 ++++++++++++++++++++++------------------- scripts/qapi-visit.py | 62 +++++++++++++++++++++++++--------------------- 5 files changed, 143 insertions(+), 115 deletions(-)scripts/qapi-commands.py +24
-19@@ -255,16 +255,17 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): self._regy += gen_register_command(name, success_response) -(input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line() +def main(argv): + (input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line() -blurb = ''' + blurb = ''' * Schema-defined QAPI/QMP commands ''' -genc = QAPIGenC(blurb, __doc__) -genh = QAPIGenH(blurb, __doc__) + genc = QAPIGenC(blurb, __doc__) + genh = QAPIGenH(blurb, __doc__) -genc.body(mcgen(''' + genc.body(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" #include "qemu/module.h"@@ -278,24 +279,28 @@ genc.body(mcgen(''' #include "%(prefix)sqmp-commands.h" ''', - prefix=prefix)) + prefix=prefix)) -genh.body(mcgen(''' + genh.body(mcgen(''' #include "%(prefix)sqapi-types.h" #include "qapi/qmp/qdict.h" #include "qapi/qmp/dispatch.h" void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds); ''', - prefix=prefix, c_prefix=c_name(prefix, protect=False))) - -schema = QAPISchema(input_file) -vis = QAPISchemaGenCommandVisitor(prefix) -schema.visit(vis) -genc.body(vis.defn) -genh.body(vis.decl) - -if do_c: - genc.write(output_dir, prefix + 'qmp-marshal.c') -if do_h: - genh.write(output_dir, prefix + 'qmp-commands.h') + prefix=prefix, c_prefix=c_name(prefix, protect=False))) + + schema = QAPISchema(input_file) + vis = QAPISchemaGenCommandVisitor(prefix) + schema.visit(vis) + genc.body(vis.defn) + genh.body(vis.decl) + + if do_c: + genc.write(output_dir, prefix + 'qmp-marshal.c') + if do_h: + genh.write(output_dir, prefix + 'qmp-commands.h') + + +if __name__ == '__main__': + main(sys.argv)scripts/qapi-event.py +24
-19@@ -171,16 +171,17 @@ class QAPISchemaGenEventVisitor(QAPISchemaVisitor): self._event_names.append(name) -(input_file, output_dir, do_c, do_h, prefix, dummy) = parse_command_line() +def main(argv): + (input_file, output_dir, do_c, do_h, prefix, dummy) = parse_command_line() -blurb = ''' + blurb = ''' * Schema-defined QAPI/QMP events ''' -genc = QAPIGenC(blurb, __doc__) -genh = QAPIGenH(blurb, __doc__) + genc = QAPIGenC(blurb, __doc__) + genh = QAPIGenH(blurb, __doc__) -genc.body(mcgen(''' + genc.body(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" #include "%(prefix)sqapi-event.h"@@ -190,23 +191,27 @@ genc.body(mcgen(''' #include "qapi/qmp-event.h" ''', - prefix=prefix)) + prefix=prefix)) -genh.body(mcgen(''' + genh.body(mcgen(''' #include "qapi/util.h" #include "qapi/qmp/qdict.h" #include "%(prefix)sqapi-types.h" ''', - prefix=prefix)) - -schema = QAPISchema(input_file) -vis = QAPISchemaGenEventVisitor(prefix) -schema.visit(vis) -genc.body(vis.defn) -genh.body(vis.decl) - -if do_c: - genc.write(output_dir, prefix + 'qapi-event.c') -if do_h: - genh.write(output_dir, prefix + 'qapi-event.h') + prefix=prefix)) + + schema = QAPISchema(input_file) + vis = QAPISchemaGenEventVisitor(prefix) + schema.visit(vis) + genc.body(vis.defn) + genh.body(vis.decl) + + if do_c: + genc.write(output_dir, prefix + 'qapi-event.c') + if do_h: + genh.write(output_dir, prefix + 'qapi-event.h') + + +if __name__ == '__main__': + main(sys.argv)scripts/qapi-introspect.py +30
-24@@ -167,38 +167,44 @@ const char %(c_name)s[] = %(c_string)s; arg_type = arg_type or self._schema.the_empty_object_type self._gen_json(name, 'event', {'arg-type': self._use_type(arg_type)}) -# Debugging aid: unmask QAPI schema's type names -# We normally mask them, because they're not QMP wire ABI -opt_unmask = False -(input_file, output_dir, do_c, do_h, prefix, opts) = \ - parse_command_line('u', ['unmask-non-abi-names']) +def main(argv): + # Debugging aid: unmask QAPI schema's type names + # We normally mask them, because they're not QMP wire ABI + opt_unmask = False -for o, a in opts: - if o in ('-u', '--unmask-non-abi-names'): - opt_unmask = True + (input_file, output_dir, do_c, do_h, prefix, opts) = \ + parse_command_line('u', ['unmask-non-abi-names']) -blurb = ''' + for o, a in opts: + if o in ('-u', '--unmask-non-abi-names'): + opt_unmask = True + + blurb = ''' * QAPI/QMP schema introspection ''' -genc = QAPIGenC(blurb, __doc__) -genh = QAPIGenH(blurb, __doc__) + genc = QAPIGenC(blurb, __doc__) + genh = QAPIGenH(blurb, __doc__) -genc.body(mcgen(''' + genc.body(mcgen(''' #include "qemu/osdep.h" #include "%(prefix)sqmp-introspect.h" ''', - prefix=prefix)) - -schema = QAPISchema(input_file) -vis = QAPISchemaGenIntrospectVisitor(prefix, opt_unmask) -schema.visit(vis) -genc.body(vis.defn) -genh.body(vis.decl) - -if do_c: - genc.write(output_dir, prefix + 'qmp-introspect.c') -if do_h: - genh.write(output_dir, prefix + 'qmp-introspect.h') + prefix=prefix)) + + schema = QAPISchema(input_file) + vis = QAPISchemaGenIntrospectVisitor(prefix, opt_unmask) + schema.visit(vis) + genc.body(vis.defn) + genh.body(vis.decl) + + if do_c: + genc.write(output_dir, prefix + 'qmp-introspect.c') + if do_h: + genh.write(output_dir, prefix + 'qmp-introspect.h') + + +if __name__ == '__main__': + main(sys.argv)scripts/qapi-types.py +31
-25@@ -240,45 +240,51 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self.decl += gen_object(name, None, [variants.tag_member], variants) self._gen_type_cleanup(name) -# If you link code generated from multiple schemata, you want only one -# instance of the code for built-in types. Generate it only when -# opt_builtins, enabled by command line option -b. See also -# QAPISchemaGenTypeVisitor.visit_end(). -opt_builtins = False -(input_file, output_dir, do_c, do_h, prefix, opts) = \ - parse_command_line('b', ['builtins']) +def main(argv): + # If you link code generated from multiple schemata, you want only one + # instance of the code for built-in types. Generate it only when + # opt_builtins, enabled by command line option -b. See also + # QAPISchemaGenTypeVisitor.visit_end(). + opt_builtins = False -for o, a in opts: - if o in ('-b', '--builtins'): - opt_builtins = True + (input_file, output_dir, do_c, do_h, prefix, opts) = \ + parse_command_line('b', ['builtins']) -blurb = ''' + for o, a in opts: + if o in ('-b', '--builtins'): + opt_builtins = True + + blurb = ''' * Schema-defined QAPI types ''' -genc = QAPIGenC(blurb, __doc__) -genh = QAPIGenH(blurb, __doc__) + genc = QAPIGenC(blurb, __doc__) + genh = QAPIGenH(blurb, __doc__) -genc.body(mcgen(''' + genc.body(mcgen(''' #include "qemu/osdep.h" #include "qapi/dealloc-visitor.h" #include "%(prefix)sqapi-types.h" #include "%(prefix)sqapi-visit.h" ''', - prefix=prefix)) + prefix=prefix)) -genh.body(mcgen(''' + genh.body(mcgen(''' #include "qapi/util.h" ''')) -schema = QAPISchema(input_file) -vis = QAPISchemaGenTypeVisitor(opt_builtins) -schema.visit(vis) -genc.body(vis.defn) -genh.body(vis.decl) + schema = QAPISchema(input_file) + vis = QAPISchemaGenTypeVisitor(opt_builtins) + schema.visit(vis) + genc.body(vis.defn) + genh.body(vis.decl) -if do_c: - genc.write(output_dir, prefix + 'qapi-types.c') -if do_h: - genh.write(output_dir, prefix + 'qapi-types.h') + if do_c: + genc.write(output_dir, prefix + 'qapi-types.c') + if do_h: + genh.write(output_dir, prefix + 'qapi-types.h') + + +if __name__ == '__main__': + main(sys.argv)scripts/qapi-visit.py +34
-29@@ -323,49 +323,55 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): self.decl += gen_visit_decl(name) self.defn += gen_visit_alternate(name, variants) -# If you link code generated from multiple schemata, you want only one -# instance of the code for built-in types. Generate it only when -# opt_builtins, enabled by command line option -b. See also -# QAPISchemaGenVisitVisitor.visit_end(). -opt_builtins = False -(input_file, output_dir, do_c, do_h, prefix, opts) = \ - parse_command_line('b', ['builtins']) +def main(argv): + # If you link code generated from multiple schemata, you want only one + # instance of the code for built-in types. Generate it only when + # opt_builtins, enabled by command line option -b. See also + # QAPISchemaGenVisitVisitor.visit_end(). + opt_builtins = False -for o, a in opts: - if o in ('-b', '--builtins'): - opt_builtins = True + (input_file, output_dir, do_c, do_h, prefix, opts) = \ + parse_command_line('b', ['builtins']) -blurb = ''' + for o, a in opts: + if o in ('-b', '--builtins'): + opt_builtins = True + + blurb = ''' * Schema-defined QAPI visitors ''' -genc = QAPIGenC(blurb, __doc__) -genh = QAPIGenH(blurb, __doc__) + genc = QAPIGenC(blurb, __doc__) + genh = QAPIGenH(blurb, __doc__) -genc.body(mcgen(''' + genc.body(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" #include "qapi/error.h" #include "%(prefix)sqapi-visit.h" ''', - prefix=prefix)) + prefix=prefix)) -genh.body(mcgen(''' + genh.body(mcgen(''' #include "qapi/visitor.h" #include "qapi/qmp/qerror.h" #include "%(prefix)sqapi-types.h" ''', - prefix=prefix)) - -schema = QAPISchema(input_file) -vis = QAPISchemaGenVisitVisitor(opt_builtins) -schema.visit(vis) -genc.body(vis.defn) -genh.body(vis.decl) - -if do_c: - genc.write(output_dir, prefix + 'qapi-visit.c') -if do_h: - genh.write(output_dir, prefix + 'qapi-visit.h') + prefix=prefix)) + + schema = QAPISchema(input_file) + vis = QAPISchemaGenVisitVisitor(opt_builtins) + schema.visit(vis) + genc.body(vis.defn) + genh.body(vis.decl) + + if do_c: + genc.write(output_dir, prefix + 'qapi-visit.c') + if do_h: + genh.write(output_dir, prefix + 'qapi-visit.h') + + +if __name__ == '__main__': + main(sys.argv) --2.13.6
RFC PATCH 6/21 qapi-gen: New common driver for code and doc generators
In reply to Markus ArmbrusterWhenever qapi-schema.json changes, we run six programs eleven times to update eleven files. This is silly. Replace the six programs by a single program that spits out all eleven files.
---
Makefile | 86 ++++++++++------------
scripts/qapi-gen.py | 41 +++++++++++
scripts/qapi/__init__.py | 0
scripts/{qapi-commands.py => qapi/commands.py} | 23 ++----
scripts/{qapi.py => qapi/common.py} | 0
scripts/{qapi2texi.py => qapi/doc.py} | 29 ++------
scripts/{qapi-event.py => qapi/events.py} | 23 ++----
scripts/{qapi-introspect.py => qapi/introspect.py} | 32 ++------
scripts/{qapi-types.py => qapi/types.py} | 34 ++-------
scripts/{qapi-visit.py => qapi/visit.py} | 34 ++-------
tests/Makefile.include | 56 +++++++-------
tests/qapi-schema/test-qapi.py | 2 +-
12 files changed, 140 insertions(+), 220 deletions(-)
create mode 100755 scripts/qapi-gen.py
create mode 100644 scripts/qapi/__init__.py
rename scripts/{qapi-commands.py => qapi/commands.py} (94%)
rename scripts/{qapi.py => qapi/common.py} (100%)
rename scripts/{qapi2texi.py => qapi/doc.py} (92%)
mode change 100755 => 100644
rename scripts/{qapi-event.py => qapi/events.py} (92%)
rename scripts/{qapi-introspect.py => qapi/introspect.py} (90%)
rename scripts/{qapi-types.py => qapi/types.py} (90%)
rename scripts/{qapi-visit.py => qapi/visit.py} (92%)Makefile +37 -49
@@ -92,6 +92,7 @@ GENERATED_FILES += qmp-commands.h qapi-types.h qapi-visit.h qapi-event.h GENERATED_FILES += qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c GENERATED_FILES += qmp-introspect.h GENERATED_FILES += qmp-introspect.c +GENERATED_FILES += qapi.texi GENERATED_FILES += trace/generated-tcg-tracers.h
@@ -477,25 +478,26 @@ qemu-ga$(EXESUF): QEMU_CFLAGS += -I qga/qapi-generated qemu-keymap$(EXESUF): LIBS += $(XKBCOMMON_LIBS) qemu-keymap$(EXESUF): QEMU_CFLAGS += $(XKBCOMMON_CFLAGS) -gen-out-type = $(subst .,-,$(suffix $@)) +qapi-py = $(SRC_PATH)/scripts/qapi/commands.py \ +$(SRC_PATH)/scripts/qapi/events.py \ +$(SRC_PATH)/scripts/qapi/introspect.py \ +$(SRC_PATH)/scripts/qapi/types.py \ +$(SRC_PATH)/scripts/qapi/visit.py \ +$(SRC_PATH)/scripts/qapi/common.py \ +$(SRC_PATH)/scripts/qapi/doc.py \ +$(SRC_PATH)/scripts/ordereddict.py \ +$(SRC_PATH)/scripts/qapi-gen.py -qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py - -qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h :\ -$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \ - $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \ - "GEN","$@") -qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h :\ -$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \ - $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \ - "GEN","$@") -qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c :\ -$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \ - $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \ - "GEN","$@") +qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h \ +qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h \ +qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c \ +qga/qapi-generated/qga-qapi.texi: \ +qga/qapi-generated/qapi-gen-timestamp ; +qga/qapi-generated/qapi-gen-timestamp: $(SRC_PATH)/qga/qapi-schema.json $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o qga/qapi-generated -p "qga-" $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \ $(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
@@ -512,31 +514,18 @@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \ $(SRC_PATH)/qapi/transaction.json \ $(SRC_PATH)/qapi/ui.json -qapi-types.c qapi-types.h :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-types.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \ - $(gen-out-type) -o "." -b $<, \ - "GEN","$@") -qapi-visit.c qapi-visit.h :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \ - $(gen-out-type) -o "." -b $<, \ - "GEN","$@") -qapi-event.c qapi-event.h :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-event.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-event.py \ - $(gen-out-type) -o "." $<, \ - "GEN","$@") -qmp-commands.h qmp-marshal.c :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \ - $(gen-out-type) -o "." $<, \ - "GEN","$@") -qmp-introspect.h qmp-introspect.c :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-introspect.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-introspect.py \ - $(gen-out-type) -o "." $<, \ - "GEN","$@") +qapi-types.c qapi-types.h \ +qapi-visit.c qapi-visit.h \ +qmp-commands.h qmp-marshal.c \ +qapi-event.c qapi-event.h \ +qmp-introspect.h qmp-introspect.c \ +qapi.texi: \ +qapi-gen-timestamp ; +qapi-gen-timestamp: $(qapi-modules) $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o "." -b $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@ QGALIB_GEN=$(addprefix qga/qapi-generated/, qga-qapi-types.h qga-qapi-visit.h qga-qmp-commands.h) $(qga-obj-y): $(QGALIB_GEN)
@@ -596,6 +585,7 @@ clean: rm -f trace/generated-tracers-dtrace.dtrace* rm -f trace/generated-tracers-dtrace.h* rm -f $(foreach f,$(GENERATED_FILES),$(f) $(f)-timestamp) + rm -f qapi-gen-timestamp rm -rf qapi-generated rm -rf qga/qapi-generated for d in $(ALL_SUBDIRS); do \
@@ -803,13 +793,11 @@ qemu-monitor-info.texi: $(SRC_PATH)/hmp-commands-info.hx $(SRC_PATH)/scripts/hxt qemu-img-cmds.texi: $(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/scripts/hxtool $(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > $@,"GEN","$@") -docs/interop/qemu-qmp-qapi.texi docs/interop/qemu-ga-qapi.texi: $(SRC_PATH)/scripts/qapi2texi.py $(qapi-py) +docs/interop/qemu-qmp-qapi.texi: qapi.texi + @cp -p $< $@ -docs/interop/qemu-qmp-qapi.texi: $(qapi-modules) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi2texi.py $< > $@,"GEN","$@") - -docs/interop/qemu-ga-qapi.texi: $(SRC_PATH)/qga/qapi-schema.json - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi2texi.py $< > $@,"GEN","$@") +docs/interop/qemu-ga-qapi.texi: qga/qapi-generated/qga-qapi.texi + @cp -p $< $@ qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi qemu-monitor-info.texi qemu.1: qemu-option-trace.texi
scripts/qapi-gen.py new file +41 -0
41 lines not on this page · raw message
scripts/qapi/__init__.py new file +0 -0
scripts/qapi/commands.py renamed from scripts/qapi-commands.py +5 -18
@@ -13,7 +13,7 @@ This work is licensed under the terms of the GNU GPL, version 2. See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * def gen_command_decl(name, arg_type, boxed, ret_type):
@@ -255,13 +255,8 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): self._regy += gen_register_command(name, success_response) -def main(argv): - (input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line() - - blurb = ''' - * Schema-defined QAPI/QMP commands -''' - +def gen_commands(schema, output_dir, prefix): + blurb = ' * Schema-defined QAPI/QMP commands' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)
@@ -290,17 +285,9 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds); ''', prefix=prefix, c_prefix=c_name(prefix, protect=False))) - schema = QAPISchema(input_file) vis = QAPISchemaGenCommandVisitor(prefix) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qmp-marshal.c') - if do_h: - genh.write(output_dir, prefix + 'qmp-commands.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qmp-marshal.c') + genh.write(output_dir, prefix + 'qmp-commands.h')
scripts/qapi/common.py renamed from scripts/qapi.py +0 -0
scripts/qapi/doc.py renamed from scripts/qapi2texi.py +8 -21
@@ -4,10 +4,9 @@ # This work is licensed under the terms of the GNU LGPL, version 2+. # See the COPYING file in the top-level directory. """This script produces the documentation of a qapi schema in texinfo format""" + import re -import sys - -import qapi +import qapi.common MSG_FMT = """ @deftypefn {type} {{}} {name}
@@ -196,7 +195,7 @@ def texi_entity(doc, what, base=None, variants=None, + texi_sections(doc)) -class QAPISchemaGenDocVisitor(qapi.QAPISchemaVisitor): +class QAPISchemaGenDocVisitor(qapi.common.QAPISchemaVisitor): def __init__(self): self.out = None self.cur_doc = None
@@ -271,20 +270,8 @@ def texi_schema(schema): return gen.out -def main(argv): - """Takes schema argument, prints result to stdout""" - if len(argv) != 2: - print >>sys.stderr, "%s: need exactly 1 argument: SCHEMA" % argv[0] - sys.exit(1) - - schema = qapi.QAPISchema(argv[1]) - if not qapi.doc_required: - print >>sys.stderr, ("%s: need pragma 'doc-required' " - "to generate documentation" % argv[0]) - sys.exit(1) - print '@c AUTOMATICALLY GENERATED, DO NOT MODIFY\n' - print texi_schema(schema), - - -if __name__ == '__main__': - main(sys.argv) +def gen_doc(schema, output_dir, prefix): + if qapi.common.doc_required: + gen = qapi.common.QAPIGenDoc() + gen.body(texi_schema(schema)) + gen.write(output_dir, prefix + 'qapi.texi')
scripts/qapi/events.py renamed from scripts/qapi-event.py +5 -18
@@ -12,7 +12,7 @@ This work is licensed under the terms of the GNU GPL, version 2. See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * def build_event_send_proto(name, arg_type, boxed):
@@ -171,13 +171,8 @@ class QAPISchemaGenEventVisitor(QAPISchemaVisitor): self._event_names.append(name) -def main(argv): - (input_file, output_dir, do_c, do_h, prefix, dummy) = parse_command_line() - - blurb = ''' - * Schema-defined QAPI/QMP events -''' - +def gen_events(schema, output_dir, prefix): + blurb = ' * Schema-defined QAPI/QMP events' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)
@@ -201,17 +196,9 @@ def main(argv): ''', prefix=prefix)) - schema = QAPISchema(input_file) vis = QAPISchemaGenEventVisitor(prefix) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qapi-event.c') - if do_h: - genh.write(output_dir, prefix + 'qapi-event.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qapi-event.c') + genh.write(output_dir, prefix + 'qapi-event.h')
scripts/qapi/introspect.py renamed from scripts/qapi-introspect.py +5 -27
@@ -10,7 +10,7 @@ This work is licensed under the terms of the GNU GPL, version 2. See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * # Caveman's json.dumps() replacement (we're stuck at Python 2.4)
@@ -168,22 +168,8 @@ const char %(c_name)s[] = %(c_string)s; self._gen_json(name, 'event', {'arg-type': self._use_type(arg_type)}) -def main(argv): - # Debugging aid: unmask QAPI schema's type names - # We normally mask them, because they're not QMP wire ABI - opt_unmask = False - - (input_file, output_dir, do_c, do_h, prefix, opts) = \ - parse_command_line('u', ['unmask-non-abi-names']) - - for o, a in opts: - if o in ('-u', '--unmask-non-abi-names'): - opt_unmask = True - - blurb = ''' - * QAPI/QMP schema introspection -''' - +def gen_introspect(schema, output_dir, prefix, opt_unmask): + blurb = ' * QAPI/QMP schema introspection' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)
@@ -194,17 +180,9 @@ def main(argv): ''', prefix=prefix)) - schema = QAPISchema(input_file) vis = QAPISchemaGenIntrospectVisitor(prefix, opt_unmask) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qmp-introspect.c') - if do_h: - genh.write(output_dir, prefix + 'qmp-introspect.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qmp-introspect.c') + genh.write(output_dir, prefix + 'qmp-introspect.h')
scripts/qapi/types.py renamed from scripts/qapi-types.py +5 -29
@@ -13,7 +13,7 @@ This work is licensed under the terms of the GNU GPL, version 2. # See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * # variants must be emitted before their container; track what has already
@@ -241,24 +241,8 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self._gen_type_cleanup(name) -def main(argv): - # If you link code generated from multiple schemata, you want only one - # instance of the code for built-in types. Generate it only when - # opt_builtins, enabled by command line option -b. See also - # QAPISchemaGenTypeVisitor.visit_end(). - opt_builtins = False - - (input_file, output_dir, do_c, do_h, prefix, opts) = \ - parse_command_line('b', ['builtins']) - - for o, a in opts: - if o in ('-b', '--builtins'): - opt_builtins = True - - blurb = ''' - * Schema-defined QAPI types -''' - +def gen_types(schema, output_dir, prefix, opt_builtins): + blurb = ' * Schema-defined QAPI types' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)
@@ -274,17 +258,9 @@ def main(argv): #include "qapi/util.h" ''')) - schema = QAPISchema(input_file) vis = QAPISchemaGenTypeVisitor(opt_builtins) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qapi-types.c') - if do_h: - genh.write(output_dir, prefix + 'qapi-types.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qapi-types.c') + genh.write(output_dir, prefix + 'qapi-types.h')
scripts/qapi/visit.py renamed from scripts/qapi-visit.py +5 -29
@@ -13,7 +13,7 @@ This work is licensed under the terms of the GNU GPL, version 2. See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * def gen_visit_decl(name, scalar=False):
@@ -324,24 +324,8 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): self.defn += gen_visit_alternate(name, variants) -def main(argv): - # If you link code generated from multiple schemata, you want only one - # instance of the code for built-in types. Generate it only when - # opt_builtins, enabled by command line option -b. See also - # QAPISchemaGenVisitVisitor.visit_end(). - opt_builtins = False - - (input_file, output_dir, do_c, do_h, prefix, opts) = \ - parse_command_line('b', ['builtins']) - - for o, a in opts: - if o in ('-b', '--builtins'): - opt_builtins = True - - blurb = ''' - * Schema-defined QAPI visitors -''' - +def gen_visit(schema, output_dir, prefix, opt_builtins): + blurb = ' * Schema-defined QAPI visitors' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)
@@ -361,17 +345,9 @@ def main(argv): ''', prefix=prefix)) - schema = QAPISchema(input_file) vis = QAPISchemaGenVisitVisitor(opt_builtins) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qapi-visit.c') - if do_h: - genh.write(output_dir, prefix + 'qapi-visit.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qapi-visit.c') + genh.write(output_dir, prefix + 'qapi-visit.h')
tests/Makefile.include +28 -28
75 lines not on this page · raw message
tests/qapi-schema/test-qapi.py +1 -1
@@ -10,7 +10,7 @@ # See the COPYING file in the top-level directory. # -from qapi import * +from qapi.common import * from pprint import pprint import os import sys
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 06/21] qapi-gen: New common driver for code and doc generators
Quoted 3 lines · Markus Armbruster
Whenever qapi-schema.json changes, we run six programs eleven times to update eleven files. This is silly. Replace the six programs by a single program that spits out all eleven files.
Yay, about time!
One program, but still invoked multiple times:
Quoted 13 lines
Signed-off-by Markus Armbruster <[email protected]>--- Makefile | 86 ++++++++++------------ scripts/qapi-gen.py | 41 +++++++++++ scripts/qapi/__init__.py | 0 scripts/{qapi-commands.py => qapi/commands.py} | 23 ++---- scripts/{qapi.py => qapi/common.py} | 0 scripts/{qapi2texi.py => qapi/doc.py} | 29 ++------ scripts/{qapi-event.py => qapi/events.py} | 23 ++---- scripts/{qapi-introspect.py => qapi/introspect.py} | 32 ++------ scripts/{qapi-types.py => qapi/types.py} | 34 ++------- scripts/{qapi-visit.py => qapi/visit.py} | 34 ++-------
Maybe the commit message should mention:
This requires moving some files around for proper use in python.
Quoted 9 lines
tests/Makefile.include | 56 +++++++-------
tests/qapi-schema/test-qapi.py | 2 +-
12 files changed, 140 insertions(+), 220 deletions(-)
create mode 100755 scripts/qapi-gen.py
create mode 100644 scripts/qapi/__init__.py
rename scripts/{qapi-commands.py => qapi/commands.py} (94%)
rename scripts/{qapi.py => qapi/common.py} (100%)
rename scripts/{qapi2texi.py => qapi/doc.py} (92%)
mode change 100755 => 100644Unintinentional? We're inconsistent on which of our *.py files are executable in git. Does it matter whether a file that is designed for use as a module is marked executable (if so, perhaps 5/21 should be adding the x attribute on other files, rather than this patch removing it on the doc generator).
Makefile qemu-ga$(EXESUF): QEMU_CFLAGS += -I qga/qapi-generated · line 500
Quoted 12 lines
+++ b/Makefile
Makefile +10
-0+qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h \ +qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h \ +qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c \ +qga/qapi-generated/qga-qapi.texi: \ +qga/qapi-generated/qapi-gen-timestamp ; +qga/qapi-generated/qapi-gen-timestamp: $(SRC_PATH)/qga/qapi-schema.json $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o qga/qapi-generated -p "qga-" $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@
once for qga,
Makefile qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \ · line 528
Quoted 12 lines
Makefile +12
-0+qapi-types.c qapi-types.h \ +qapi-visit.c qapi-visit.h \ +qmp-commands.h qmp-marshal.c \ +qapi-event.c qapi-event.h \ +qmp-introspect.h qmp-introspect.c \ +qapi.texi: \ +qapi-gen-timestamp ; +qapi-gen-timestamp: $(qapi-modules) $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o "." -b $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@
and again for the main level. Still, a definite improvement!
Quoted 13 lines
+++
b/scripts/qapi-gen.pyscripts/qapi-gen.py new file +11
-0+def main(argv): + (input_file, output_dir, do_c, do_h, prefix, opts) = \ + parse_command_line('bu', ['builtins', 'unmask-non-abi-names']) + + opt_builtins = False + opt_unmask = False + + for o, a in opts: + if o in ('-b', '--builtins'): + opt_builtins = True + if o in ('-u', '--unmask-non-abi-names'):
scripts/qapi-gen.py line 37
Quoted 10 lines
scripts/qapi-gen.py new file +10
-0+ opt_unmask = True + + schema = QAPISchema(input_file) + + gen_types(schema, output_dir, prefix, opt_builtins) + gen_visit(schema, output_dir, prefix, opt_builtins) + gen_commands(schema, output_dir, prefix) + gen_events(schema, output_dir, prefix) + gen_introspect(schema, output_dir, prefix, opt_unmask) + gen_doc(schema, output_dir, prefix)
Rather simple, but definitely nicer all in one python file than as a series of make rules.
scripts/qapi/commands.py class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): · line 259
Quoted 13 lines
scripts/qapi/commands.py renamed from scripts/qapi-commands.py +2
-7@@ -255,13 +255,8 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): self._regy += gen_register_command(name, success_response) -def main(argv): - (input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line() - - blurb = ''' - * Schema-defined QAPI/QMP commands -''' - +def gen_commands(schema, output_dir, prefix): + blurb = ' * Schema-defined QAPI/QMP commands'
Some churn on the definition of blurb; worth tidying this in the introduction earlier in the series?
Quoted 14 lines
rename to
scripts/qapi/introspect.py
index09e7d1f140..2153060f2c100644scripts/qapi/introspect.py renamed from scripts/qapi-introspect.py +1
-1@@ -10,7 +10,7 @@ This work is licensed under the terms of the GNU GPL, version 2. See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * # Caveman's json.dumps() replacement (we're stuck at Python 2.4)
scripts/qapi/introspect.py const char %(c_name)s[] = %(c_string)s; · line 177
Quoted 10 lines
scripts/qapi/introspect.py renamed from scripts/qapi-introspect.py +0
-7@@ -168,22 +168,8 @@ const char %(c_name)s[] = %(c_string)s; self._gen_json(name, 'event', {'arg-type': self._use_type(arg_type)}) -def main(argv): - # Debugging aid: unmask QAPI schema's type names - # We normally mask them, because they're not QMP wire ABI - opt_unmask = False - - (input_file, output_dir, do_c, do_h, prefix, opts) = \ - parse_command_line('u', ['unmask-non-abi-names'])
Hmm - we didn't have any docs about this hidden command line option; I see you still preserved it, but it may be worth mentioning in your pending doc updates for the series respin.
Quoted 7 lines
tests/Makefile.include +0
-1@@ -23,7 +23,16 @@ check-help: ifneq ($(wildcard config-host.mak),) export SRC_PATH -qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py
tests/Makefile.include check-help: · line 35
Quoted 10 lines
tests/Makefile.include +10
-0+# TODO don't duplicate $(SRC_PATH)/Makefile's qapi-py here +qapi-py = $(SRC_PATH)/scripts/qapi/commands.py \ +$(SRC_PATH)/scripts/qapi/events.py \ +$(SRC_PATH)/scripts/qapi/introspect.py \ +$(SRC_PATH)/scripts/qapi/types.py \ +$(SRC_PATH)/scripts/qapi/visit.py \ +$(SRC_PATH)/scripts/qapi/common.py \ +$(SRC_PATH)/scripts/qapi/doc.py \ +$(SRC_PATH)/scripts/ordereddict.py \ +$(SRC_PATH)/scripts/qapi-gen.py
So, were you counting these in the 11 generated files mentioned in the commit message? :)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 06/21] qapi-gen: New common driver for code and doc generators
Quoted 23 lines · Eric Blake
Quoted 3 lines · Markus Armbruster
Whenever qapi-schema.json changes, we run six programs eleven times to update eleven files. This is silly. Replace the six programs by a single program that spits out all eleven files.
Yay, about time!
One program, but still invoked multiple times:
Quoted 13 lines
Signed-off-by Markus Armbruster <[email protected]>--- Makefile | 86 ++++++++++------------ scripts/qapi-gen.py | 41 +++++++++++ scripts/qapi/__init__.py | 0 scripts/{qapi-commands.py => qapi/commands.py} | 23 ++---- scripts/{qapi.py => qapi/common.py} | 0 scripts/{qapi2texi.py => qapi/doc.py} | 29 ++------ scripts/{qapi-event.py => qapi/events.py} | 23 ++---- scripts/{qapi-introspect.py => qapi/introspect.py} | 32 ++------ scripts/{qapi-types.py => qapi/types.py} | 34 ++------- scripts/{qapi-visit.py => qapi/visit.py} | 34 ++-------
Quoted 3 lines
Maybe the commit message should mention:
This requires moving some files around for proper use in python.
Yes, I should mention that I wrap the QAPI modules in a Python package.
Quoted 10 lines
Quoted 9 lines
tests/Makefile.include | 56 +++++++------- tests/qapi-schema/test-qapi.py | 2 +- 12 files changed, 140 insertions(+), 220 deletions(-) create mode 100755 scripts/qapi-gen.py create mode 100644 scripts/qapi/__init__.py rename scripts/{qapi-commands.py => qapi/commands.py} (94%) rename scripts/{qapi.py => qapi/common.py} (100%) rename scripts/{qapi2texi.py => qapi/doc.py} (92%) mode change 100755 => 100644
Quoted 5 lines
Unintinentional? We're inconsistent on which of our *.py files are executable in git. Does it matter whether a file that is designed for use as a module is marked executable (if so, perhaps 5/21 should be adding the x attribute on other files, rather than this patch removing it on the doc generator).
qapi2texi.py is no longer runnable as a standalone program after this patch.
So are qapi-{commands,events,introspect,types,visit}.py, but they never had the executable bit set.
Quoted 14 lines
Makefile qemu-ga$(EXESUF): QEMU_CFLAGS += -I qga/qapi-generated · line 500
Quoted 12 lines
+++ b/Makefile
Makefile +10
-0+qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h \ +qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h \ +qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c \ +qga/qapi-generated/qga-qapi.texi: \ +qga/qapi-generated/qapi-gen-timestamp ; +qga/qapi-generated/qapi-gen-timestamp: $(SRC_PATH)/qga/qapi-schema.json $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o qga/qapi-generated -p "qga-" $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@once for qga,
That's the QAPI/QGA schema. The commit message talks about the QAPI/QMP schema. I wish the two weren't named the same.
Modularization might make fusing them possible. Whether that's a good idea I don't know.
Quoted 6 lines
Quoted 6 lines
Makefile +6
-0+qapi-types.c qapi-types.h \ +qapi-visit.c qapi-visit.h \ +qmp-commands.h qmp-marshal.c \ +qapi-event.c qapi-event.h \ +qmp-introspect.h qmp-introspect.c \ +qapi.texi: \
Quoted 8 lines
Makefile qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \ · line 528
Quoted 6 lines
Makefile +6
-0+qapi-gen-timestamp ; +qapi-gen-timestamp: $(qapi-modules) $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o "." -b $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@and again for the main level. Still, a definite improvement!
Perhaps I can find a way to clarify the commit message.
Quoted 31 lines
Quoted 13 lines
+++
b/scripts/qapi-gen.pyscripts/qapi-gen.py new file +11
-0+def main(argv): + (input_file, output_dir, do_c, do_h, prefix, opts) = \ + parse_command_line('bu', ['builtins', 'unmask-non-abi-names']) + + opt_builtins = False + opt_unmask = False + + for o, a in opts: + if o in ('-b', '--builtins'): + opt_builtins = True + if o in ('-u', '--unmask-non-abi-names'):scripts/qapi-gen.py line 37
Quoted 10 lines
scripts/qapi-gen.py new file +10
-0+ opt_unmask = True + + schema = QAPISchema(input_file) + + gen_types(schema, output_dir, prefix, opt_builtins) + gen_visit(schema, output_dir, prefix, opt_builtins) + gen_commands(schema, output_dir, prefix) + gen_events(schema, output_dir, prefix) + gen_introspect(schema, output_dir, prefix, opt_unmask) + gen_doc(schema, output_dir, prefix)Rather simple, but definitely nicer all in one python file than as a series of make rules.
Quoted 4 lines
scripts/qapi/commands.py renamed from scripts/qapi-commands.py +0
-0@@ -255,13 +255,8 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): self._regy += gen_register_command(name, success_response)
Quoted 12 lines
scripts/qapi/commands.py class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): · line 259
Quoted 9 lines
scripts/qapi/commands.py renamed from scripts/qapi-commands.py +2
-7-def main(argv): - (input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line() - - blurb = ''' - * Schema-defined QAPI/QMP commands -''' - +def gen_commands(schema, output_dir, prefix): + blurb = ' * Schema-defined QAPI/QMP commands'Some churn on the definition of blurb; worth tidying this in the introduction earlier in the series?
Doesn't seem worth a separate patch, but I can try to reshuffle things to reduce churn.
Quoted 17 lines
Quoted 17 lines
rename to
scripts/qapi/introspect.py
index09e7d1f140..2153060f2c100644scripts/qapi/introspect.py renamed from scripts/qapi-introspect.py +1
-1@@ -10,7 +10,7 @@ This work is licensed under the terms of the GNU GPL, version 2. See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * # Caveman's json.dumps() replacement (we're stuck at Python 2.4)@@ -168,22 +168,8 @@ const char %(c_name)s[] = %(c_string)s; self._gen_json(name, 'event', {'arg-type': self._use_type(arg_type)})
Quoted 11 lines
scripts/qapi/introspect.py const char %(c_name)s[] = %(c_string)s; · line 177
Quoted 7 lines
scripts/qapi/introspect.py renamed from scripts/qapi-introspect.py +0
-7-def main(argv): - # Debugging aid: unmask QAPI schema's type names - # We normally mask them, because they're not QMP wire ABI - opt_unmask = False - - (input_file, output_dir, do_c, do_h, prefix, opts) = \ - parse_command_line('u', ['unmask-non-abi-names'])Hmm - we didn't have any docs about this hidden command line option; I see you still preserved it, but it may be worth mentioning in your pending doc updates for the series respin.
Maybe.
Providing --help would probably be more useful. We should convert qapi-gen.py to argparse.
Quoted 6 lines
Quoted 6 lines
tests/Makefile.include +0
-0@@ -23,7 +23,16 @@ check-help: ifneq ($(wildcard config-host.mak),) export SRC_PATH
Quoted 14 lines
tests/Makefile.include check-help: · line 35
Quoted 11 lines
tests/Makefile.include +10
-1-qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py +# TODO don't duplicate $(SRC_PATH)/Makefile's qapi-py here +qapi-py = $(SRC_PATH)/scripts/qapi/commands.py \ +$(SRC_PATH)/scripts/qapi/events.py \ +$(SRC_PATH)/scripts/qapi/introspect.py \ +$(SRC_PATH)/scripts/qapi/types.py \ +$(SRC_PATH)/scripts/qapi/visit.py \ +$(SRC_PATH)/scripts/qapi/common.py \ +$(SRC_PATH)/scripts/qapi/doc.py \ +$(SRC_PATH)/scripts/ordereddict.py \ +$(SRC_PATH)/scripts/qapi-gen.pySo, were you counting these in the 11 generated files mentioned in the commit message? :)
I'm not sure I understand what you mean here...
Re: [Qemu-devel] [PATCH RFC 06/21] qapi-gen: New common driver for code and doc generators
Quoted 21 lines · Markus Armbruster
Quoted 10 lines · Eric Blake
Quoted 3 lines · Markus Armbruster
Whenever qapi-schema.json changes, we run six programs eleven times to update eleven files. This is silly. Replace the six programs by a single program that spits out all eleven files.
Yay, about time!
One program, but still invoked multiple times:
Quoted 8 lines
Quoted 2 lines
rename scripts/{qapi2texi.py => qapi/doc.py} (92%) mode change 100755 => 100644Unintinentional? We're inconsistent on which of our *.py files are executable in git. Does it matter whether a file that is designed for use as a module is marked executable (if so, perhaps 5/21 should be adding the x attribute on other files, rather than this patch removing it on the doc generator).
Quoted 5 lines
qapi2texi.pyis no longer runnable as a standalone program after this patch.So are qapi-{commands,events,introspect,types,visit}.py, but they never had the executable bit set.
Okay, so dropping the bit consistently makes sense; still, a mention in the commit message wouldn't hurt.
Quoted 16 lines
Quoted 15 lines
Makefile qemu-ga$(EXESUF): QEMU_CFLAGS += -I qga/qapi-generated · line 500
Quoted 13 lines
+++ b/Makefile
Makefile +10
-0+qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h \ +qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h \ +qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c \ +qga/qapi-generated/qga-qapi.texi: \ +qga/qapi-generated/qapi-gen-timestamp ; +qga/qapi-generated/qapi-gen-timestamp: $(SRC_PATH)/qga/qapi-schema.json $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o qga/qapi-generated -p "qga-" $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@once for qga,
Quoted 2 lines
That's the QAPI/QGA schema. The commit message talks about the QAPI/QMP schema. I wish the two weren't named the same.
7 files here,...
Quoted 18 lines
Modularization might make fusing them possible. Whether that's a good idea I don't know.
Quoted 14 lines
Makefile qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \ · line 528
Quoted 12 lines
Makefile +12
-0+qapi-types.c qapi-types.h \ +qapi-visit.c qapi-visit.h \ +qmp-commands.h qmp-marshal.c \ +qapi-event.c qapi-event.h \ +qmp-introspect.h qmp-introspect.c \ +qapi.texi: \ +qapi-gen-timestamp ; +qapi-gen-timestamp: $(qapi-modules) $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o "." -b $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@and again for the main level. Still, a definite improvement!
11 files here,...
Quoted 3 lines
Perhaps I can find a way to clarify the commit message.
[1]
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
scripts/qapi/commands.py renamed from scripts/qapi-commands.py +0
-4-def main(argv): - (input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line() - - blurb = '''Quoted 8 lines
scripts/qapi/commands.py class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): · line 259
Quoted 5 lines
- * Schema-defined QAPI/QMP commands
-'''scripts/qapi/commands.py renamed from scripts/qapi-commands.py +2
-1- +def gen_commands(schema, output_dir, prefix): + blurb = ' * Schema-defined QAPI/QMP commands'Some churn on the definition of blurb; worth tidying this in the introduction earlier in the series?
Quoted 2 lines
Doesn't seem worth a separate patch, but I can try to reshuffle things to reduce churn.
Yeah, my question was more about the conversion between multiline '''...''' to single-line '...'; why not just use the single-line construct earlier in the series when introducing the blurb variable. You are right that creating blurb didn't need a separate patch, just less churn over the series.
Quoted 16 lines
Quoted 15 lines
tests/Makefile.include check-help: · line 35
Quoted 12 lines
tests/Makefile.include +10
-1-qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py +# TODO don't duplicate $(SRC_PATH)/Makefile's qapi-py here +qapi-py = $(SRC_PATH)/scripts/qapi/commands.py \ +$(SRC_PATH)/scripts/qapi/events.py \ +$(SRC_PATH)/scripts/qapi/introspect.py \ +$(SRC_PATH)/scripts/qapi/types.py \ +$(SRC_PATH)/scripts/qapi/visit.py \ +$(SRC_PATH)/scripts/qapi/common.py \ +$(SRC_PATH)/scripts/qapi/doc.py \ +$(SRC_PATH)/scripts/ordereddict.py \ +$(SRC_PATH)/scripts/qapi-gen.pySo, were you counting these in the 11 generated files mentioned in the commit message? :)
Quoted 1 line
I'm not sure I understand what you mean here...
[1] and 9 more files. So, the commit message only mentioned the 11 QMP files, rather than all 27 (if I counted right) QAPI generated files. My comments were trying to point out that you simplified more than just the QMP code generation into fewer script invocations, and the counts looked off since out of the three spots converted, only one of the spots had 11 files.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 06/21] qapi-gen: New common driver for code and doc generators
Quoted 22 lines · Eric Blake
Quoted 21 lines · Markus Armbruster
Quoted 18 lines · Eric Blake
Quoted 3 lines · Markus Armbruster
Whenever qapi-schema.json changes, we run six programs eleven times to update eleven files. This is silly. Replace the six programs by a single program that spits out all eleven files.
Yay, about time!
One program, but still invoked multiple times:
Quoted 2 lines
rename scripts/{qapi2texi.py => qapi/doc.py} (92%) mode change 100755 => 100644Unintinentional? We're inconsistent on which of our *.py files are executable in git. Does it matter whether a file that is designed for use as a module is marked executable (if so, perhaps 5/21 should be adding the x attribute on other files, rather than this patch removing it on the doc generator).
Quoted 8 lines
Quoted 5 lines
qapi2texi.pyis no longer runnable as a standalone program after this patch.So are qapi-{commands,events,introspect,types,visit}.py, but they never had the executable bit set.
Okay, so dropping the bit consistently makes sense; still, a mention in the commit message wouldn't hurt.
Can do.
Quoted 61 lines
Quoted 15 lines
Quoted 14 lines
Makefile qemu-ga$(EXESUF): QEMU_CFLAGS += -I qga/qapi-generated · line 500
Quoted 12 lines
+++ b/Makefile
Makefile +10
-0+qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h \ +qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h \ +qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c \ +qga/qapi-generated/qga-qapi.texi: \ +qga/qapi-generated/qapi-gen-timestamp ; +qga/qapi-generated/qapi-gen-timestamp: $(SRC_PATH)/qga/qapi-schema.json $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o qga/qapi-generated -p "qga-" $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@once for qga,
Quoted 2 lines
That's the QAPI/QGA schema. The commit message talks about the QAPI/QMP schema. I wish the two weren't named the same.
7 files here,...
Quoted 18 lines
Modularization might make fusing them possible. Whether that's a good idea I don't know.
Quoted 14 lines
Makefile qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \ · line 528
Quoted 12 lines
Makefile +12
-0+qapi-types.c qapi-types.h \ +qapi-visit.c qapi-visit.h \ +qmp-commands.h qmp-marshal.c \ +qapi-event.c qapi-event.h \ +qmp-introspect.h qmp-introspect.c \ +qapi.texi: \ +qapi-gen-timestamp ; +qapi-gen-timestamp: $(qapi-modules) $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o "." -b $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@and again for the main level. Still, a definite improvement!
11 files here,...
Quoted 3 lines
Perhaps I can find a way to clarify the commit message.
[1]
Quoted 13 lines
Quoted 12 lines
scripts/qapi/commands.py class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): · line 259
Quoted 9 lines
scripts/qapi/commands.py renamed from scripts/qapi-commands.py +2
-7-def main(argv): - (input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line() - - blurb = ''' - * Schema-defined QAPI/QMP commands -''' - +def gen_commands(schema, output_dir, prefix): + blurb = ' * Schema-defined QAPI/QMP commands'Some churn on the definition of blurb; worth tidying this in the introduction earlier in the series?
Quoted 6 lines
Quoted 2 lines
Doesn't seem worth a separate patch, but I can try to reshuffle things to reduce churn.
Yeah, my question was more about the conversion between multiline '''...''' to single-line '...'; why not just use the single-line construct earlier in the series when introducing the blurb variable.
Introduced in PATCH 01:
-c_comment = '''
-/*
- * schema-defined QMP->QAPI command dispatch
+blurb = '''
+ * Schema-defined QAPI/QMP commands
*
* Copyright IBM, Corp. 2011
*
* Authors:
* Anthony Liguori <[email protected]>
- *
- * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
- * See the COPYING.LIB file in the top-level directory.
- *
- */
-'''
Shortened in PATCH 02:
blurb = '''
* Schema-defined QAPI/QMP commands
- *
- * Copyright IBM, Corp. 2011
- *
- * Authors:
- * Anthony Liguori <[email protected]>
'''
Reformatted in PATCH 06 (see above).
Moved in PATCH 16 to visitor's __init__ for types and visits (the rest aren't implemented, yet):
class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
- def __init__(self, opt_builtins):
+ def __init__(self, opt_builtins, prefix):
self._opt_builtins = opt_builtins
- self.decl = None
- self.defn = None
- self._fwdecl = None
- self._btin = None
+ self._prefix = prefix
+ blurb = ' * Schema-defined QAPI types'
+ self._genc = QAPIGenC(blurb, __doc__)
+ self._genh = QAPIGenH(blurb, __doc__)
Variable eliminated there in PATCH 17:
- blurb = ' * Schema-defined QAPI types'
- self._genc = QAPIGenC(blurb, __doc__)
- self._genh = QAPIGenH(blurb, __doc__)
+ self._module = {}
+ self._add_module(None, ' * Built-in QAPI types')I could delay the reformatting until PATCH 16, or do it in PATCH 02. Feels like a wash to me, but if you have a preference...
Quoted 19 lines
You are right that creating blurb didn't need a separate patch, just less churn over the series.
Quoted 16 lines
Quoted 15 lines
tests/Makefile.include check-help: · line 35
Quoted 12 lines
tests/Makefile.include +10
-1-qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py +# TODO don't duplicate $(SRC_PATH)/Makefile's qapi-py here +qapi-py = $(SRC_PATH)/scripts/qapi/commands.py \ +$(SRC_PATH)/scripts/qapi/events.py \ +$(SRC_PATH)/scripts/qapi/introspect.py \ +$(SRC_PATH)/scripts/qapi/types.py \ +$(SRC_PATH)/scripts/qapi/visit.py \ +$(SRC_PATH)/scripts/qapi/common.py \ +$(SRC_PATH)/scripts/qapi/doc.py \ +$(SRC_PATH)/scripts/ordereddict.py \ +$(SRC_PATH)/scripts/qapi-gen.pySo, were you counting these in the 11 generated files mentioned in the commit message? :)
Quoted 8 lines
Quoted 1 line
I'm not sure I understand what you mean here...
[1] and 9 more files. So, the commit message only mentioned the 11 QMP files, rather than all 27 (if I counted right) QAPI generated files. My comments were trying to point out that you simplified more than just the QMP code generation into fewer script invocations, and the counts looked off since out of the three spots converted, only one of the spots had 11 files.
The commit message talks only about the QAPI/QMP schema. It's confusing because our poor taste in file names creates ambiguity: does qapi-schema.json refer to ./qapi-schema.json (i.e. the QAPI/QMP one), qga/qapi-schema.json, or both?
Perhaps I should rename qapi-schema.json to qapi/schema.json.
The commit message needs a note along the lines of "same for qga/qapi-schema.json".
Re: [Qemu-devel] [PATCH RFC 06/21] qapi-gen: New common driver for code and doc generators
Quoted 22 lines · Markus Armbruster
Quoted 13 lines
Quoted 13 lines
Quoted 12 lines
scripts/qapi/commands.py class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): · line 259
Quoted 9 lines
scripts/qapi/commands.py renamed from scripts/qapi-commands.py +2
-7-def main(argv): - (input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line() - - blurb = ''' - * Schema-defined QAPI/QMP commands -''' - +def gen_commands(schema, output_dir, prefix): + blurb = ' * Schema-defined QAPI/QMP commands'Some churn on the definition of blurb; worth tidying this in the introduction earlier in the series?
Quoted 6 lines
Quoted 2 lines
Doesn't seem worth a separate patch, but I can try to reshuffle things to reduce churn.
Yeah, my question was more about the conversion between multiline '''...''' to single-line '...'; why not just use the single-line construct earlier in the series when introducing the blurb variable.
Introduced in PATCH 01:
Quoted 7 lines
-c_comment = '''
-/*
- * schema-defined QMP->QAPI command dispatch
+blurb = '''
+ * Schema-defined QAPI/QMP commands
*
* Copyright IBM, Corp. 2011Multiline made sense here.
Quoted 9 lines
Shortened in PATCH 02:
blurb = '''
* Schema-defined QAPI/QMP commands
- *
- * Copyright IBM, Corp. 2011
- *
- * Authors:
- * Anthony Liguori <[email protected]>
'''
Quoted 1 line
Where it is now a single line...
Quoted 7 lines
Variable eliminated there in PATCH 17:
- blurb = ' * Schema-defined QAPI types' - self._genc = QAPIGenC(blurb, __doc__) - self._genh = QAPIGenH(blurb, __doc__) + self._module = {} + self._add_module(None, ' * Built-in QAPI types')
Oh, I didn't even notice that!
Quoted 3 lines
I could delay the reformatting until PATCH 16, or do it in PATCH 02. Feels like a wash to me, but if you have a preference...
I guess my preference is to reformat to single line in PATCH 02, then patch 6 doesn't have to touch it, and patch 16 can still get rid of it. But, as it disappears by the end of the series anyways, I'm also okay if you don't bother (churn is not necessarily bad, if it is still easy to review).
Quoted 6 lines
The commit message talks only about the QAPI/QMP schema. It's confusing because our poor taste in file names creates ambiguity: does qapi-schema.json refer to ./qapi-schema.json (i.e. the QAPI/QMP one),
qga/qapi-schema.json, or both?Perhaps I should rename qapi-schema.json to
qapi/schema.json.
I'd be in favor of such a change. We'd also rename qga/qapi-schema.json to qga/schema.json?
Quoted 4 lines
The commit message needs a note along the lines of "same for
qga/qapi-schema.json".
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 06/21] qapi-gen: New common driver for code and doc generators
In reply to Markus ArmbrusterQuoted 4 lines · Markus Armbruster
Whenever qapi-schema.json changes, we run six programs eleven times to update eleven files. This is silly. Replace the six programs by a single program that spits out all eleven files.
Now we will need documentation update.
Also greping for the renamed files:
git grep -E 'qapi-commands|qapi2texi|qapi-event.py|qapi-intros|qapi-types.py|qapi-visit.py' docs/devel/qapi-code-gen.txt:=== scripts/qapi-types.py ===docs/devel/qapi-code-gen.txt: $ python scripts/qapi-types.py
--output-dir="qapi-generated" \
docs/devel/qapi-code-gen.txt:=== scripts/qapi-visit.py ===docs/devel/qapi-code-gen.txt: $ python scripts/qapi-visit.py
--output-dir="qapi-generated"
docs/devel/qapi-code-gen.txt:=== scripts/qapi-commands.py ===docs/devel/qapi-code-gen.txt: generated byqapi-visit.py are used todocs/devel/qapi-code-gen.txt: $ python scripts/qapi-commands.py
--output-dir="qapi-generated"
docs/devel/qapi-code-gen.txt:=== scripts/qapi-event.py ===docs/devel/qapi-code-gen.txt: $ python scripts/qapi-event.py
--output-dir="qapi-generated"
docs/devel/qapi-code-gen.txt:=== scripts/qapi-introspect.py ===docs/devel/qapi-code-gen.txt: $ python scripts/qapi-introspect.py
--output-dir="qapi-generated"monitor.c: * qapi-introspect.py's output actually conforms to the schema. qapi-schema.json:# Documentation generated with qapi2texi.py is in
source order, with
Quoted 282 lines
Signed-off-by Markus Armbruster <[email protected]>--- Makefile | 86 ++++++++++------------ scripts/qapi-gen.py | 41 +++++++++++ scripts/qapi/__init__.py | 0 scripts/{qapi-commands.py => qapi/commands.py} | 23 ++---- scripts/{qapi.py => qapi/common.py} | 0 scripts/{qapi2texi.py => qapi/doc.py} | 29 ++------ scripts/{qapi-event.py => qapi/events.py} | 23 ++---- scripts/{qapi-introspect.py => qapi/introspect.py} | 32 ++------ scripts/{qapi-types.py => qapi/types.py} | 34 ++------- scripts/{qapi-visit.py => qapi/visit.py} | 34 ++------- tests/Makefile.include | 56 +++++++------- tests/qapi-schema/test-qapi.py | 2 +- 12 files changed, 140 insertions(+), 220 deletions(-) create mode 100755 scripts/qapi-gen.py create mode 100644 scripts/qapi/__init__.py rename scripts/{qapi-commands.py => qapi/commands.py} (94%) rename scripts/{qapi.py => qapi/common.py} (100%) rename scripts/{qapi2texi.py => qapi/doc.py} (92%) mode change 100755 => 100644 rename scripts/{qapi-event.py => qapi/events.py} (92%) rename scripts/{qapi-introspect.py => qapi/introspect.py} (90%) rename scripts/{qapi-types.py => qapi/types.py} (90%) rename scripts/{qapi-visit.py => qapi/visit.py} (92%)Makefile +37
-49@@ -92,6 +92,7 @@ GENERATED_FILES += qmp-commands.h qapi-types.h qapi-visit.h qapi-event.h GENERATED_FILES += qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c GENERATED_FILES += qmp-introspect.h GENERATED_FILES += qmp-introspect.c +GENERATED_FILES += qapi.texi GENERATED_FILES += trace/generated-tcg-tracers.h@@ -477,25 +478,26 @@ qemu-ga$(EXESUF): QEMU_CFLAGS += -I qga/qapi-generated qemu-keymap$(EXESUF): LIBS += $(XKBCOMMON_LIBS) qemu-keymap$(EXESUF): QEMU_CFLAGS += $(XKBCOMMON_CFLAGS) -gen-out-type = $(subst .,-,$(suffix $@)) +qapi-py = $(SRC_PATH)/scripts/qapi/commands.py \ +$(SRC_PATH)/scripts/qapi/events.py \ +$(SRC_PATH)/scripts/qapi/introspect.py \ +$(SRC_PATH)/scripts/qapi/types.py \ +$(SRC_PATH)/scripts/qapi/visit.py \ +$(SRC_PATH)/scripts/qapi/common.py \ +$(SRC_PATH)/scripts/qapi/doc.py \ +$(SRC_PATH)/scripts/ordereddict.py \ +$(SRC_PATH)/scripts/qapi-gen.py -qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py - -qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h :\ -$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \ - $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \ - "GEN","$@") -qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h :\ -$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \ - $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \ - "GEN","$@") -qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c :\ -$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \ - $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \ - "GEN","$@") +qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h \ +qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h \ +qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c \ +qga/qapi-generated/qga-qapi.texi: \ +qga/qapi-generated/qapi-gen-timestamp ; +qga/qapi-generated/qapi-gen-timestamp: $(SRC_PATH)/qga/qapi-schema.json $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o qga/qapi-generated -p "qga-" $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \ $(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \@@ -512,31 +514,18 @@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \ $(SRC_PATH)/qapi/transaction.json \ $(SRC_PATH)/qapi/ui.json -qapi-types.c qapi-types.h :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-types.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \ - $(gen-out-type) -o "." -b $<, \ - "GEN","$@") -qapi-visit.c qapi-visit.h :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \ - $(gen-out-type) -o "." -b $<, \ - "GEN","$@") -qapi-event.c qapi-event.h :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-event.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-event.py \ - $(gen-out-type) -o "." $<, \ - "GEN","$@") -qmp-commands.h qmp-marshal.c :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \ - $(gen-out-type) -o "." $<, \ - "GEN","$@") -qmp-introspect.h qmp-introspect.c :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-introspect.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-introspect.py \ - $(gen-out-type) -o "." $<, \ - "GEN","$@") +qapi-types.c qapi-types.h \ +qapi-visit.c qapi-visit.h \ +qmp-commands.h qmp-marshal.c \ +qapi-event.c qapi-event.h \ +qmp-introspect.h qmp-introspect.c \ +qapi.texi: \ +qapi-gen-timestamp ; +qapi-gen-timestamp: $(qapi-modules) $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o "." -b $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@ QGALIB_GEN=$(addprefix qga/qapi-generated/, qga-qapi-types.h qga-qapi-visit.h qga-qmp-commands.h) $(qga-obj-y): $(QGALIB_GEN)@@ -596,6 +585,7 @@ clean: rm -f trace/generated-tracers-dtrace.dtrace* rm -f trace/generated-tracers-dtrace.h* rm -f $(foreach f,$(GENERATED_FILES),$(f) $(f)-timestamp) + rm -f qapi-gen-timestamp rm -rf qapi-generated rm -rf qga/qapi-generated for d in $(ALL_SUBDIRS); do \@@ -803,13 +793,11 @@ qemu-monitor-info.texi: $(SRC_PATH)/hmp-commands-info.hx $(SRC_PATH)/scripts/hxt qemu-img-cmds.texi: $(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/scripts/hxtool $(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > $@,"GEN","$@") -docs/interop/qemu-qmp-qapi.texi docs/interop/qemu-ga-qapi.texi: $(SRC_PATH)/scripts/qapi2texi.py $(qapi-py) +docs/interop/qemu-qmp-qapi.texi: qapi.texi + @cp -p $< $@ -docs/interop/qemu-qmp-qapi.texi: $(qapi-modules) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi2texi.py $< > $@,"GEN","$@") - -docs/interop/qemu-ga-qapi.texi: $(SRC_PATH)/qga/qapi-schema.json - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi2texi.py $< > $@,"GEN","$@") +docs/interop/qemu-ga-qapi.texi: qga/qapi-generated/qga-qapi.texi + @cp -p $< $@ qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi qemu-monitor-info.texi qemu.1: qemu-option-trace.texiscripts/qapi-gen.py new file +41
-0@@ -0,0 +1,41 @@ +#!/usr/bin/env python +# QAPI generator +# +# This work is licensed under the terms of the GNU GPL, version 2 or later. +# See the COPYING file in the top-level directory. + +import sys +from qapi.common import parse_command_line, QAPISchema +from qapi.types import gen_types +from qapi.visit import gen_visit +from qapi.commands import gen_commands +from qapi.events import gen_events +from qapi.introspect import gen_introspect +from qapi.doc import gen_doc + + +def main(argv): + (input_file, output_dir, do_c, do_h, prefix, opts) = \ + parse_command_line('bu', ['builtins', 'unmask-non-abi-names']) + + opt_builtins = False + opt_unmask = False + + for o, a in opts: + if o in ('-b', '--builtins'): + opt_builtins = True + if o in ('-u', '--unmask-non-abi-names'): + opt_unmask = True + + schema = QAPISchema(input_file) + + gen_types(schema, output_dir, prefix, opt_builtins) + gen_visit(schema, output_dir, prefix, opt_builtins) + gen_commands(schema, output_dir, prefix) + gen_events(schema, output_dir, prefix) + gen_introspect(schema, output_dir, prefix, opt_unmask) + gen_doc(schema, output_dir, prefix) + + +if __name__ == '__main__': + main(sys.argv)scripts/qapi/__init__.py new file +0
-0scripts/qapi/commands.py renamed from scripts/qapi-commands.py +5
-18@@ -13,7 +13,7 @@ This work is licensed under the terms of the GNU GPL, version 2. See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * def gen_command_decl(name, arg_type, boxed, ret_type):@@ -255,13 +255,8 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): self._regy += gen_register_command(name, success_response) -def main(argv): - (input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line() - - blurb = ''' - * Schema-defined QAPI/QMP commands -''' - +def gen_commands(schema, output_dir, prefix): + blurb = ' * Schema-defined QAPI/QMP commands' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)@@ -290,17 +285,9 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds); ''', prefix=prefix, c_prefix=c_name(prefix, protect=False))) - schema = QAPISchema(input_file) vis = QAPISchemaGenCommandVisitor(prefix) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qmp-marshal.c') - if do_h: - genh.write(output_dir, prefix + 'qmp-commands.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qmp-marshal.c') + genh.write(output_dir, prefix + 'qmp-commands.h')scripts/qapi/common.py renamed from scripts/qapi.py +0
-0scripts/qapi/doc.py renamed from scripts/qapi2texi.py +2
-3@@ -4,10 +4,9 @@ # This work is licensed under the terms of the GNU LGPL, version 2+. # See the COPYING file in the top-level directory. """This script produces the documentation of a qapi schema in texinfo format""" + import re -import sys - -import qapi +import qapi.common MSG_FMT = """ @deftypefn {type} {{}} {name}
scripts/qapi/doc.py def texi_entity(doc, what, base=None, variants=None, · line 198
Quoted 5 lines
scripts/qapi/doc.py renamed from scripts/qapi2texi.py +1
-1@@ -196,7 +195,7 @@ def texi_entity(doc, what, base=None, variants=None, + texi_sections(doc)) -class QAPISchemaGenDocVisitor(qapi.QAPISchemaVisitor): +class QAPISchemaGenDocVisitor(qapi.common.QAPISchemaVisitor):
Would be a bit easier to read and more consitent with a top-level "from qapi.common import QAPISchemaVisitor"
Quoted 273 lines
scripts/qapi/doc.py renamed from scripts/qapi2texi.py +5
-17def __init__(self): self.out = None self.cur_doc = None@@ -271,20 +270,8 @@ def texi_schema(schema): return gen.out -def main(argv): - """Takes schema argument, prints result to stdout""" - if len(argv) != 2: - print >>sys.stderr, "%s: need exactly 1 argument: SCHEMA" % argv[0] - sys.exit(1) - - schema = qapi.QAPISchema(argv[1]) - if not qapi.doc_required: - print >>sys.stderr, ("%s: need pragma 'doc-required' " - "to generate documentation" % argv[0]) - sys.exit(1) - print '@c AUTOMATICALLY GENERATED, DO NOT MODIFY\n' - print texi_schema(schema), - - -if __name__ == '__main__': - main(sys.argv) +def gen_doc(schema, output_dir, prefix): + if qapi.common.doc_required: + gen = qapi.common.QAPIGenDoc() + gen.body(texi_schema(schema)) + gen.write(output_dir, prefix + 'qapi.texi')scripts/qapi/events.py renamed from scripts/qapi-event.py +5
-18@@ -12,7 +12,7 @@ This work is licensed under the terms of the GNU GPL, version 2. See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * def build_event_send_proto(name, arg_type, boxed):@@ -171,13 +171,8 @@ class QAPISchemaGenEventVisitor(QAPISchemaVisitor): self._event_names.append(name) -def main(argv): - (input_file, output_dir, do_c, do_h, prefix, dummy) = parse_command_line() - - blurb = ''' - * Schema-defined QAPI/QMP events -''' - +def gen_events(schema, output_dir, prefix): + blurb = ' * Schema-defined QAPI/QMP events' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)@@ -201,17 +196,9 @@ def main(argv): ''', prefix=prefix)) - schema = QAPISchema(input_file) vis = QAPISchemaGenEventVisitor(prefix) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qapi-event.c') - if do_h: - genh.write(output_dir, prefix + 'qapi-event.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qapi-event.c') + genh.write(output_dir, prefix + 'qapi-event.h')scripts/qapi/introspect.py renamed from scripts/qapi-introspect.py +5
-27@@ -10,7 +10,7 @@ This work is licensed under the terms of the GNU GPL, version 2. See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * # Caveman's json.dumps() replacement (we're stuck at Python 2.4)@@ -168,22 +168,8 @@ const char %(c_name)s[] = %(c_string)s; self._gen_json(name, 'event', {'arg-type': self._use_type(arg_type)}) -def main(argv): - # Debugging aid: unmask QAPI schema's type names - # We normally mask them, because they're not QMP wire ABI - opt_unmask = False - - (input_file, output_dir, do_c, do_h, prefix, opts) = \ - parse_command_line('u', ['unmask-non-abi-names']) - - for o, a in opts: - if o in ('-u', '--unmask-non-abi-names'): - opt_unmask = True - - blurb = ''' - * QAPI/QMP schema introspection -''' - +def gen_introspect(schema, output_dir, prefix, opt_unmask): + blurb = ' * QAPI/QMP schema introspection' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)@@ -194,17 +180,9 @@ def main(argv): ''', prefix=prefix)) - schema = QAPISchema(input_file) vis = QAPISchemaGenIntrospectVisitor(prefix, opt_unmask) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qmp-introspect.c') - if do_h: - genh.write(output_dir, prefix + 'qmp-introspect.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qmp-introspect.c') + genh.write(output_dir, prefix + 'qmp-introspect.h')scripts/qapi/types.py renamed from scripts/qapi-types.py +5
-29@@ -13,7 +13,7 @@ This work is licensed under the terms of the GNU GPL, version 2. # See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * # variants must be emitted before their container; track what has already@@ -241,24 +241,8 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self._gen_type_cleanup(name) -def main(argv): - # If you link code generated from multiple schemata, you want only one - # instance of the code for built-in types. Generate it only when - # opt_builtins, enabled by command line option -b. See also - # QAPISchemaGenTypeVisitor.visit_end(). - opt_builtins = False - - (input_file, output_dir, do_c, do_h, prefix, opts) = \ - parse_command_line('b', ['builtins']) - - for o, a in opts: - if o in ('-b', '--builtins'): - opt_builtins = True - - blurb = ''' - * Schema-defined QAPI types -''' - +def gen_types(schema, output_dir, prefix, opt_builtins): + blurb = ' * Schema-defined QAPI types' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)@@ -274,17 +258,9 @@ def main(argv): #include "qapi/util.h" ''')) - schema = QAPISchema(input_file) vis = QAPISchemaGenTypeVisitor(opt_builtins) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qapi-types.c') - if do_h: - genh.write(output_dir, prefix + 'qapi-types.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qapi-types.c') + genh.write(output_dir, prefix + 'qapi-types.h')scripts/qapi/visit.py renamed from scripts/qapi-visit.py +5
-29@@ -13,7 +13,7 @@ This work is licensed under the terms of the GNU GPL, version 2. See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * def gen_visit_decl(name, scalar=False):@@ -324,24 +324,8 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): self.defn += gen_visit_alternate(name, variants) -def main(argv): - # If you link code generated from multiple schemata, you want only one - # instance of the code for built-in types. Generate it only when - # opt_builtins, enabled by command line option -b. See also - # QAPISchemaGenVisitVisitor.visit_end(). - opt_builtins = False - - (input_file, output_dir, do_c, do_h, prefix, opts) = \ - parse_command_line('b', ['builtins']) - - for o, a in opts: - if o in ('-b', '--builtins'): - opt_builtins = True - - blurb = ''' - * Schema-defined QAPI visitors -''' - +def gen_visit(schema, output_dir, prefix, opt_builtins): + blurb = ' * Schema-defined QAPI visitors' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)@@ -361,17 +345,9 @@ def main(argv): ''', prefix=prefix)) - schema = QAPISchema(input_file) vis = QAPISchemaGenVisitVisitor(opt_builtins) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qapi-visit.c') - if do_h: - genh.write(output_dir, prefix + 'qapi-visit.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qapi-visit.c') + genh.write(output_dir, prefix + 'qapi-visit.h')tests/Makefile.include +0
-0
tests/Makefile.include check-help: · line 26
Quoted 5 lines
tests/Makefile.include +1
-1@@ -23,7 +23,16 @@ check-help: ifneq ($(wildcard config-host.mak),) export SRC_PATH -qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py +# TODO don't duplicate $(SRC_PATH)/Makefile's qapi-py here
hmm, it doesn't look necessary, since the makefile is included from there. Right?
Quoted 86 lines
tests/Makefile.include +27
-27+qapi-py = $(SRC_PATH)/scripts/qapi/commands.py \ +$(SRC_PATH)/scripts/qapi/events.py \ +$(SRC_PATH)/scripts/qapi/introspect.py \ +$(SRC_PATH)/scripts/qapi/types.py \ +$(SRC_PATH)/scripts/qapi/visit.py \ +$(SRC_PATH)/scripts/qapi/common.py \ +$(SRC_PATH)/scripts/qapi/doc.py \ +$(SRC_PATH)/scripts/ordereddict.py \ +$(SRC_PATH)/scripts/qapi-gen.py # Get the list of all supported sysemu targets SYSEMU_TARGET_LIST := $(subst -softmmu.mak,,$(notdir \@@ -642,34 +651,24 @@ tests/test-logging$(EXESUF): tests/test-logging.o $(test-util-obj-y) tests/test-replication$(EXESUF): tests/test-replication.o $(test-util-obj-y) \ $(test-block-obj-y) -tests/test-qapi-types.c tests/test-qapi-types.h :\ -$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \ - $(gen-out-type) -o tests -p "test-" $<, \ - "GEN","$@") -tests/test-qapi-visit.c tests/test-qapi-visit.h :\ -$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \ - $(gen-out-type) -o tests -p "test-" $<, \ - "GEN","$@") -tests/test-qmp-commands.h tests/test-qmp-marshal.c :\ -$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \ - $(gen-out-type) -o tests -p "test-" $<, \ - "GEN","$@") -tests/test-qapi-event.c tests/test-qapi-event.h :\ -$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-event.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-event.py \ - $(gen-out-type) -o tests -p "test-" $<, \ - "GEN","$@") -tests/test-qmp-introspect.c tests/test-qmp-introspect.h :\ -$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-introspect.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-introspect.py \ - $(gen-out-type) -o tests -p "test-" $<, \ - "GEN","$@") +tests/test-qapi-types.c tests/test-qapi-types.h \ +tests/test-qapi-visit.c tests/test-qapi-visit.h \ +tests/test-qmp-commands.h tests/test-qmp-marshal.c \ +tests/test-qapi-event.c tests/test-qapi-event.h \ +tests/test-qmp-introspect.c tests/test-qmp-introspect.h: \ +tests/test-qapi-gen-timestamp ; +tests/test-qapi-gen-timestamp: $(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o tests -p "test-" $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@ -tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.json $(SRC_PATH)/scripts/qapi2texi.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi2texi.py $< > $@,"GEN","$@") +tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.json $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o tests/qapi-schema -p "doc-good-" $<, \ + "GEN","$@") + @mv tests/qapi-schema/doc-good-qapi.texi $@ + @rm -f tests/qapi-schema/doc-good-qapi-*.[ch] tests/qapi-schema/doc-good-qmp-*.[ch] tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y)@@ -937,6 +936,7 @@ check-clean: $(MAKE) -C tests/tcg clean rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) + rm -f tests/test-qapi-gen-timestamp clean: check-cleantests/qapi-schema/test-qapi.py +1
-2@@ -10,7 +10,7 @@ # See the COPYING file in the top-level directory. # -from qapi import * +from qapi.common import * from pprint import pprint import os import sys --
Quoted 2 lines
2.13.6
good to me otherwise,
Re: [Qemu-devel] [PATCH RFC 06/21] qapi-gen: New common driver for code and doc generators
Quoted 7 lines · Marc-Andre Lureau
Quoted 4 lines · Markus Armbruster
Whenever qapi-schema.json changes, we run six programs eleven times to update eleven files. This is silly. Replace the six programs by a single program that spits out all eleven files.
Now we will need documentation update.
Absolutely! One reason this is RFC.
Quoted 309 lines
Also greping for the renamed files:
git grep -E 'qapi-commands|qapi2texi|qapi-event.py|qapi-intros|qapi-types.py|qapi-visit.py' docs/devel/qapi-code-gen.txt:===
scripts/qapi-types.py===docs/devel/qapi-code-gen.txt: $ pythonscripts/qapi-types.py
--output-dir="qapi-generated" \
docs/devel/qapi-code-gen.txt:===scripts/qapi-visit.py===docs/devel/qapi-code-gen.txt: $ pythonscripts/qapi-visit.py
--output-dir="qapi-generated"
docs/devel/qapi-code-gen.txt:===scripts/qapi-commands.py===docs/devel/qapi-code-gen.txt: generated byqapi-visit.pyare used todocs/devel/qapi-code-gen.txt: $ pythonscripts/qapi-commands.py
--output-dir="qapi-generated"
docs/devel/qapi-code-gen.txt:===scripts/qapi-event.py===docs/devel/qapi-code-gen.txt: $ pythonscripts/qapi-event.py
--output-dir="qapi-generated"
docs/devel/qapi-code-gen.txt:===scripts/qapi-introspect.py===docs/devel/qapi-code-gen.txt: $ pythonscripts/qapi-introspect.py
--output-dir="qapi-generated"monitor.c: * qapi-introspect.py's output actually conforms to the schema. qapi-schema.json:# Documentation generated withqapi2texi.pyis in
source order, withQuoted 285 lines
Signed-off-by Markus Armbruster <[email protected]>--- Makefile | 86 ++++++++++------------ scripts/qapi-gen.py | 41 +++++++++++ scripts/qapi/__init__.py | 0 scripts/{qapi-commands.py => qapi/commands.py} | 23 ++---- scripts/{qapi.py => qapi/common.py} | 0 scripts/{qapi2texi.py => qapi/doc.py} | 29 ++------ scripts/{qapi-event.py => qapi/events.py} | 23 ++---- scripts/{qapi-introspect.py => qapi/introspect.py} | 32 ++------ scripts/{qapi-types.py => qapi/types.py} | 34 ++------- scripts/{qapi-visit.py => qapi/visit.py} | 34 ++------- tests/Makefile.include | 56 +++++++------- tests/qapi-schema/test-qapi.py | 2 +- 12 files changed, 140 insertions(+), 220 deletions(-) create mode 100755 scripts/qapi-gen.py create mode 100644 scripts/qapi/__init__.py rename scripts/{qapi-commands.py => qapi/commands.py} (94%) rename scripts/{qapi.py => qapi/common.py} (100%) rename scripts/{qapi2texi.py => qapi/doc.py} (92%) mode change 100755 => 100644 rename scripts/{qapi-event.py => qapi/events.py} (92%) rename scripts/{qapi-introspect.py => qapi/introspect.py} (90%) rename scripts/{qapi-types.py => qapi/types.py} (90%) rename scripts/{qapi-visit.py => qapi/visit.py} (92%)Makefile +37
-49@@ -92,6 +92,7 @@ GENERATED_FILES += qmp-commands.h qapi-types.h qapi-visit.h qapi-event.h GENERATED_FILES += qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c GENERATED_FILES += qmp-introspect.h GENERATED_FILES += qmp-introspect.c +GENERATED_FILES += qapi.texi GENERATED_FILES += trace/generated-tcg-tracers.h@@ -477,25 +478,26 @@ qemu-ga$(EXESUF): QEMU_CFLAGS += -I qga/qapi-generated qemu-keymap$(EXESUF): LIBS += $(XKBCOMMON_LIBS) qemu-keymap$(EXESUF): QEMU_CFLAGS += $(XKBCOMMON_CFLAGS) -gen-out-type = $(subst .,-,$(suffix $@)) +qapi-py = $(SRC_PATH)/scripts/qapi/commands.py \ +$(SRC_PATH)/scripts/qapi/events.py \ +$(SRC_PATH)/scripts/qapi/introspect.py \ +$(SRC_PATH)/scripts/qapi/types.py \ +$(SRC_PATH)/scripts/qapi/visit.py \ +$(SRC_PATH)/scripts/qapi/common.py \ +$(SRC_PATH)/scripts/qapi/doc.py \ +$(SRC_PATH)/scripts/ordereddict.py \ +$(SRC_PATH)/scripts/qapi-gen.py -qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py - -qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h :\ -$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \ - $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \ - "GEN","$@") -qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h :\ -$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \ - $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \ - "GEN","$@") -qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c :\ -$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \ - $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \ - "GEN","$@") +qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h \ +qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h \ +qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c \ +qga/qapi-generated/qga-qapi.texi: \ +qga/qapi-generated/qapi-gen-timestamp ; +qga/qapi-generated/qapi-gen-timestamp: $(SRC_PATH)/qga/qapi-schema.json $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o qga/qapi-generated -p "qga-" $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \ $(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \@@ -512,31 +514,18 @@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \ $(SRC_PATH)/qapi/transaction.json \ $(SRC_PATH)/qapi/ui.json -qapi-types.c qapi-types.h :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-types.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \ - $(gen-out-type) -o "." -b $<, \ - "GEN","$@") -qapi-visit.c qapi-visit.h :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \ - $(gen-out-type) -o "." -b $<, \ - "GEN","$@") -qapi-event.c qapi-event.h :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-event.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-event.py \ - $(gen-out-type) -o "." $<, \ - "GEN","$@") -qmp-commands.h qmp-marshal.c :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \ - $(gen-out-type) -o "." $<, \ - "GEN","$@") -qmp-introspect.h qmp-introspect.c :\ -$(qapi-modules) $(SRC_PATH)/scripts/qapi-introspect.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-introspect.py \ - $(gen-out-type) -o "." $<, \ - "GEN","$@") +qapi-types.c qapi-types.h \ +qapi-visit.c qapi-visit.h \ +qmp-commands.h qmp-marshal.c \ +qapi-event.c qapi-event.h \ +qmp-introspect.h qmp-introspect.c \ +qapi.texi: \ +qapi-gen-timestamp ; +qapi-gen-timestamp: $(qapi-modules) $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o "." -b $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@ QGALIB_GEN=$(addprefix qga/qapi-generated/, qga-qapi-types.h qga-qapi-visit.h qga-qmp-commands.h) $(qga-obj-y): $(QGALIB_GEN)@@ -596,6 +585,7 @@ clean: rm -f trace/generated-tracers-dtrace.dtrace* rm -f trace/generated-tracers-dtrace.h* rm -f $(foreach f,$(GENERATED_FILES),$(f) $(f)-timestamp) + rm -f qapi-gen-timestamp rm -rf qapi-generated rm -rf qga/qapi-generated for d in $(ALL_SUBDIRS); do \@@ -803,13 +793,11 @@ qemu-monitor-info.texi: $(SRC_PATH)/hmp-commands-info.hx $(SRC_PATH)/scripts/hxt qemu-img-cmds.texi: $(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/scripts/hxtool $(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > $@,"GEN","$@") -docs/interop/qemu-qmp-qapi.texi docs/interop/qemu-ga-qapi.texi: $(SRC_PATH)/scripts/qapi2texi.py $(qapi-py) +docs/interop/qemu-qmp-qapi.texi: qapi.texi + @cp -p $< $@ -docs/interop/qemu-qmp-qapi.texi: $(qapi-modules) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi2texi.py $< > $@,"GEN","$@") - -docs/interop/qemu-ga-qapi.texi: $(SRC_PATH)/qga/qapi-schema.json - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi2texi.py $< > $@,"GEN","$@") +docs/interop/qemu-ga-qapi.texi: qga/qapi-generated/qga-qapi.texi + @cp -p $< $@ qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi qemu-monitor-info.texi qemu.1: qemu-option-trace.texiscripts/qapi-gen.py new file +41
-0@@ -0,0 +1,41 @@ +#!/usr/bin/env python +# QAPI generator +# +# This work is licensed under the terms of the GNU GPL, version 2 or later. +# See the COPYING file in the top-level directory. + +import sys +from qapi.common import parse_command_line, QAPISchema +from qapi.types import gen_types +from qapi.visit import gen_visit +from qapi.commands import gen_commands +from qapi.events import gen_events +from qapi.introspect import gen_introspect +from qapi.doc import gen_doc + + +def main(argv): + (input_file, output_dir, do_c, do_h, prefix, opts) = \ + parse_command_line('bu', ['builtins', 'unmask-non-abi-names']) + + opt_builtins = False + opt_unmask = False + + for o, a in opts: + if o in ('-b', '--builtins'): + opt_builtins = True + if o in ('-u', '--unmask-non-abi-names'): + opt_unmask = True + + schema = QAPISchema(input_file) + + gen_types(schema, output_dir, prefix, opt_builtins) + gen_visit(schema, output_dir, prefix, opt_builtins) + gen_commands(schema, output_dir, prefix) + gen_events(schema, output_dir, prefix) + gen_introspect(schema, output_dir, prefix, opt_unmask) + gen_doc(schema, output_dir, prefix) + + +if __name__ == '__main__': + main(sys.argv)scripts/qapi/__init__.py new file +0
-0scripts/qapi/commands.py renamed from scripts/qapi-commands.py +5
-18@@ -13,7 +13,7 @@ This work is licensed under the terms of the GNU GPL, version 2. See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * def gen_command_decl(name, arg_type, boxed, ret_type):@@ -255,13 +255,8 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor): self._regy += gen_register_command(name, success_response) -def main(argv): - (input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line() - - blurb = ''' - * Schema-defined QAPI/QMP commands -''' - +def gen_commands(schema, output_dir, prefix): + blurb = ' * Schema-defined QAPI/QMP commands' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)@@ -290,17 +285,9 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds); ''', prefix=prefix, c_prefix=c_name(prefix, protect=False))) - schema = QAPISchema(input_file) vis = QAPISchemaGenCommandVisitor(prefix) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qmp-marshal.c') - if do_h: - genh.write(output_dir, prefix + 'qmp-commands.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qmp-marshal.c') + genh.write(output_dir, prefix + 'qmp-commands.h')scripts/qapi/common.py renamed from scripts/qapi.py +0
-0scripts/qapi/doc.py renamed from scripts/qapi2texi.py +2
-3@@ -4,10 +4,9 @@ # This work is licensed under the terms of the GNU LGPL, version 2+. # See the COPYING file in the top-level directory. """This script produces the documentation of a qapi schema in texinfo format""" + import re -import sys - -import qapi +import qapi.common MSG_FMT = """ @deftypefn {type} {{}} {name}@@ -196,7 +195,7 @@ def texi_entity(doc, what, base=None, variants=None, + texi_sections(doc))
Quoted 5 lines
scripts/qapi/doc.py def texi_entity(doc, what, base=None, variants=None, · line 198
Quoted 2 lines
scripts/qapi/doc.py renamed from scripts/qapi2texi.py +1
-1-class QAPISchemaGenDocVisitor(qapi.QAPISchemaVisitor): +class QAPISchemaGenDocVisitor(qapi.common.QAPISchemaVisitor):Would be a bit easier to read and more consitent with a top-level "from qapi.common import QAPISchemaVisitor"
Can do.
Quoted 276 lines
Quoted 276 lines
scripts/qapi/doc.py renamed from scripts/qapi2texi.py +5
-17def __init__(self): self.out = None self.cur_doc = None@@ -271,20 +270,8 @@ def texi_schema(schema): return gen.out -def main(argv): - """Takes schema argument, prints result to stdout""" - if len(argv) != 2: - print >>sys.stderr, "%s: need exactly 1 argument: SCHEMA" % argv[0] - sys.exit(1) - - schema = qapi.QAPISchema(argv[1]) - if not qapi.doc_required: - print >>sys.stderr, ("%s: need pragma 'doc-required' " - "to generate documentation" % argv[0]) - sys.exit(1) - print '@c AUTOMATICALLY GENERATED, DO NOT MODIFY\n' - print texi_schema(schema), - - -if __name__ == '__main__': - main(sys.argv) +def gen_doc(schema, output_dir, prefix): + if qapi.common.doc_required: + gen = qapi.common.QAPIGenDoc() + gen.body(texi_schema(schema)) + gen.write(output_dir, prefix + 'qapi.texi')scripts/qapi/events.py renamed from scripts/qapi-event.py +5
-18@@ -12,7 +12,7 @@ This work is licensed under the terms of the GNU GPL, version 2. See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * def build_event_send_proto(name, arg_type, boxed):@@ -171,13 +171,8 @@ class QAPISchemaGenEventVisitor(QAPISchemaVisitor): self._event_names.append(name) -def main(argv): - (input_file, output_dir, do_c, do_h, prefix, dummy) = parse_command_line() - - blurb = ''' - * Schema-defined QAPI/QMP events -''' - +def gen_events(schema, output_dir, prefix): + blurb = ' * Schema-defined QAPI/QMP events' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)@@ -201,17 +196,9 @@ def main(argv): ''', prefix=prefix)) - schema = QAPISchema(input_file) vis = QAPISchemaGenEventVisitor(prefix) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qapi-event.c') - if do_h: - genh.write(output_dir, prefix + 'qapi-event.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qapi-event.c') + genh.write(output_dir, prefix + 'qapi-event.h')scripts/qapi/introspect.py renamed from scripts/qapi-introspect.py +5
-27@@ -10,7 +10,7 @@ This work is licensed under the terms of the GNU GPL, version 2. See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * # Caveman's json.dumps() replacement (we're stuck at Python 2.4)@@ -168,22 +168,8 @@ const char %(c_name)s[] = %(c_string)s; self._gen_json(name, 'event', {'arg-type': self._use_type(arg_type)}) -def main(argv): - # Debugging aid: unmask QAPI schema's type names - # We normally mask them, because they're not QMP wire ABI - opt_unmask = False - - (input_file, output_dir, do_c, do_h, prefix, opts) = \ - parse_command_line('u', ['unmask-non-abi-names']) - - for o, a in opts: - if o in ('-u', '--unmask-non-abi-names'): - opt_unmask = True - - blurb = ''' - * QAPI/QMP schema introspection -''' - +def gen_introspect(schema, output_dir, prefix, opt_unmask): + blurb = ' * QAPI/QMP schema introspection' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)@@ -194,17 +180,9 @@ def main(argv): ''', prefix=prefix)) - schema = QAPISchema(input_file) vis = QAPISchemaGenIntrospectVisitor(prefix, opt_unmask) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qmp-introspect.c') - if do_h: - genh.write(output_dir, prefix + 'qmp-introspect.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qmp-introspect.c') + genh.write(output_dir, prefix + 'qmp-introspect.h')scripts/qapi/types.py renamed from scripts/qapi-types.py +5
-29@@ -13,7 +13,7 @@ This work is licensed under the terms of the GNU GPL, version 2. # See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * # variants must be emitted before their container; track what has already@@ -241,24 +241,8 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self._gen_type_cleanup(name) -def main(argv): - # If you link code generated from multiple schemata, you want only one - # instance of the code for built-in types. Generate it only when - # opt_builtins, enabled by command line option -b. See also - # QAPISchemaGenTypeVisitor.visit_end(). - opt_builtins = False - - (input_file, output_dir, do_c, do_h, prefix, opts) = \ - parse_command_line('b', ['builtins']) - - for o, a in opts: - if o in ('-b', '--builtins'): - opt_builtins = True - - blurb = ''' - * Schema-defined QAPI types -''' - +def gen_types(schema, output_dir, prefix, opt_builtins): + blurb = ' * Schema-defined QAPI types' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)@@ -274,17 +258,9 @@ def main(argv): #include "qapi/util.h" ''')) - schema = QAPISchema(input_file) vis = QAPISchemaGenTypeVisitor(opt_builtins) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qapi-types.c') - if do_h: - genh.write(output_dir, prefix + 'qapi-types.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qapi-types.c') + genh.write(output_dir, prefix + 'qapi-types.h')scripts/qapi/visit.py renamed from scripts/qapi-visit.py +5
-29@@ -13,7 +13,7 @@ This work is licensed under the terms of the GNU GPL, version 2. See the COPYING file in the top-level directory. """ -from qapi import * +from qapi.common import * def gen_visit_decl(name, scalar=False):@@ -324,24 +324,8 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): self.defn += gen_visit_alternate(name, variants) -def main(argv): - # If you link code generated from multiple schemata, you want only one - # instance of the code for built-in types. Generate it only when - # opt_builtins, enabled by command line option -b. See also - # QAPISchemaGenVisitVisitor.visit_end(). - opt_builtins = False - - (input_file, output_dir, do_c, do_h, prefix, opts) = \ - parse_command_line('b', ['builtins']) - - for o, a in opts: - if o in ('-b', '--builtins'): - opt_builtins = True - - blurb = ''' - * Schema-defined QAPI visitors -''' - +def gen_visit(schema, output_dir, prefix, opt_builtins): + blurb = ' * Schema-defined QAPI visitors' genc = QAPIGenC(blurb, __doc__) genh = QAPIGenH(blurb, __doc__)@@ -361,17 +345,9 @@ def main(argv): ''', prefix=prefix)) - schema = QAPISchema(input_file) vis = QAPISchemaGenVisitVisitor(opt_builtins) schema.visit(vis) genc.body(vis.defn) genh.body(vis.decl) - - if do_c: - genc.write(output_dir, prefix + 'qapi-visit.c') - if do_h: - genh.write(output_dir, prefix + 'qapi-visit.h') - - -if __name__ == '__main__': - main(sys.argv) + genc.write(output_dir, prefix + 'qapi-visit.c') + genh.write(output_dir, prefix + 'qapi-visit.h')tests/Makefile.include +0
-0@@ -23,7 +23,16 @@ check-help: ifneq ($(wildcard config-host.mak),) export SRC_PATH
Quoted 5 lines
tests/Makefile.include check-help: · line 26
Quoted 2 lines
tests/Makefile.include +1
-1-qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py +# TODO don't duplicate $(SRC_PATH)/Makefile's qapi-py herehmm, it doesn't look necessary, since the makefile is included from there. Right?
That's what I thought, so I deleted it, breaking my build. I still don't quite understand what's happening here.
Quoted 90 lines
Quoted 86 lines
tests/Makefile.include +27
-27+qapi-py = $(SRC_PATH)/scripts/qapi/commands.py \ +$(SRC_PATH)/scripts/qapi/events.py \ +$(SRC_PATH)/scripts/qapi/introspect.py \ +$(SRC_PATH)/scripts/qapi/types.py \ +$(SRC_PATH)/scripts/qapi/visit.py \ +$(SRC_PATH)/scripts/qapi/common.py \ +$(SRC_PATH)/scripts/qapi/doc.py \ +$(SRC_PATH)/scripts/ordereddict.py \ +$(SRC_PATH)/scripts/qapi-gen.py # Get the list of all supported sysemu targets SYSEMU_TARGET_LIST := $(subst -softmmu.mak,,$(notdir \@@ -642,34 +651,24 @@ tests/test-logging$(EXESUF): tests/test-logging.o $(test-util-obj-y) tests/test-replication$(EXESUF): tests/test-replication.o $(test-util-obj-y) \ $(test-block-obj-y) -tests/test-qapi-types.c tests/test-qapi-types.h :\ -$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \ - $(gen-out-type) -o tests -p "test-" $<, \ - "GEN","$@") -tests/test-qapi-visit.c tests/test-qapi-visit.h :\ -$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \ - $(gen-out-type) -o tests -p "test-" $<, \ - "GEN","$@") -tests/test-qmp-commands.h tests/test-qmp-marshal.c :\ -$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \ - $(gen-out-type) -o tests -p "test-" $<, \ - "GEN","$@") -tests/test-qapi-event.c tests/test-qapi-event.h :\ -$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-event.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-event.py \ - $(gen-out-type) -o tests -p "test-" $<, \ - "GEN","$@") -tests/test-qmp-introspect.c tests/test-qmp-introspect.h :\ -$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-introspect.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-introspect.py \ - $(gen-out-type) -o tests -p "test-" $<, \ - "GEN","$@") +tests/test-qapi-types.c tests/test-qapi-types.h \ +tests/test-qapi-visit.c tests/test-qapi-visit.h \ +tests/test-qmp-commands.h tests/test-qmp-marshal.c \ +tests/test-qapi-event.c tests/test-qapi-event.h \ +tests/test-qmp-introspect.c tests/test-qmp-introspect.h: \ +tests/test-qapi-gen-timestamp ; +tests/test-qapi-gen-timestamp: $(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o tests -p "test-" $<, \ + "GEN","$(@:%-timestamp=%)") + @>$@ -tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.json $(SRC_PATH)/scripts/qapi2texi.py $(qapi-py) - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi2texi.py $< > $@,"GEN","$@") +tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.json $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ + -o tests/qapi-schema -p "doc-good-" $<, \ + "GEN","$@") + @mv tests/qapi-schema/doc-good-qapi.texi $@ + @rm -f tests/qapi-schema/doc-good-qapi-*.[ch] tests/qapi-schema/doc-good-qmp-*.[ch] tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y)@@ -937,6 +936,7 @@ check-clean: $(MAKE) -C tests/tcg clean rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) + rm -f tests/test-qapi-gen-timestamp clean: check-cleantests/qapi-schema/test-qapi.py +1
-2@@ -10,7 +10,7 @@ # See the COPYING file in the top-level directory. # -from qapi import * +from qapi.common import * from pprint import pprint import os import sys --Quoted 2 lines
2.13.6
good to me otherwise,
Quoted 1 line
Reviewed-by Marc-André Lureau <[email protected]>
Thanks!
Re: [Qemu-devel] [PATCH RFC 06/21] qapi-gen: New common driver for code and doc generators
Quoted 3 lines · Markus Armbruster
Quoted 1 line · Marc-Andre Lureau
On Fri, Feb 2, 2018 at 2:03 PM, Markus Armbruster <[email protected]> wrote:
[...]
Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
scripts/qapi/doc.py renamed from scripts/qapi2texi.py +2
-3@@ -4,10 +4,9 @@ # This work is licensed under the terms of the GNU LGPL, version 2+. # See the COPYING file in the top-level directory. """This script produces the documentation of a qapi schema in texinfo format""" + import re -import sys - -import qapi +import qapi.common MSG_FMT = """ @deftypefn {type} {{}} {name}@@ -196,7 +195,7 @@ def texi_entity(doc, what, base=None, variants=None, + texi_sections(doc))
Quoted 7 lines
Quoted 5 lines
scripts/qapi/doc.py def texi_entity(doc, what, base=None, variants=None, · line 198
Quoted 2 lines
scripts/qapi/doc.py renamed from scripts/qapi2texi.py +1
-1-class QAPISchemaGenDocVisitor(qapi.QAPISchemaVisitor): +class QAPISchemaGenDocVisitor(qapi.common.QAPISchemaVisitor):Would be a bit easier to read and more consitent with a top-level "from qapi.common import QAPISchemaVisitor"
Can do.
The obvious patch (appended) doesn't work, because doc_required is always False in gen_doc(). WTF?!?
[...]
scripts/qapi/doc.py +4 -4
@@ -7,7 +7,7 @@ from __future__ import print_function import re -import qapi.common +from qapi.common import doc_required, QAPIGenDoc, QAPISchemaVisitor MSG_FMT = """ @deftypefn {type} {{}} {name}
@@ -196,7 +196,7 @@ def texi_entity(doc, what, base=None, variants=None, + texi_sections(doc)) -class QAPISchemaGenDocVisitor(qapi.common.QAPISchemaVisitor): +class QAPISchemaGenDocVisitor(QAPISchemaVisitor): def __init__(self): self.out = None self.cur_doc = None
@@ -272,7 +272,7 @@ def texi_schema(schema): def gen_doc(schema, output_dir, prefix): - if qapi.common.doc_required: - gen = qapi.common.QAPIGenDoc() + if doc_required: + gen = QAPIGenDoc() gen.add(texi_schema(schema)) gen.write(output_dir, prefix + 'qapi.texi')
RFC PATCH 7/21 qapi: Move parse_command_line() next to its only use
In reply to Markus Armbruster--- scripts/qapi-gen.py | 52 +++++++++++++++++++++++++++++++++++++++++++++++- scripts/qapi/common.py | 54 -------------------------------------------------- 2 files changed, 51 insertions(+), 55 deletions(-)
scripts/qapi-gen.py +51 -1
@@ -4,8 +4,10 @@ # This work is licensed under the terms of the GNU GPL, version 2 or later. # See the COPYING file in the top-level directory. +import getopt +import re import sys -from qapi.common import parse_command_line, QAPISchema +from qapi.common import QAPISchema from qapi.types import gen_types from qapi.visit import gen_visit from qapi.commands import gen_commands
@@ -14,6 +16,54 @@ from qapi.introspect import gen_introspect from qapi.doc import gen_doc +def parse_command_line(extra_options='', extra_long_options=[]): + + try: + opts, args = getopt.gnu_getopt(sys.argv[1:], + 'chp:o:' + extra_options, + ['source', 'header', 'prefix=', + 'output-dir='] + extra_long_options) + except getopt.GetoptError as err: + print >>sys.stderr, "%s: %s" % (sys.argv[0], str(err)) + sys.exit(1) + + output_dir = '' + prefix = '' + do_c = False + do_h = False + extra_opts = [] + + for oa in opts: + o, a = oa + if o in ('-p', '--prefix'): + match = re.match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', a) + if match.end() != len(a): + print >>sys.stderr, \ + "%s: 'funny character '%s' in argument of --prefix" \ + % (sys.argv[0], a[match.end()]) + sys.exit(1) + prefix = a + elif o in ('-o', '--output-dir'): + output_dir = a + '/' + elif o in ('-c', '--source'): + do_c = True + elif o in ('-h', '--header'): + do_h = True + else: + extra_opts.append(oa) + + if not do_c and not do_h: + do_c = True + do_h = True + + if len(args) != 1: + print >>sys.stderr, "%s: need exactly one argument" % sys.argv[0] + sys.exit(1) + fname = args[0] + + return (fname, output_dir, do_c, do_h, prefix, extra_opts) + + def main(argv): (input_file, output_dir, do_c, do_h, prefix, opts) = \ parse_command_line('bu', ['builtins', 'unmask-non-abi-names'])
scripts/qapi/common.py +0 -54
66 lines not on this page · raw message
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 07/21] qapi: Move parse_command_line() next to its only use
Quoted 6 lines · Markus Armbruster
Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi-gen.py | 52 +++++++++++++++++++++++++++++++++++++++++++++++- scripts/qapi/common.py | 54 -------------------------------------------------- 2 files changed, 51 insertions(+), 55 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 07/21] qapi: Move parse_command_line() next to its only use
In reply to Markus ArmbrusterQuoted 1 line · Markus Armbruster
Signed-off-by Markus Armbruster <[email protected]>
Quoted 151 lines
--- scripts/qapi-gen.py | 52 +++++++++++++++++++++++++++++++++++++++++++++++- scripts/qapi/common.py | 54 -------------------------------------------------- 2 files changed, 51 insertions(+), 55 deletions(-)scripts/qapi-gen.py +51
-1@@ -4,8 +4,10 @@ # This work is licensed under the terms of the GNU GPL, version 2 or later. # See the COPYING file in the top-level directory. +import getopt +import re import sys -from qapi.common import parse_command_line, QAPISchema +from qapi.common import QAPISchema from qapi.types import gen_types from qapi.visit import gen_visit from qapi.commands import gen_commands@@ -14,6 +16,54 @@ from qapi.introspect import gen_introspect from qapi.doc import gen_doc +def parse_command_line(extra_options='', extra_long_options=[]): + + try: + opts, args = getopt.gnu_getopt(sys.argv[1:], + 'chp:o:' + extra_options, + ['source', 'header', 'prefix=', + 'output-dir='] + extra_long_options) + except getopt.GetoptError as err: + print >>sys.stderr, "%s: %s" % (sys.argv[0], str(err)) + sys.exit(1) + + output_dir = '' + prefix = '' + do_c = False + do_h = False + extra_opts = [] + + for oa in opts: + o, a = oa + if o in ('-p', '--prefix'): + match = re.match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', a) + if match.end() != len(a): + print >>sys.stderr, \ + "%s: 'funny character '%s' in argument of --prefix" \ + % (sys.argv[0], a[match.end()]) + sys.exit(1) + prefix = a + elif o in ('-o', '--output-dir'): + output_dir = a + '/' + elif o in ('-c', '--source'): + do_c = True + elif o in ('-h', '--header'): + do_h = True + else: + extra_opts.append(oa) + + if not do_c and not do_h: + do_c = True + do_h = True + + if len(args) != 1: + print >>sys.stderr, "%s: need exactly one argument" % sys.argv[0] + sys.exit(1) + fname = args[0] + + return (fname, output_dir, do_c, do_h, prefix, extra_opts) + + def main(argv): (input_file, output_dir, do_c, do_h, prefix, opts) = \ parse_command_line('bu', ['builtins', 'unmask-non-abi-names'])scripts/qapi/common.py +0
-55@@ -12,7 +12,6 @@ # See the COPYING file in the top-level directory. import errno -import getopt import os import re import string@@ -1917,59 +1916,6 @@ def build_params(arg_type, boxed, extra): # -# Common command line parsing -# - - -def parse_command_line(extra_options='', extra_long_options=[]): - - try: - opts, args = getopt.gnu_getopt(sys.argv[1:], - 'chp:o:' + extra_options, - ['source', 'header', 'prefix=', - 'output-dir='] + extra_long_options) - except getopt.GetoptError as err: - print >>sys.stderr, "%s: %s" % (sys.argv[0], str(err)) - sys.exit(1) - - output_dir = '' - prefix = '' - do_c = False - do_h = False - extra_opts = [] - - for oa in opts: - o, a = oa - if o in ('-p', '--prefix'): - match = re.match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', a) - if match.end() != len(a): - print >>sys.stderr, \ - "%s: 'funny character '%s' in argument of --prefix" \ - % (sys.argv[0], a[match.end()]) - sys.exit(1) - prefix = a - elif o in ('-o', '--output-dir'): - output_dir = a + '/' - elif o in ('-c', '--source'): - do_c = True - elif o in ('-h', '--header'): - do_h = True - else: - extra_opts.append(oa) - - if not do_c and not do_h: - do_c = True - do_h = True - - if len(args) != 1: - print >>sys.stderr, "%s: need exactly one argument" % sys.argv[0] - sys.exit(1) - fname = args[0] - - return (fname, output_dir, do_c, do_h, prefix, extra_opts) - - -# # Accumulate and write output # --2.13.6
RFC PATCH 8/21 qapi: Touch generated files only when they change
In reply to Markus ArmbrusterA massive number of objects depends on QAPI-generated headers. In my "build everything" tree, it's roughly 4500 out of 4800. This is particularly annoying when only some of the generated files change, say for a doc fix.
Improve qapi-gen.py to touch its output files only if they actually change. Rebuild time for a QAPI doc fix drops from many minutes to a few seconds. Rebuilds get faster for certain code changes, too. For instance, adding a simple QMP event now recompiles less than 200 instead of 4500 objects. But adding a QAPI type is as bad as ever; we clearly got more work to do.
--- scripts/qapi/common.py | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
scripts/qapi/common.py +9 -2
@@ -1944,9 +1944,16 @@ class QAPIGen(object): except os.error as e: if e.errno != errno.EEXIST: raise - f = open(os.path.join(output_dir, fname), 'w') - f.write(self.top(fname) + self._preamble + self._body + fd = os.open(os.path.join(output_dir, fname), + os.O_RDWR | os.O_CREAT, 0666) + f = os.fdopen(fd, 'r+') + text = (self.top(fname) + self._preamble + self._body + self.bottom(fname)) + oldtext = f.read(len(text) + 1) + if text != oldtext: + f.seek(0) + f.truncate(0) + f.write(text) f.close()
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 08/21] qapi: Touch generated files only when they change
Quoted 11 lines · Markus Armbruster
A massive number of objects depends on QAPI-generated headers. In my "build everything" tree, it's roughly 4500 out of 4800. This is particularly annoying when only some of the generated files change, say for a doc fix.
Improve
qapi-gen.pyto touch its output files only if they actually change. Rebuild time for a QAPI doc fix drops from many minutes to a few seconds. Rebuilds get faster for certain code changes, too. For instance, adding a simple QMP event now recompiles less than 200 instead of 4500 objects. But adding a QAPI type is as bad as ever; we clearly got more work to do.
The last phrase sounds quite colloquial. It may have been intentional; but if not, s/we/we've/
Quoted 16 lines
Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)scripts/qapi/common.py +0
-2@@ -1944,9 +1944,16 @@ class QAPIGen(object): except os.error as e: if e.errno != errno.EEXIST: raise - f = open(os.path.join(output_dir, fname), 'w') - f.write(self.top(fname) + self._preamble + self._body
scripts/qapi/common.py class QAPIGen(object): · line 1956
Quoted 10 lines
scripts/qapi/common.py +9
-0+ fd = os.open(os.path.join(output_dir, fname), + os.O_RDWR | os.O_CREAT, 0666) + f = os.fdopen(fd, 'r+') + text = (self.top(fname) + self._preamble + self._body + self.bottom(fname)) + oldtext = f.read(len(text) + 1) + if text != oldtext: + f.seek(0) + f.truncate(0) + f.write(text)
In-memory rewrite rather than playing games with a secondary file combined with the rename(2) syscall. It means you're not atomic (an external reader has a window of time where they can see an incomplete version of the file). But I think make dependency rules mean we don't care - even in a parallel rule, as long as all clients depend on the timestamp file, and the timestamp file isn't updated until all our rewrites (if any) have settled, then we don't create any of those external readers that can find the window of incomplete contents.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 08/21] qapi: Touch generated files only when they change
Quoted 13 lines · Eric Blake
Quoted 11 lines · Markus Armbruster
A massive number of objects depends on QAPI-generated headers. In my "build everything" tree, it's roughly 4500 out of 4800. This is particularly annoying when only some of the generated files change, say for a doc fix.
Improve
qapi-gen.pyto touch its output files only if they actually change. Rebuild time for a QAPI doc fix drops from many minutes to a few seconds. Rebuilds get faster for certain code changes, too. For instance, adding a simple QMP event now recompiles less than 200 instead of 4500 objects. But adding a QAPI type is as bad as ever; we clearly got more work to do.
Quoted 2 lines
The last phrase sounds quite colloquial. It may have been intentional; but if not, s/we/we've/
I'll change it.
Quoted 22 lines
Quoted 22 lines
Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)scripts/qapi/common.py +6
-2@@ -1944,9 +1944,16 @@ class QAPIGen(object): except os.error as e: if e.errno != errno.EEXIST: raise - f = open(os.path.join(output_dir, fname), 'w') - f.write(self.top(fname) + self._preamble + self._body + fd = os.open(os.path.join(output_dir, fname), + os.O_RDWR | os.O_CREAT, 0666) + f = os.fdopen(fd, 'r+') + text = (self.top(fname) + self._preamble + self._body + self.bottom(fname)) + oldtext = f.read(len(text) + 1) + if text != oldtext:
Quoted 8 lines
scripts/qapi/common.py class QAPIGen(object): · line 1956
Quoted 3 lines
scripts/qapi/common.py +3
-0+ f.seek(0) + f.truncate(0) + f.write(text)In-memory rewrite rather than playing games with a secondary file combined with the rename(2) syscall. It means you're not atomic (an external reader has a window of time where they can see an incomplete version of the file).
Same as before.
Quoted 7 lines
But I think make dependency rules mean we don't
care - even in a parallel rule, as long as all clients depend on the timestamp file, and the timestamp file isn't updated until all our rewrites (if any) have settled, then we don't create any of those external readers that can find the window of incomplete contents.Reviewed-by Eric Blake <[email protected]>
Thanks!
RFC PATCH 9/21 qapi: Don't absolutize include file name in error messages
In reply to Markus ArmbrusterError messages print absolute filenames of included files even gave a relative one on the command line:
PYTHONPATH=scripts python -B tests/qapi-schema/test-qapi.py tests/qapi-schema/include-cycle.json
In file included from tests/qapi-schema/include-cycle.json:1:
In file included from /work/armbru/qemu/tests/qapi-schema/include-cycle-b.json:1:
/work/armbru/qemu/tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json
Improve this to
In file included from tests/qapi-schema/include-cycle.json:1:
In file included from tests/qapi-schema/include-cycle-b.json:1:
tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json
--- scripts/qapi/common.py | 12 ++++++------ tests/qapi-schema/include-no-file.err | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-)
scripts/qapi/common.py +6 -6
@@ -255,9 +255,8 @@ class QAPIDoc(object): class QAPISchemaParser(object): def __init__(self, fp, previously_included=[], incl_info=None): - abs_fname = os.path.abspath(fp.name) self.fname = fp.name - previously_included.append(abs_fname) + previously_included.append(os.path.abspath(fp.name)) self.incl_info = incl_info self.src = fp.read() if self.src == '' or self.src[-1] != '\n':
@@ -288,7 +287,7 @@ class QAPISchemaParser(object): if not isinstance(include, str): raise QAPISemError(info, "Value of 'include' must be a string") - self._include(include, info, os.path.dirname(abs_fname), + self._include(include, info, os.path.dirname(self.fname), previously_included) elif "pragma" in expr: self.reject_expr_doc(cur_doc)
@@ -321,7 +320,8 @@ class QAPISchemaParser(object): % doc.symbol) def _include(self, include, info, base_dir, previously_included): - incl_abs_fname = os.path.join(base_dir, include) + incl_fname = os.path.join(base_dir, include) + incl_abs_fname = os.path.abspath(incl_fname) # catch inclusion cycle inf = info while inf:
@@ -333,9 +333,9 @@ class QAPISchemaParser(object): if incl_abs_fname in previously_included: return try: - fobj = open(incl_abs_fname, 'r') + fobj = open(incl_fname, 'r') except IOError as e: - raise QAPISemError(info, '%s: %s' % (e.strerror, include)) + raise QAPISemError(info, '%s: %s' % (e.strerror, incl_fname)) exprs_include = QAPISchemaParser(fobj, previously_included, info) self.exprs.extend(exprs_include.exprs) self.docs.extend(exprs_include.docs)
tests/qapi-schema/include-no-file.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/include-no-file.json:1: No such file or directory: include-no-file-sub.json +tests/qapi-schema/include-no-file.json:1: No such file or directory: tests/qapi-schema/include-no-file-sub.json
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 09/21] qapi: Don't absolutize include file name in error messages
Quoted 1 line · Markus Armbruster
Error messages print absolute filenames of included files even gave a
s/even gave/even when given/
Quoted 7 lines
relative one on the command line:
PYTHONPATH=scripts python -B
tests/qapi-schema/test-qapi.pytests/qapi-schema/include-cycle.json
In file included fromtests/qapi-schema/include-cycle.json:1:
In file included from /work/armbru/qemu/tests/qapi-schema/include-cycle-b.json:1:
/work/armbru/qemu/tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json
Quoted 5 lines
Improve this to
In file included from
tests/qapi-schema/include-cycle.json:1:
In file included fromtests/qapi-schema/include-cycle-b.json:1:
tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json
Nice, and makes developing new qapi tests a little less painful since it's less modification to qapi-schema/*.err additions.
Quoted 7 lines
Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 12 ++++++------ tests/qapi-schema/include-no-file.err | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 09/21] qapi: Don't absolutize include file name in error messages
Quoted 4 lines · Eric Blake
Quoted 1 line · Markus Armbruster
Error messages print absolute filenames of included files even gave a
s/even gave/even when given/
I meant to write "even if the user gave". Is that okay?
Quoted 7 lines
Quoted 7 lines
relative one on the command line:
PYTHONPATH=scripts python -B
tests/qapi-schema/test-qapi.pytests/qapi-schema/include-cycle.json
In file included fromtests/qapi-schema/include-cycle.json:1:
In file included from /work/armbru/qemu/tests/qapi-schema/include-cycle-b.json:1:
/work/armbru/qemu/tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json
Quoted 8 lines
Quoted 5 lines
Improve this to
In file included from
tests/qapi-schema/include-cycle.json:1:
In file included fromtests/qapi-schema/include-cycle-b.json:1:
tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.jsonNice, and makes developing new qapi tests a little less painful since it's less modification to qapi-schema/*.err additions.
Probably not, as our make rule strips off $(SRC_PATH):
@perl -p -e 's|\Q$(SRC_PATH)\E/||g' $*.test.err | diff -q $(SRC_PATH)/$*.err -
I've kept that, because it might also occur in stack backtraces. I think.
Quoted 8 lines
Quoted 6 lines
Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 12 ++++++------ tests/qapi-schema/include-no-file.err | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-)Reviewed-by Eric Blake <[email protected]>
Thanks!
Re: [Qemu-devel] [PATCH RFC 09/21] qapi: Don't absolutize include file name in error messages
Quoted 8 lines · Markus Armbruster
Quoted 4 lines · Eric Blake
Quoted 1 line · Markus Armbruster
Error messages print absolute filenames of included files even gave a
s/even gave/even when given/
I meant to write "even if the user gave". Is that okay?
Yes, that works too.
Quoted 17 lines
Quoted 8 lines
Quoted 8 lines
relative one on the command line:
PYTHONPATH=scripts python -B
tests/qapi-schema/test-qapi.pytests/qapi-schema/include-cycle.json
In file included fromtests/qapi-schema/include-cycle.json:1:
In file included from /work/armbru/qemu/tests/qapi-schema/include-cycle-b.json:1:
/work/armbru/qemu/tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.jsonQuoted 8 lines
Quoted 5 lines
Improve this to
In file included from
tests/qapi-schema/include-cycle.json:1:
In file included fromtests/qapi-schema/include-cycle-b.json:1:
tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.jsonNice, and makes developing new qapi tests a little less painful since it's less modification to qapi-schema/*.err additions.
Quoted 6 lines
Probably not, as our make rule strips off $(SRC_PATH):
@perl -p -e 's|\Q$(SRC_PATH)\E/||g' $*.test.err | diff -q $(SRC_PATH)/$*.err -I've kept that, because it might also occur in stack backtraces. I think.
I still recall having to hand-edit .err files when doing a naive "run the test to get the failure, then 'mv' the bad files into the expected filenames, then rerun the tests"; so I'm not sure if our make rule was properly munging absolute names out of the file in the first place. I'm not too fussed about it, though, as adding new tests is less frequent and it's still fairly easy to learn if you did it right or not by whether 'make check' succeeds.
Quoted 12 lines
Quoted 9 lines
Quoted 7 lines
Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 12 ++++++------ tests/qapi-schema/include-no-file.err | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-)Reviewed-by Eric Blake <[email protected]>Thanks!
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 09/21] qapi: Don't absolutize include file name in error messages
Quoted 31 lines · Eric Blake
Quoted 8 lines · Markus Armbruster
Quoted 4 lines · Eric Blake
Quoted 1 line · Markus Armbruster
Error messages print absolute filenames of included files even gave a
s/even gave/even when given/
I meant to write "even if the user gave". Is that okay?
Yes, that works too.
Quoted 19 lines
Quoted 8 lines
Quoted 8 lines
relative one on the command line:
PYTHONPATH=scripts python -B
tests/qapi-schema/test-qapi.pytests/qapi-schema/include-cycle.json
In file included fromtests/qapi-schema/include-cycle.json:1:
In file included from /work/armbru/qemu/tests/qapi-schema/include-cycle-b.json:1:
/work/armbru/qemu/tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.jsonQuoted 8 lines
Quoted 5 lines
Improve this to
In file included from
tests/qapi-schema/include-cycle.json:1:
In file included fromtests/qapi-schema/include-cycle-b.json:1:
tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.jsonNice, and makes developing new qapi tests a little less painful since it's less modification to qapi-schema/*.err additions.
Probably not, as our make rule strips off $(SRC_PATH):
Quoted 8 lines
Quoted 4 lines
@perl -p -e 's|\Q$(SRC_PATH)\E/||g' $*.test.err | diff -q $(SRC_PATH)/$*.err -I've kept that, because it might also occur in stack backtraces. I think.
I still recall having to hand-edit .err files when doing a naive "run the test to get the failure, then 'mv' the bad files into the expected filenames, then rerun the tests";
Whenever the Perl script does something, you can't simply move the .test.err to .err. Annoying.
A make target check-accept could automate the work.
However, with this patch, the Perl script should do something less often. I guess that's your point.
Sorry for being slow on the update :)
Quoted 17 lines
so I'm not sure if our make rule was
properly munging absolute names out of the file in the first place. I'm not too fussed about it, though, as adding new tests is less frequent and it's still fairly easy to learn if you did it right or not by whether 'make check' succeeds.Quoted 11 lines
Quoted 8 lines
Quoted 6 lines
Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 12 ++++++------ tests/qapi-schema/include-no-file.err | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-)Reviewed-by Eric Blake <[email protected]>Thanks!
Re: [Qemu-devel] [PATCH RFC 09/21] qapi: Don't absolutize include file name in error messages
Quoted 6 lines · Markus Armbruster
Quoted 6 lines
Quoted 6 lines
Probably not, as our make rule strips off $(SRC_PATH):
@perl -p -e 's|\Q$(SRC_PATH)\E/||g' $*.test.err | diff -q $(SRC_PATH)/$*.err -I've kept that, because it might also occur in stack backtraces. I think.
Ideally, we don't have any stack backtraces ;) But we have indeed historically checked in .err files with a backtrace that a later patch in the series cleans up, so that part of the munging does indeed play a role in a clean 'make check' for documenting an expected weakness in the generator.
Quoted 7 lines
Quoted 4 lines
I still recall having to hand-edit .err files when doing a naive "run the test to get the failure, then 'mv' the bad files into the expected filenames, then rerun the tests";
Whenever the Perl script does something, you can't simply move the .test.err to .err. Annoying.
We could also munge the original output prior to creating .test.err; but that also has drawbacks if it eats something that would have been useful for debugging the test.
Quoted 2 lines
A make target check-accept could automate the work.
Yeah, but I'm not sure how often we'd need to invoke it, or even remember that it exists. But it would at least avoid the issue of having to debug a munged .test.err.
Quoted 5 lines
However, with this patch, the Perl script should do something less often. I guess that's your point.
Sorry for being slow on the update :)
No problem; I figure you've got a lot on your mind as you are trying to tie down loose ends before some much deserved time off.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 09/21] qapi: Don't absolutize include file name in error messages
In reply to Markus ArmbrusterQuoted 8 lines · Markus Armbruster
Error messages print absolute filenames of included files even gave a relative one on the command line:
PYTHONPATH=scripts python -B
tests/qapi-schema/test-qapi.pytests/qapi-schema/include-cycle.json
In file included fromtests/qapi-schema/include-cycle.json:1:
In file included from /work/armbru/qemu/tests/qapi-schema/include-cycle-b.json:1:
/work/armbru/qemu/tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json
Quoted 7 lines
Improve this to
In file included from
tests/qapi-schema/include-cycle.json:1:
In file included fromtests/qapi-schema/include-cycle-b.json:1:
tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.jsonSigned-off-by Markus Armbruster <[email protected]>
most of the necessary refactoring/split is done in previous patch,
Quoted 61 lines
--- scripts/qapi/common.py | 12 ++++++------ tests/qapi-schema/include-no-file.err | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-)scripts/qapi/common.py +6
-6@@ -255,9 +255,8 @@ class QAPIDoc(object): class QAPISchemaParser(object): def __init__(self, fp, previously_included=[], incl_info=None): - abs_fname = os.path.abspath(fp.name) self.fname = fp.name - previously_included.append(abs_fname) + previously_included.append(os.path.abspath(fp.name)) self.incl_info = incl_info self.src = fp.read() if self.src == '' or self.src[-1] != '\n':@@ -288,7 +287,7 @@ class QAPISchemaParser(object): if not isinstance(include, str): raise QAPISemError(info, "Value of 'include' must be a string") - self._include(include, info, os.path.dirname(abs_fname), + self._include(include, info, os.path.dirname(self.fname), previously_included) elif "pragma" in expr: self.reject_expr_doc(cur_doc)@@ -321,7 +320,8 @@ class QAPISchemaParser(object): % doc.symbol) def _include(self, include, info, base_dir, previously_included): - incl_abs_fname = os.path.join(base_dir, include) + incl_fname = os.path.join(base_dir, include) + incl_abs_fname = os.path.abspath(incl_fname) # catch inclusion cycle inf = info while inf:@@ -333,9 +333,9 @@ class QAPISchemaParser(object): if incl_abs_fname in previously_included: return try: - fobj = open(incl_abs_fname, 'r') + fobj = open(incl_fname, 'r') except IOError as e: - raise QAPISemError(info, '%s: %s' % (e.strerror, include)) + raise QAPISemError(info, '%s: %s' % (e.strerror, incl_fname)) exprs_include = QAPISchemaParser(fobj, previously_included, info) self.exprs.extend(exprs_include.exprs) self.docs.extend(exprs_include.docs)tests/qapi-schema/include-no-file.err +1
-2@@ -1 +1 @@ -tests/qapi-schema/include-no-file.json:1: No such file or directory: include-no-file-sub.json +tests/qapi-schema/include-no-file.json:1: No such file or directory: tests/qapi-schema/include-no-file-sub.json --2.13.6
RFC PATCH 10/21 qapi/common: Eliminate QAPISchema.exprs
In reply to Markus Armbruster--- scripts/qapi/common.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
scripts/qapi/common.py +4 -4
@@ -1457,13 +1457,13 @@ class QAPISchema(object): def __init__(self, fname): try: parser = QAPISchemaParser(open(fname, 'r')) - self.exprs = check_exprs(parser.exprs) + exprs = check_exprs(parser.exprs) self.docs = parser.docs self._entity_dict = {} self._predefining = True self._def_predefineds() self._predefining = False - self._def_exprs() + self._def_exprs(exprs) self.check() except QAPIError as err: print >>sys.stderr, err
@@ -1648,8 +1648,8 @@ class QAPISchema(object): name, info, doc, 'arg', self._make_members(data, info)) self._def_entity(QAPISchemaEvent(name, info, doc, data, boxed)) - def _def_exprs(self): - for expr_elem in self.exprs: + def _def_exprs(self, exprs): + for expr_elem in exprs: expr = expr_elem['expr'] info = expr_elem['info'] doc = expr_elem.get('doc')
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 10/21] qapi/common: Eliminate QAPISchema.exprs
Quoted 5 lines · Markus Armbruster
Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH RFC 10/21] qapi/common: Eliminate QAPISchema.exprs
In reply to Markus ArmbrusterQuoted 1 line · Markus Armbruster
Signed-off-by Markus Armbruster <[email protected]>
Quoted 38 lines
--- scripts/qapi/common.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)scripts/qapi/common.py +4
-5@@ -1457,13 +1457,13 @@ class QAPISchema(object): def __init__(self, fname): try: parser = QAPISchemaParser(open(fname, 'r')) - self.exprs = check_exprs(parser.exprs) + exprs = check_exprs(parser.exprs) self.docs = parser.docs self._entity_dict = {} self._predefining = True self._def_predefineds() self._predefining = False - self._def_exprs() + self._def_exprs(exprs) self.check() except QAPIError as err: print >>sys.stderr, err@@ -1648,8 +1648,8 @@ class QAPISchema(object): name, info, doc, 'arg', self._make_members(data, info)) self._def_entity(QAPISchemaEvent(name, info, doc, data, boxed)) - def _def_exprs(self): - for expr_elem in self.exprs: + def _def_exprs(self, exprs): + for expr_elem in exprs: expr = expr_elem['expr'] info = expr_elem['info'] doc = expr_elem.get('doc') --2.13.6
RFC PATCH 11/21 qapi: Lift error reporting from QAPISchema.__init__() to callers
In reply to Markus Armbruster--- scripts/qapi-gen.py | 8 ++++++-- scripts/qapi/common.py | 23 +++++++++-------------- tests/qapi-schema/test-qapi.py | 8 +++++++- 3 files changed, 22 insertions(+), 17 deletions(-)
scripts/qapi-gen.py +6 -2
@@ -7,7 +7,7 @@ import getopt import re import sys -from qapi.common import QAPISchema +from qapi.common import QAPIError, QAPISchema from qapi.types import gen_types from qapi.visit import gen_visit from qapi.commands import gen_commands
@@ -77,7 +77,11 @@ def main(argv): if o in ('-u', '--unmask-non-abi-names'): opt_unmask = True - schema = QAPISchema(input_file) + try: + schema = QAPISchema(input_file) + except QAPIError as err: + print >>sys.stderr, err + exit(1) gen_types(schema, output_dir, prefix, opt_builtins) gen_visit(schema, output_dir, prefix, opt_builtins)
scripts/qapi/common.py +9 -14
@@ -15,7 +15,6 @@ import errno import os import re import string -import sys from ordereddict import OrderedDict builtin_types = {
@@ -1455,19 +1454,15 @@ class QAPISchemaEvent(QAPISchemaEntity): class QAPISchema(object): def __init__(self, fname): - try: - parser = QAPISchemaParser(open(fname, 'r')) - exprs = check_exprs(parser.exprs) - self.docs = parser.docs - self._entity_dict = {} - self._predefining = True - self._def_predefineds() - self._predefining = False - self._def_exprs(exprs) - self.check() - except QAPIError as err: - print >>sys.stderr, err - exit(1) + parser = QAPISchemaParser(open(fname, 'r')) + exprs = check_exprs(parser.exprs) + self.docs = parser.docs + self._entity_dict = {} + self._predefining = True + self._def_predefineds() + self._predefining = False + self._def_exprs(exprs) + self.check() def _def_entity(self, ent): # Only the predefined types are allowed to not have info
tests/qapi-schema/test-qapi.py +7 -1
@@ -53,7 +53,13 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor): for v in variants.variants: print ' case %s: %s' % (v.name, v.type.name) -schema = QAPISchema(sys.argv[1]) + +try: + schema = QAPISchema(sys.argv[1]) +except QAPIError as err: + print >>sys.stderr, err + exit(1) + schema.visit(QAPISchemaTestVisitor()) for doc in schema.docs:
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 11/21] qapi: Lift error reporting from QAPISchema.__init__() to callers
Quoted 1 line · Markus Armbruster
Signed-off-by Markus Armbruster <[email protected]>
Quoted 95 lines
--- scripts/qapi-gen.py | 8 ++++++-- scripts/qapi/common.py | 23 +++++++++-------------- tests/qapi-schema/test-qapi.py | 8 +++++++- 3 files changed, 22 insertions(+), 17 deletions(-)scripts/qapi-gen.py +6
-2@@ -7,7 +7,7 @@ import getopt import re import sys -from qapi.common import QAPISchema +from qapi.common import QAPIError, QAPISchema from qapi.types import gen_types from qapi.visit import gen_visit from qapi.commands import gen_commands@@ -77,7 +77,11 @@ def main(argv): if o in ('-u', '--unmask-non-abi-names'): opt_unmask = True - schema = QAPISchema(input_file) + try: + schema = QAPISchema(input_file) + except QAPIError as err: + print >>sys.stderr, err + exit(1) gen_types(schema, output_dir, prefix, opt_builtins) gen_visit(schema, output_dir, prefix, opt_builtins)scripts/qapi/common.py +9
-14@@ -15,7 +15,6 @@ import errno import os import re import string -import sys from ordereddict import OrderedDict builtin_types = {@@ -1455,19 +1454,15 @@ class QAPISchemaEvent(QAPISchemaEntity): class QAPISchema(object): def __init__(self, fname): - try: - parser = QAPISchemaParser(open(fname, 'r')) - exprs = check_exprs(parser.exprs) - self.docs = parser.docs - self._entity_dict = {} - self._predefining = True - self._def_predefineds() - self._predefining = False - self._def_exprs(exprs) - self.check() - except QAPIError as err: - print >>sys.stderr, err - exit(1) + parser = QAPISchemaParser(open(fname, 'r')) + exprs = check_exprs(parser.exprs) + self.docs = parser.docs + self._entity_dict = {} + self._predefining = True + self._def_predefineds() + self._predefining = False + self._def_exprs(exprs) + self.check() def _def_entity(self, ent): # Only the predefined types are allowed to not have infotests/qapi-schema/test-qapi.py +7
-2@@ -53,7 +53,13 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor): for v in variants.variants: print ' case %s: %s' % (v.name, v.type.name) -schema = QAPISchema(sys.argv[1]) + +try: + schema = QAPISchema(sys.argv[1]) +except QAPIError as err: + print >>sys.stderr, err + exit(1) + schema.visit(QAPISchemaTestVisitor()) for doc in schema.docs: --2.13.6
RFC PATCH 12/21 qapi: Concentrate QAPISchemaParser.exprs updates in .__init__()
In reply to Markus Armbruster--- scripts/qapi/common.py | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
scripts/qapi/common.py +9 -6
@@ -286,8 +286,12 @@ class QAPISchemaParser(object): if not isinstance(include, str): raise QAPISemError(info, "Value of 'include' must be a string") - self._include(include, info, os.path.dirname(self.fname), - previously_included) + exprs_include = self._include(include, info, + os.path.dirname(self.fname), + previously_included) + if exprs_include: + self.exprs.extend(exprs_include.exprs) + self.docs.extend(exprs_include.docs) elif "pragma" in expr: self.reject_expr_doc(cur_doc) if len(expr) != 1:
@@ -330,14 +334,13 @@ class QAPISchemaParser(object): # skip multiple include of the same file if incl_abs_fname in previously_included: - return + return None + try: fobj = open(incl_fname, 'r') except IOError as e: raise QAPISemError(info, '%s: %s' % (e.strerror, incl_fname)) - exprs_include = QAPISchemaParser(fobj, previously_included, info) - self.exprs.extend(exprs_include.exprs) - self.docs.extend(exprs_include.docs) + return QAPISchemaParser(fobj, previously_included, info) def _pragma(self, name, value, info): global doc_required, returns_whitelist, name_case_whitelist
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 12/21] qapi: Concentrate QAPISchemaParser.exprs updates in .__init__()
Quoted 1 line · Markus Armbruster
Signed-off-by Markus Armbruster <[email protected]>
It's not obvious the motivation behind this change (beside behind more elegant), but
Quoted 44 lines
--- scripts/qapi/common.py | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)scripts/qapi/common.py +9
-7@@ -286,8 +286,12 @@ class QAPISchemaParser(object): if not isinstance(include, str): raise QAPISemError(info, "Value of 'include' must be a string") - self._include(include, info, os.path.dirname(self.fname), - previously_included) + exprs_include = self._include(include, info, + os.path.dirname(self.fname), + previously_included) + if exprs_include: + self.exprs.extend(exprs_include.exprs) + self.docs.extend(exprs_include.docs) elif "pragma" in expr: self.reject_expr_doc(cur_doc) if len(expr) != 1:@@ -330,14 +334,13 @@ class QAPISchemaParser(object): # skip multiple include of the same file if incl_abs_fname in previously_included: - return + return None + try: fobj = open(incl_fname, 'r') except IOError as e: raise QAPISemError(info, '%s: %s' % (e.strerror, incl_fname)) - exprs_include = QAPISchemaParser(fobj, previously_included, info) - self.exprs.extend(exprs_include.exprs) - self.docs.extend(exprs_include.docs) + return QAPISchemaParser(fobj, previously_included, info) def _pragma(self, name, value, info): global doc_required, returns_whitelist, name_case_whitelist --2.13.6
Re: [Qemu-devel] [PATCH RFC 12/21] qapi: Concentrate QAPISchemaParser.exprs updates in .__init__()
Quoted 5 lines · Marc-Andre Lureau
Quoted 1 line · Markus Armbruster
Signed-off-by Markus Armbruster <[email protected]>It's not obvious the motivation behind this change (beside behind more elegant), but
At one point during my search for a good way to record includes in the parse tree, updating .exprs in two places got in the way, so I cleaned it up. I guess it's not in the way of the solution I eventually chose, but I consider it a worthwhile little cleanup on its own.
Quoted 1 line
Reviewed-by Marc-André Lureau <[email protected]>
Thanks!
RFC PATCH 13/21 qapi: Record 'include' directives in parse tree
In reply to Markus ArmbrusterThe parse tree is a list of expressions. Except include expressions currently get replaced by the included file's parse tree.
Instead of throwing away the include expression, keep it with the file name expanded so you don't have to track the including file's directory to make sense of it.
A future commit will put this include expression to use.
--- scripts/qapi/common.py | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-)
scripts/qapi/common.py +17 -4
@@ -286,8 +286,11 @@ class QAPISchemaParser(object): if not isinstance(include, str): raise QAPISemError(info, "Value of 'include' must be a string") - exprs_include = self._include(include, info, - os.path.dirname(self.fname), + incl_fname = os.path.join(os.path.dirname(self.fname), + include) + self.exprs.append({'expr': {'include': incl_fname}, + 'info': info}) + exprs_include = self._include(include, info, incl_fname, previously_included) if exprs_include: self.exprs.extend(exprs_include.exprs)
@@ -322,8 +325,7 @@ class QAPISchemaParser(object): "Documentation for '%s' is not followed by the definition" % doc.symbol) - def _include(self, include, info, base_dir, previously_included): - incl_fname = os.path.join(base_dir, include) + def _include(self, include, info, incl_fname, previously_included): incl_abs_fname = os.path.abspath(incl_fname) # catch inclusion cycle inf = info
@@ -889,6 +891,9 @@ def check_exprs(exprs): info = expr_elem['info'] doc = expr_elem.get('doc') + if 'include' in expr: + continue + if not doc and doc_required: raise QAPISemError(info, "Expression missing documentation comment")
@@ -927,6 +932,9 @@ def check_exprs(exprs): # Try again for hidden UnionKind enum for expr_elem in exprs: + if 'include' in expr: + continue + expr = expr_elem['expr'] if 'union' in expr and not discriminator_find_enum_define(expr): name = '%sKind' % expr['union']
@@ -939,6 +947,9 @@ def check_exprs(exprs): # Validate that exprs make sense for expr_elem in exprs: + if 'include' in expr: + continue + expr = expr_elem['expr'] info = expr_elem['info'] doc = expr_elem.get('doc')
@@ -1663,6 +1674,8 @@ class QAPISchema(object): self._def_command(expr, info, doc) elif 'event' in expr: self._def_event(expr, info, doc) + elif 'include' in expr: + pass else: assert False
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 13/21] qapi: Record 'include' directives in parse tree
Quoted 10 lines · Markus Armbruster
The parse tree is a list of expressions. Except include expressions currently get replaced by the included file's parse tree.
Instead of throwing away the include expression, keep it with the file name expanded so you don't have to track the including file's directory to make sense of it.
A future commit will put this include expression to use.
Signed-off-by Markus Armbruster <[email protected]>
Quoted 74 lines
--- scripts/qapi/common.py | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-)scripts/qapi/common.py +17
-5@@ -286,8 +286,11 @@ class QAPISchemaParser(object): if not isinstance(include, str): raise QAPISemError(info, "Value of 'include' must be a string") - exprs_include = self._include(include, info, - os.path.dirname(self.fname), + incl_fname = os.path.join(os.path.dirname(self.fname), + include) + self.exprs.append({'expr': {'include': incl_fname}, + 'info': info}) + exprs_include = self._include(include, info, incl_fname, previously_included) if exprs_include: self.exprs.extend(exprs_include.exprs)@@ -322,8 +325,7 @@ class QAPISchemaParser(object): "Documentation for '%s' is not followed by the definition" % doc.symbol) - def _include(self, include, info, base_dir, previously_included): - incl_fname = os.path.join(base_dir, include) + def _include(self, include, info, incl_fname, previously_included): incl_abs_fname = os.path.abspath(incl_fname) # catch inclusion cycle inf = info@@ -889,6 +891,9 @@ def check_exprs(exprs): info = expr_elem['info'] doc = expr_elem.get('doc') + if 'include' in expr: + continue + if not doc and doc_required: raise QAPISemError(info, "Expression missing documentation comment")@@ -927,6 +932,9 @@ def check_exprs(exprs): # Try again for hidden UnionKind enum for expr_elem in exprs: + if 'include' in expr: + continue + expr = expr_elem['expr'] if 'union' in expr and not discriminator_find_enum_define(expr): name = '%sKind' % expr['union']@@ -939,6 +947,9 @@ def check_exprs(exprs): # Validate that exprs make sense for expr_elem in exprs: + if 'include' in expr: + continue + expr = expr_elem['expr'] info = expr_elem['info'] doc = expr_elem.get('doc')@@ -1663,6 +1674,8 @@ class QAPISchema(object): self._def_command(expr, info, doc) elif 'event' in expr: self._def_event(expr, info, doc) + elif 'include' in expr: + pass else: assert False --2.13.6
RFC PATCH 14/21 qapi: Generate in source order
In reply to Markus ArmbrusterThe generators' conversion to visitors (merge commit 9e72681d16) changed the processing order of entities from source order to alphabetical order. The next commit needs source order, so change it back.
--- scripts/qapi/common.py | 4 +- tests/qapi-schema/comments.out | 2 +- tests/qapi-schema/doc-bad-section.out | 4 +- tests/qapi-schema/doc-good.out | 32 ++-- tests/qapi-schema/empty.out | 2 +- tests/qapi-schema/event-case.out | 2 +- tests/qapi-schema/ident-with-escape.out | 6 +- tests/qapi-schema/include-relpath.out | 2 +- tests/qapi-schema/include-repetition.out | 2 +- tests/qapi-schema/include-simple.out | 2 +- tests/qapi-schema/indented-expr.out | 2 +- tests/qapi-schema/qapi-schema-test.out | 320 +++++++++++++++---------------- 12 files changed, 191 insertions(+), 189 deletions(-)
scripts/qapi/common.py +3 -1
@@ -1471,6 +1471,7 @@ class QAPISchema(object): parser = QAPISchemaParser(open(fname, 'r')) exprs = check_exprs(parser.exprs) self.docs = parser.docs + self._entity_list = [] self._entity_dict = {} self._predefining = True self._def_predefineds()
@@ -1482,6 +1483,7 @@ class QAPISchema(object): # Only the predefined types are allowed to not have info assert ent.info or self._predefining assert ent.name not in self._entity_dict + self._entity_list.append(ent) self._entity_dict[ent.name] = ent def lookup_entity(self, name, typ=None):
@@ -1685,7 +1687,7 @@ class QAPISchema(object): def visit(self, visitor): visitor.visit_begin(self) - for (name, entity) in sorted(self._entity_dict.items()): + for entity in self._entity_list: if visitor.visit_needed(entity): entity.visit(visitor) visitor.visit_end()
tests/qapi-schema/comments.out +1 -1
@@ -1,4 +1,4 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE enum Status ['good', 'bad', 'ugly'] -object q_empty
tests/qapi-schema/doc-bad-section.out +2 -2
@@ -1,7 +1,7 @@ -enum Enum ['one', 'two'] +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE -object q_empty +enum Enum ['one', 'two'] doc symbol=Enum body= == Produces *invalid* texinfo
tests/qapi-schema/doc-good.out +16 -16
@@ -1,35 +1,35 @@ +object q_empty +enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] + prefix QTYPE +enum Enum ['one', 'two'] object Base member base1: Enum optional=False -enum Enum ['one', 'two'] +object Variant1 + member var1: str optional=False +object Variant2 object Object base Base tag base1 case one: Variant1 case two: Variant2 -enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] - prefix QTYPE +object q_obj_Variant1-wrapper + member data: Variant1 optional=False +object q_obj_Variant2-wrapper + member data: Variant2 optional=False +enum SugaredUnionKind ['one', 'two'] object SugaredUnion member type: SugaredUnionKind optional=False tag type case one: q_obj_Variant1-wrapper case two: q_obj_Variant2-wrapper -enum SugaredUnionKind ['one', 'two'] -object Variant1 - member var1: str optional=False -object Variant2 -command cmd q_obj_cmd-arg -> Object - gen=True success_response=True boxed=False -command cmd-boxed Object -> None - gen=True success_response=True boxed=True -object q_empty -object q_obj_Variant1-wrapper - member data: Variant1 optional=False -object q_obj_Variant2-wrapper - member data: Variant2 optional=False object q_obj_cmd-arg member arg1: int optional=False member arg2: str optional=True member arg3: bool optional=False +command cmd q_obj_cmd-arg -> Object + gen=True success_response=True boxed=False +command cmd-boxed Object -> None + gen=True success_response=True boxed=True doc freeform body= = Section
tests/qapi-schema/empty.out +1 -1
@@ -1,3 +1,3 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE -object q_empty
tests/qapi-schema/event-case.out +1 -1
@@ -1,5 +1,5 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE event oops None boxed=False -object q_empty
tests/qapi-schema/ident-with-escape.out +3 -3
@@ -1,7 +1,7 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE -command fooA q_obj_fooA-arg -> None - gen=True success_response=True boxed=False -object q_empty object q_obj_fooA-arg member bar1: str optional=False +command fooA q_obj_fooA-arg -> None + gen=True success_response=True boxed=False
tests/qapi-schema/include-relpath.out +1 -1
@@ -1,4 +1,4 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE enum Status ['good', 'bad', 'ugly'] -object q_empty
tests/qapi-schema/include-repetition.out +1 -1
@@ -1,4 +1,4 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE enum Status ['good', 'bad', 'ugly'] -object q_empty
tests/qapi-schema/include-simple.out +1 -1
@@ -1,4 +1,4 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE enum Status ['good', 'bad', 'ugly'] -object q_empty
tests/qapi-schema/indented-expr.out +1 -1
@@ -1,7 +1,7 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE command eins None -> None gen=True success_response=True boxed=False -object q_empty command zwei None -> None gen=True success_response=True boxed=False
tests/qapi-schema/qapi-schema-test.out +160 -160
380 lines not on this page · raw message
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 14/21] qapi: Generate in source order
Quoted 34 lines · Markus Armbruster
The generators' conversion to visitors (merge commit
9e72681d16) changed the processing order of entities from source order to alphabetical order. The next commit needs source order, so change it back.Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 4 +- tests/qapi-schema/comments.out | 2 +- tests/qapi-schema/doc-bad-section.out | 4 +- tests/qapi-schema/doc-good.out | 32 ++-- tests/qapi-schema/empty.out | 2 +- tests/qapi-schema/event-case.out | 2 +- tests/qapi-schema/ident-with-escape.out | 6 +- tests/qapi-schema/include-relpath.out | 2 +- tests/qapi-schema/include-repetition.out | 2 +- tests/qapi-schema/include-simple.out | 2 +- tests/qapi-schema/indented-expr.out | 2 +- tests/qapi-schema/qapi-schema-test.out | 320 +++++++++++++++---------------- 12 files changed, 191 insertions(+), 189 deletions(-)scripts/qapi/common.py +1
-0@@ -1471,6 +1471,7 @@ class QAPISchema(object): parser = QAPISchemaParser(open(fname, 'r')) exprs = check_exprs(parser.exprs) self.docs = parser.docs + self._entity_list = [] self._entity_dict = {} self._predefining = True self._def_predefineds()
scripts/qapi/common.py class QAPISchema(object): · line 1487
Quoted 5 lines
scripts/qapi/common.py +1
-0@@ -1482,6 +1483,7 @@ class QAPISchema(object): # Only the predefined types are allowed to not have info assert ent.info or self._predefining assert ent.name not in self._entity_dict + self._entity_list.append(ent) self._entity_dict[ent.name] = ent
Why not use the OrderedDict instead?
Quoted 558 lines
scripts/qapi/common.py +1
-1def lookup_entity(self, name, typ=None):@@ -1685,7 +1687,7 @@ class QAPISchema(object): def visit(self, visitor): visitor.visit_begin(self) - for (name, entity) in sorted(self._entity_dict.items()): + for entity in self._entity_list: if visitor.visit_needed(entity): entity.visit(visitor) visitor.visit_end()tests/qapi-schema/comments.out +1
-1@@ -1,4 +1,4 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE enum Status ['good', 'bad', 'ugly'] -object q_emptytests/qapi-schema/doc-bad-section.out +2
-2@@ -1,7 +1,7 @@ -enum Enum ['one', 'two'] +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE -object q_empty +enum Enum ['one', 'two'] doc symbol=Enum body= == Produces *invalid* texinfotests/qapi-schema/doc-good.out +16
-16@@ -1,35 +1,35 @@ +object q_empty +enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] + prefix QTYPE +enum Enum ['one', 'two'] object Base member base1: Enum optional=False -enum Enum ['one', 'two'] +object Variant1 + member var1: str optional=False +object Variant2 object Object base Base tag base1 case one: Variant1 case two: Variant2 -enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] - prefix QTYPE +object q_obj_Variant1-wrapper + member data: Variant1 optional=False +object q_obj_Variant2-wrapper + member data: Variant2 optional=False +enum SugaredUnionKind ['one', 'two'] object SugaredUnion member type: SugaredUnionKind optional=False tag type case one: q_obj_Variant1-wrapper case two: q_obj_Variant2-wrapper -enum SugaredUnionKind ['one', 'two'] -object Variant1 - member var1: str optional=False -object Variant2 -command cmd q_obj_cmd-arg -> Object - gen=True success_response=True boxed=False -command cmd-boxed Object -> None - gen=True success_response=True boxed=True -object q_empty -object q_obj_Variant1-wrapper - member data: Variant1 optional=False -object q_obj_Variant2-wrapper - member data: Variant2 optional=False object q_obj_cmd-arg member arg1: int optional=False member arg2: str optional=True member arg3: bool optional=False +command cmd q_obj_cmd-arg -> Object + gen=True success_response=True boxed=False +command cmd-boxed Object -> None + gen=True success_response=True boxed=True doc freeform body= = Sectiontests/qapi-schema/empty.out +1
-1@@ -1,3 +1,3 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE -object q_emptytests/qapi-schema/event-case.out +1
-1@@ -1,5 +1,5 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE event oops None boxed=False -object q_emptytests/qapi-schema/ident-with-escape.out +3
-3@@ -1,7 +1,7 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE -command fooA q_obj_fooA-arg -> None - gen=True success_response=True boxed=False -object q_empty object q_obj_fooA-arg member bar1: str optional=False +command fooA q_obj_fooA-arg -> None + gen=True success_response=True boxed=Falsetests/qapi-schema/include-relpath.out +1
-1@@ -1,4 +1,4 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE enum Status ['good', 'bad', 'ugly'] -object q_emptytests/qapi-schema/include-repetition.out +1
-1@@ -1,4 +1,4 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE enum Status ['good', 'bad', 'ugly'] -object q_emptytests/qapi-schema/include-simple.out +1
-1@@ -1,4 +1,4 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE enum Status ['good', 'bad', 'ugly'] -object q_emptytests/qapi-schema/indented-expr.out +1
-1@@ -1,7 +1,7 @@ +object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE command eins None -> None gen=True success_response=True boxed=False -object q_empty command zwei None -> None gen=True success_response=True boxed=Falsetests/qapi-schema/qapi-schema-test.out +160
-161@@ -1,87 +1,129 @@ -alternate AltEnumBool - tag type - case e: EnumOne - case b: bool -alternate AltEnumInt - tag type - case e: EnumOne - case i: int -alternate AltEnumNum - tag type - case e: EnumOne - case n: number -alternate AltNumEnum - tag type - case n: number - case e: EnumOne -alternate AltStrObj - tag type - case s: str - case o: TestStruct -event EVENT_A None - boxed=False -event EVENT_B None - boxed=False -event EVENT_C q_obj_EVENT_C-arg - boxed=False -event EVENT_D q_obj_EVENT_D-arg - boxed=False -event EVENT_E UserDefZero - boxed=True -event EVENT_F UserDefAlternate - boxed=True +object q_empty +enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] + prefix QTYPE +object TestStruct + member integer: int optional=False + member boolean: bool optional=False + member string: str optional=False +object NestedEnumsOne + member enum1: EnumOne optional=False + member enum2: EnumOne optional=True + member enum3: EnumOne optional=False + member enum4: EnumOne optional=True +enum MyEnum [] object Empty1 object Empty2 base Empty1 +command user_def_cmd0 Empty2 -> Empty2 + gen=True success_response=True boxed=False +enum QEnumTwo ['value1', 'value2'] + prefix QENUM_TWO +object UserDefOne + base UserDefZero + member string: str optional=False + member enum1: EnumOne optional=True enum EnumOne ['value1', 'value2', 'value3'] -object EventStructOne - member struct1: UserDefOne optional=False +object UserDefZero + member integer: int optional=False +object UserDefTwoDictDict + member userdef: UserDefOne optional=False member string: str optional=False - member enum2: EnumOne optional=True +object UserDefTwoDict + member string1: str optional=False + member dict2: UserDefTwoDictDict optional=False + member dict3: UserDefTwoDictDict optional=True +object UserDefTwo + member string0: str optional=False + member dict1: UserDefTwoDict optional=False object ForceArrays member unused1: UserDefOneList optional=False member unused2: UserDefTwoList optional=False member unused3: TestStructList optional=False -enum MyEnum [] -object NestedEnumsOne - member enum1: EnumOne optional=False - member enum2: EnumOne optional=True - member enum3: EnumOne optional=False - member enum4: EnumOne optional=True -enum QEnumTwo ['value1', 'value2'] - prefix QENUM_TWO -enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] - prefix QTYPE -object TestStruct - member integer: int optional=False - member boolean: bool optional=False - member string: str optional=False object UserDefA member boolean: bool optional=False member a_b: int optional=True -alternate UserDefAlternate - tag type - case udfu: UserDefFlatUnion - case e: EnumOne - case i: int - case n: null object UserDefB member intb: int optional=False member a-b: bool optional=True -object UserDefC - member string1: str optional=False - member string2: str optional=False object UserDefFlatUnion base UserDefUnionBase tag enum1 case value1: UserDefA case value2: UserDefB case value3: UserDefB +object UserDefUnionBase + base UserDefZero + member string: str optional=False + member enum1: EnumOne optional=False +object q_obj_UserDefFlatUnion2-base + member integer: int optional=True + member string: str optional=False + member enum1: QEnumTwo optional=False object UserDefFlatUnion2 base q_obj_UserDefFlatUnion2-base tag enum1 case value1: UserDefC case value2: UserDefB +object WrapAlternate + member alt: UserDefAlternate optional=False +alternate UserDefAlternate + tag type + case udfu: UserDefFlatUnion + case e: EnumOne + case i: int + case n: null +object UserDefC + member string1: str optional=False + member string2: str optional=False +alternate AltEnumBool + tag type + case e: EnumOne + case b: bool +alternate AltEnumNum + tag type + case e: EnumOne + case n: number +alternate AltNumEnum + tag type + case n: number + case e: EnumOne +alternate AltEnumInt + tag type + case e: EnumOne + case i: int +alternate AltStrObj + tag type + case s: str + case o: TestStruct +object q_obj_intList-wrapper + member data: intList optional=False +object q_obj_int8List-wrapper + member data: int8List optional=False +object q_obj_int16List-wrapper + member data: int16List optional=False +object q_obj_int32List-wrapper + member data: int32List optional=False +object q_obj_int64List-wrapper + member data: int64List optional=False +object q_obj_uint8List-wrapper + member data: uint8List optional=False +object q_obj_uint16List-wrapper + member data: uint16List optional=False +object q_obj_uint32List-wrapper + member data: uint32List optional=False +object q_obj_uint64List-wrapper + member data: uint64List optional=False +object q_obj_numberList-wrapper + member data: numberList optional=False +object q_obj_boolList-wrapper + member data: boolList optional=False +object q_obj_strList-wrapper + member data: strList optional=False +object q_obj_sizeList-wrapper + member data: sizeList optional=False +object q_obj_anyList-wrapper + member data: anyList optional=False +enum UserDefNativeListUnionKind ['integer', 's8', 's16', 's32', 's64', 'u8', 'u16', 'u32', 'u64', 'number', 'boolean', 'string', 'sizes', 'any'] object UserDefNativeListUnion member type: UserDefNativeListUnionKind optional=False tag type@@ -99,133 +141,91 @@ object UserDefNativeListUnion case string: q_obj_strList-wrapper case sizes: q_obj_sizeList-wrapper case any: q_obj_anyList-wrapper -enum UserDefNativeListUnionKind ['integer', 's8', 's16', 's32', 's64', 'u8', 'u16', 'u32', 'u64', 'number', 'boolean', 'string', 'sizes', 'any'] -object UserDefOne - base UserDefZero - member string: str optional=False - member enum1: EnumOne optional=True +command user_def_cmd None -> None + gen=True success_response=True boxed=False +object q_obj_user_def_cmd1-arg + member ud1a: UserDefOne optional=False +command user_def_cmd1 q_obj_user_def_cmd1-arg -> None + gen=True success_response=True boxed=False +object q_obj_user_def_cmd2-arg + member ud1a: UserDefOne optional=False + member ud1b: UserDefOne optional=True +command user_def_cmd2 q_obj_user_def_cmd2-arg -> UserDefTwo + gen=True success_response=True boxed=False +object q_obj_guest-get-time-arg + member a: int optional=False + member b: int optional=True +command guest-get-time q_obj_guest-get-time-arg -> int + gen=True success_response=True boxed=False +object q_obj_guest-sync-arg + member arg: any optional=False +command guest-sync q_obj_guest-sync-arg -> any + gen=True success_response=True boxed=False +command boxed-struct UserDefZero -> None + gen=True success_response=True boxed=True +command boxed-union UserDefNativeListUnion -> None + gen=True success_response=True boxed=True object UserDefOptions member i64: intList optional=True member u64: uint64List optional=True member u16: uint16List optional=True member i64x: int optional=True member u64x: uint64 optional=True -object UserDefTwo - member string0: str optional=False - member dict1: UserDefTwoDict optional=False -object UserDefTwoDict - member string1: str optional=False - member dict2: UserDefTwoDictDict optional=False - member dict3: UserDefTwoDictDict optional=True -object UserDefTwoDictDict - member userdef: UserDefOne optional=False +object EventStructOne + member struct1: UserDefOne optional=False member string: str optional=False -object UserDefUnionBase - base UserDefZero - member string: str optional=False - member enum1: EnumOne optional=False -object UserDefZero - member integer: int optional=False -object WrapAlternate - member alt: UserDefAlternate optional=False -event __ORG.QEMU_X-EVENT __org.qemu_x-Struct + member enum2: EnumOne optional=True +event EVENT_A None + boxed=False +event EVENT_B None + boxed=False +object q_obj_EVENT_C-arg + member a: int optional=True + member b: UserDefOne optional=True + member c: str optional=False +event EVENT_C q_obj_EVENT_C-arg + boxed=False +object q_obj_EVENT_D-arg + member a: EventStructOne optional=False + member b: str optional=False + member c: str optional=True + member enum3: EnumOne optional=True +event EVENT_D q_obj_EVENT_D-arg boxed=False -alternate __org.qemu_x-Alt - tag type - case __org.qemu_x-branch: str - case b: __org.qemu_x-Base +event EVENT_E UserDefZero + boxed=True +event EVENT_F UserDefAlternate + boxed=True +enum __org.qemu_x-Enum ['__org.qemu_x-value'] object __org.qemu_x-Base member __org.qemu_x-member1: __org.qemu_x-Enum optional=False -enum __org.qemu_x-Enum ['__org.qemu_x-value'] object __org.qemu_x-Struct base __org.qemu_x-Base member __org.qemu_x-member2: str optional=False member wchar-t: int optional=True -object __org.qemu_x-Struct2 - member array: __org.qemu_x-Union1List optional=False +object q_obj_str-wrapper + member data: str optional=False +enum __org.qemu_x-Union1Kind ['__org.qemu_x-branch'] object __org.qemu_x-Union1 member type: __org.qemu_x-Union1Kind optional=False tag type case __org.qemu_x-branch: q_obj_str-wrapper -enum __org.qemu_x-Union1Kind ['__org.qemu_x-branch'] +object __org.qemu_x-Struct2 + member array: __org.qemu_x-Union1List optional=False object __org.qemu_x-Union2 base __org.qemu_x-Base tag __org.qemu_x-member1 case __org.qemu_x-value: __org.qemu_x-Struct2 -command __org.qemu_x-command q_obj___org.qemu_x-command-arg -> __org.qemu_x-Union1 - gen=True success_response=True boxed=False -command boxed-struct UserDefZero -> None - gen=True success_response=True boxed=True -command boxed-union UserDefNativeListUnion -> None - gen=True success_response=True boxed=True -command guest-get-time q_obj_guest-get-time-arg -> int - gen=True success_response=True boxed=False -command guest-sync q_obj_guest-sync-arg -> any - gen=True success_response=True boxed=False -object q_empty -object q_obj_EVENT_C-arg - member a: int optional=True - member b: UserDefOne optional=True - member c: str optional=False -object q_obj_EVENT_D-arg - member a: EventStructOne optional=False - member b: str optional=False - member c: str optional=True - member enum3: EnumOne optional=True -object q_obj_UserDefFlatUnion2-base - member integer: int optional=True - member string: str optional=False - member enum1: QEnumTwo optional=False +alternate __org.qemu_x-Alt + tag type + case __org.qemu_x-branch: str + case b: __org.qemu_x-Base +event __ORG.QEMU_X-EVENT __org.qemu_x-Struct + boxed=False object q_obj___org.qemu_x-command-arg member a: __org.qemu_x-EnumList optional=False member b: __org.qemu_x-StructList optional=False member c: __org.qemu_x-Union2 optional=False member d: __org.qemu_x-Alt optional=False -object q_obj_anyList-wrapper - member data: anyList optional=False -object q_obj_boolList-wrapper - member data: boolList optional=False -object q_obj_guest-get-time-arg - member a: int optional=False - member b: int optional=True -object q_obj_guest-sync-arg - member arg: any optional=False -object q_obj_int16List-wrapper - member data: int16List optional=False -object q_obj_int32List-wrapper - member data: int32List optional=False -object q_obj_int64List-wrapper - member data: int64List optional=False -object q_obj_int8List-wrapper - member data: int8List optional=False -object q_obj_intList-wrapper - member data: intList optional=False -object q_obj_numberList-wrapper - member data: numberList optional=False -object q_obj_sizeList-wrapper - member data: sizeList optional=False -object q_obj_str-wrapper - member data: str optional=False -object q_obj_strList-wrapper - member data: strList optional=False -object q_obj_uint16List-wrapper - member data: uint16List optional=False -object q_obj_uint32List-wrapper - member data: uint32List optional=False -object q_obj_uint64List-wrapper - member data: uint64List optional=False -object q_obj_uint8List-wrapper - member data: uint8List optional=False -object q_obj_user_def_cmd1-arg - member ud1a: UserDefOne optional=False -object q_obj_user_def_cmd2-arg - member ud1a: UserDefOne optional=False - member ud1b: UserDefOne optional=True -command user_def_cmd None -> None - gen=True success_response=True boxed=False -command user_def_cmd0 Empty2 -> Empty2 - gen=True success_response=True boxed=False -command user_def_cmd1 q_obj_user_def_cmd1-arg -> None - gen=True success_response=True boxed=False -command user_def_cmd2 q_obj_user_def_cmd2-arg -> UserDefTwo +command __org.qemu_x-command q_obj___org.qemu_x-command-arg -> __org.qemu_x-Union1 gen=True success_response=True boxed=False --2.13.6
Re: [Qemu-devel] [PATCH RFC 14/21] qapi: Generate in source order
Quoted 34 lines · Marc-Andre Lureau
Quoted 33 lines · Markus Armbruster
The generators' conversion to visitors (merge commit
9e72681d16) changed the processing order of entities from source order to alphabetical order. The next commit needs source order, so change it back.Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 4 +- tests/qapi-schema/comments.out | 2 +- tests/qapi-schema/doc-bad-section.out | 4 +- tests/qapi-schema/doc-good.out | 32 ++-- tests/qapi-schema/empty.out | 2 +- tests/qapi-schema/event-case.out | 2 +- tests/qapi-schema/ident-with-escape.out | 6 +- tests/qapi-schema/include-relpath.out | 2 +- tests/qapi-schema/include-repetition.out | 2 +- tests/qapi-schema/include-simple.out | 2 +- tests/qapi-schema/indented-expr.out | 2 +- tests/qapi-schema/qapi-schema-test.out | 320 +++++++++++++++---------------- 12 files changed, 191 insertions(+), 189 deletions(-)scripts/qapi/common.py +1
-0@@ -1471,6 +1471,7 @@ class QAPISchema(object): parser = QAPISchemaParser(open(fname, 'r')) exprs = check_exprs(parser.exprs) self.docs = parser.docs + self._entity_list = [] self._entity_dict = {} self._predefining = True self._def_predefineds()
Quoted 8 lines
scripts/qapi/common.py class QAPISchema(object): · line 1487
Quoted 6 lines
scripts/qapi/common.py +1
-0@@ -1482,6 +1483,7 @@ class QAPISchema(object): # Only the predefined types are allowed to not have info assert ent.info or self._predefining assert ent.name not in self._entity_dict + self._entity_list.append(ent) self._entity_dict[ent.name] = entWhy not use the OrderedDict instead?
Fair question! However, the next patch will create anonymous entities, which get added only to ._entity_list, not _entity_dict.
[...]
Marc-Andre Lureau
Reviewed-by Marc-André LureauRe: [Qemu-devel] [PATCH RFC 14/21] qapi: Generate in source order
Quoted 45 lines · Markus Armbruster
Quoted 34 lines · Marc-Andre Lureau
Quoted 33 lines · Markus Armbruster
The generators' conversion to visitors (merge commit
9e72681d16) changed the processing order of entities from source order to alphabetical order. The next commit needs source order, so change it back.Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 4 +- tests/qapi-schema/comments.out | 2 +- tests/qapi-schema/doc-bad-section.out | 4 +- tests/qapi-schema/doc-good.out | 32 ++-- tests/qapi-schema/empty.out | 2 +- tests/qapi-schema/event-case.out | 2 +- tests/qapi-schema/ident-with-escape.out | 6 +- tests/qapi-schema/include-relpath.out | 2 +- tests/qapi-schema/include-repetition.out | 2 +- tests/qapi-schema/include-simple.out | 2 +- tests/qapi-schema/indented-expr.out | 2 +- tests/qapi-schema/qapi-schema-test.out | 320 +++++++++++++++---------------- 12 files changed, 191 insertions(+), 189 deletions(-)scripts/qapi/common.py +1
-0@@ -1471,6 +1471,7 @@ class QAPISchema(object): parser = QAPISchemaParser(open(fname, 'r')) exprs = check_exprs(parser.exprs) self.docs = parser.docs + self._entity_list = [] self._entity_dict = {} self._predefining = True self._def_predefineds()Quoted 8 lines
scripts/qapi/common.py class QAPISchema(object): · line 1487
Quoted 6 lines
scripts/qapi/common.py +1
-0@@ -1482,6 +1483,7 @@ class QAPISchema(object): # Only the predefined types are allowed to not have info assert ent.info or self._predefining assert ent.name not in self._entity_dict + self._entity_list.append(ent) self._entity_dict[ent.name] = entWhy not use the OrderedDict instead?
Quoted 2 lines
Fair question! However, the next patch will create anonymous entities, which get added only to ._entity_list, not
_entity_dict.
I see, something we could improve or clarify, but that's ok to me as is:
Quoted 2 lines
[...]
Re: [Qemu-devel] [PATCH RFC 14/21] qapi: Generate in source order
In reply to Markus ArmbrusterQuoted 45 lines · Markus Armbruster
Quoted 34 lines · Marc-Andre Lureau
Quoted 33 lines · Markus Armbruster
The generators' conversion to visitors (merge commit
9e72681d16) changed the processing order of entities from source order to alphabetical order. The next commit needs source order, so change it back.Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 4 +- tests/qapi-schema/comments.out | 2 +- tests/qapi-schema/doc-bad-section.out | 4 +- tests/qapi-schema/doc-good.out | 32 ++-- tests/qapi-schema/empty.out | 2 +- tests/qapi-schema/event-case.out | 2 +- tests/qapi-schema/ident-with-escape.out | 6 +- tests/qapi-schema/include-relpath.out | 2 +- tests/qapi-schema/include-repetition.out | 2 +- tests/qapi-schema/include-simple.out | 2 +- tests/qapi-schema/indented-expr.out | 2 +- tests/qapi-schema/qapi-schema-test.out | 320 +++++++++++++++---------------- 12 files changed, 191 insertions(+), 189 deletions(-)scripts/qapi/common.py +1
-0@@ -1471,6 +1471,7 @@ class QAPISchema(object): parser = QAPISchemaParser(open(fname, 'r')) exprs = check_exprs(parser.exprs) self.docs = parser.docs + self._entity_list = [] self._entity_dict = {} self._predefining = True self._def_predefineds()Quoted 8 lines
scripts/qapi/common.py class QAPISchema(object): · line 1487
Quoted 6 lines
scripts/qapi/common.py +1
-0@@ -1482,6 +1483,7 @@ class QAPISchema(object): # Only the predefined types are allowed to not have info assert ent.info or self._predefining assert ent.name not in self._entity_dict + self._entity_list.append(ent) self._entity_dict[ent.name] = entWhy not use the OrderedDict instead?
Quoted 4 lines
Fair question! However, the next patch will create anonymous entities, which get added only to ._entity_list, not
_entity_dict.[...]
Flaw in QAPISchema.check():
for ent in self._entity_dict.values():
ent.check(self)
Need to iterate over entity_list, or else we fail to check anonymous entities. Currently harmless, as QAPISchemaInclude.check() does nothing. I'll fix it.
RFC PATCH 15/21 qapi: Record 'include' directives in intermediate representation
In reply to Markus ArmbrusterThe include directive permits modular QAPI schemata, but the generated code is monolithic all the same. To permit generating modular code, the front end needs to pass more information on inclusions to the back ends. The commit before last added the necessary information to the parse tree. This commit adds it to the intermediate representation and its QAPISchemaVisitor. A later commit will use this to to generate modular code.
New entity QAPISchemaInclude represents inclusions. Call new visitor method visit_include() for it, so visitors can see the sub-modules a module includes.
New QAPISchemaEntity attribute @module names the entity's source file. Call new visitor method visit_module() when it changes during a visit, so visitors can keep track of the module being visited.
--- scripts/qapi/common.py | 44 ++++++++++++++++++++++++++++---- tests/qapi-schema/comments.out | 1 + tests/qapi-schema/doc-bad-section.out | 1 + tests/qapi-schema/doc-good.out | 1 + tests/qapi-schema/event-case.out | 1 + tests/qapi-schema/ident-with-escape.out | 1 + tests/qapi-schema/include-relpath.out | 5 ++++ tests/qapi-schema/include-repetition.out | 10 ++++++++ tests/qapi-schema/include-simple.out | 3 +++ tests/qapi-schema/indented-expr.out | 1 + tests/qapi-schema/qapi-schema-test.out | 1 + tests/qapi-schema/test-qapi.py | 7 +++++ 12 files changed, 71 insertions(+), 5 deletions(-)
scripts/qapi/common.py +39 -5
@@ -981,8 +981,9 @@ def check_exprs(exprs): class QAPISchemaEntity(object): def __init__(self, name, info, doc): - assert isinstance(name, str) + assert name is None or isinstance(name, str) self.name = name + self.module = None # For explicitly defined entities, info points to the (explicit) # definition. For builtins (and their arrays), info is None. # For implicitly defined entities, info points to a place that
@@ -1011,10 +1012,16 @@ class QAPISchemaVisitor(object): def visit_end(self): pass + def visit_module(self, fname): + pass + def visit_needed(self, entity): # Default to visiting everything return True + def visit_include(self, fname, info): + pass + def visit_builtin_type(self, name, info, json_type): pass
@@ -1041,6 +1048,16 @@ class QAPISchemaVisitor(object): pass +class QAPISchemaInclude(QAPISchemaEntity): + + def __init__(self, fname, info): + QAPISchemaEntity.__init__(self, None, info, None) + self.fname = fname + + def visit(self, visitor): + visitor.visit_include(self.fname, self.info) + + class QAPISchemaType(QAPISchemaEntity): # Return the C type for common use. # For the types we commonly box, this is a pointer type.
@@ -1468,6 +1485,7 @@ class QAPISchemaEvent(QAPISchemaEntity): class QAPISchema(object): def __init__(self, fname): + self._fname = fname parser = QAPISchemaParser(open(fname, 'r')) exprs = check_exprs(parser.exprs) self.docs = parser.docs
@@ -1475,16 +1493,19 @@ class QAPISchema(object): self._entity_dict = {} self._predefining = True self._def_predefineds() - self._predefining = False self._def_exprs(exprs) self.check() def _def_entity(self, ent): # Only the predefined types are allowed to not have info assert ent.info or self._predefining - assert ent.name not in self._entity_dict + assert ent.name is None or ent.name not in self._entity_dict self._entity_list.append(ent) - self._entity_dict[ent.name] = ent + if ent.name is not None: + self._entity_dict[ent.name] = ent + if ent.info: + ent.module = os.path.relpath(ent.info['file'], + os.path.dirname(self._fname)) def lookup_entity(self, name, typ=None): ent = self._entity_dict.get(name)
@@ -1495,6 +1516,15 @@ class QAPISchema(object): def lookup_type(self, name): return self.lookup_entity(name, QAPISchemaType) + def _def_include(self, expr, info, doc): + include = expr['include'] + assert doc is None + main_info = info + while main_info['parent']: + main_info = main_info['parent'] + fname = os.path.relpath(include, os.path.dirname(main_info['file'])) + self._def_entity(QAPISchemaInclude(fname, info)) + def _def_builtin_type(self, name, json_type, c_type): self._def_entity(QAPISchemaBuiltinType(name, json_type, c_type)) # TODO As long as we have QAPI_TYPES_BUILTIN to share multiple
@@ -1677,7 +1707,7 @@ class QAPISchema(object): elif 'event' in expr: self._def_event(expr, info, doc) elif 'include' in expr: - pass + self._def_include(expr, info, doc) else: assert False
@@ -1687,8 +1717,12 @@ class QAPISchema(object): def visit(self, visitor): visitor.visit_begin(self) + module = None for entity in self._entity_list: if visitor.visit_needed(entity): + if entity.module != module: + module = entity.module + visitor.visit_module(module) entity.visit(visitor) visitor.visit_end()
tests/qapi-schema/comments.out +1 -0
@@ -1,4 +1,5 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module comments.json enum Status ['good', 'bad', 'ugly']
tests/qapi-schema/doc-bad-section.out +1 -0
@@ -1,6 +1,7 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module doc-bad-section.json enum Enum ['one', 'two'] doc symbol=Enum body=
tests/qapi-schema/doc-good.out +1 -0
@@ -1,6 +1,7 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module doc-good.json enum Enum ['one', 'two'] object Base member base1: Enum optional=False
tests/qapi-schema/event-case.out +1 -0
@@ -1,5 +1,6 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module event-case.json event oops None boxed=False
tests/qapi-schema/ident-with-escape.out +1 -0
@@ -1,6 +1,7 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module ident-with-escape.json object q_obj_fooA-arg member bar1: str optional=False command fooA q_obj_fooA-arg -> None
tests/qapi-schema/include-relpath.out +5 -0
@@ -1,4 +1,9 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module include-relpath.json +include include/relpath.json +module include/relpath.json +include include-relpath-sub.json +module include-relpath-sub.json enum Status ['good', 'bad', 'ugly']
tests/qapi-schema/include-repetition.out +10 -0
@@ -1,4 +1,14 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module include-repetition.json +include comments.json +module comments.json enum Status ['good', 'bad', 'ugly'] +module include-repetition.json +include include-repetition-sub.json +module include-repetition-sub.json +include comments.json +include comments.json +module include-repetition.json +include comments.json
tests/qapi-schema/include-simple.out +3 -0
@@ -1,4 +1,7 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module include-simple.json +include include-simple-sub.json +module include-simple-sub.json enum Status ['good', 'bad', 'ugly']
tests/qapi-schema/indented-expr.out +1 -0
@@ -1,6 +1,7 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module indented-expr.json command eins None -> None gen=True success_response=True boxed=False command zwei None -> None
tests/qapi-schema/qapi-schema-test.out +1 -0
@@ -1,6 +1,7 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module qapi-schema-test.json object TestStruct member integer: int optional=False member boolean: bool optional=False
tests/qapi-schema/test-qapi.py +7 -0
@@ -17,6 +17,13 @@ import sys class QAPISchemaTestVisitor(QAPISchemaVisitor): + + def visit_module(self, name): + print 'module %s' % name + + def visit_include(self, name, info): + print 'include %s' % name + def visit_enum_type(self, name, info, values, prefix): print 'enum %s %s' % (name, values) if prefix:
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 15/21] qapi: Record 'include' directives in intermediate representation
Quoted 12 lines · Markus Armbruster
The include directive permits modular QAPI schemata, but the generated code is monolithic all the same. To permit generating modular code, the front end needs to pass more information on inclusions to the back ends. The commit before last added the necessary information to the parse tree. This commit adds it to the intermediate representation and its QAPISchemaVisitor. A later commit will use this to to generate modular code.
New entity QAPISchemaInclude represents inclusions. Call new visitor method
visit_include()for it, so visitors can see the sub-modules a module includes.
Quoted 5 lines
New QAPISchemaEntity attribute @module names the entity's source file. Call new visitor method
visit_module()when it changes during a visit, so visitors can keep track of the module being visited.Signed-off-by Markus Armbruster <[email protected]>
Quoted 280 lines
--- scripts/qapi/common.py | 44 ++++++++++++++++++++++++++++---- tests/qapi-schema/comments.out | 1 + tests/qapi-schema/doc-bad-section.out | 1 + tests/qapi-schema/doc-good.out | 1 + tests/qapi-schema/event-case.out | 1 + tests/qapi-schema/ident-with-escape.out | 1 + tests/qapi-schema/include-relpath.out | 5 ++++ tests/qapi-schema/include-repetition.out | 10 ++++++++ tests/qapi-schema/include-simple.out | 3 +++ tests/qapi-schema/indented-expr.out | 1 + tests/qapi-schema/qapi-schema-test.out | 1 + tests/qapi-schema/test-qapi.py | 7 +++++ 12 files changed, 71 insertions(+), 5 deletions(-)scripts/qapi/common.py +39
-5@@ -981,8 +981,9 @@ def check_exprs(exprs): class QAPISchemaEntity(object): def __init__(self, name, info, doc): - assert isinstance(name, str) + assert name is None or isinstance(name, str) self.name = name + self.module = None # For explicitly defined entities, info points to the (explicit) # definition. For builtins (and their arrays), info is None. # For implicitly defined entities, info points to a place that@@ -1011,10 +1012,16 @@ class QAPISchemaVisitor(object): def visit_end(self): pass + def visit_module(self, fname): + pass + def visit_needed(self, entity): # Default to visiting everything return True + def visit_include(self, fname, info): + pass + def visit_builtin_type(self, name, info, json_type): pass@@ -1041,6 +1048,16 @@ class QAPISchemaVisitor(object): pass +class QAPISchemaInclude(QAPISchemaEntity): + + def __init__(self, fname, info): + QAPISchemaEntity.__init__(self, None, info, None) + self.fname = fname + + def visit(self, visitor): + visitor.visit_include(self.fname, self.info) + + class QAPISchemaType(QAPISchemaEntity): # Return the C type for common use. # For the types we commonly box, this is a pointer type.@@ -1468,6 +1485,7 @@ class QAPISchemaEvent(QAPISchemaEntity): class QAPISchema(object): def __init__(self, fname): + self._fname = fname parser = QAPISchemaParser(open(fname, 'r')) exprs = check_exprs(parser.exprs) self.docs = parser.docs@@ -1475,16 +1493,19 @@ class QAPISchema(object): self._entity_dict = {} self._predefining = True self._def_predefineds() - self._predefining = False self._def_exprs(exprs) self.check() def _def_entity(self, ent): # Only the predefined types are allowed to not have info assert ent.info or self._predefining - assert ent.name not in self._entity_dict + assert ent.name is None or ent.name not in self._entity_dict self._entity_list.append(ent) - self._entity_dict[ent.name] = ent + if ent.name is not None: + self._entity_dict[ent.name] = ent + if ent.info: + ent.module = os.path.relpath(ent.info['file'], + os.path.dirname(self._fname)) def lookup_entity(self, name, typ=None): ent = self._entity_dict.get(name)@@ -1495,6 +1516,15 @@ class QAPISchema(object): def lookup_type(self, name): return self.lookup_entity(name, QAPISchemaType) + def _def_include(self, expr, info, doc): + include = expr['include'] + assert doc is None + main_info = info + while main_info['parent']: + main_info = main_info['parent'] + fname = os.path.relpath(include, os.path.dirname(main_info['file'])) + self._def_entity(QAPISchemaInclude(fname, info)) + def _def_builtin_type(self, name, json_type, c_type): self._def_entity(QAPISchemaBuiltinType(name, json_type, c_type)) # TODO As long as we have QAPI_TYPES_BUILTIN to share multiple@@ -1677,7 +1707,7 @@ class QAPISchema(object): elif 'event' in expr: self._def_event(expr, info, doc) elif 'include' in expr: - pass + self._def_include(expr, info, doc) else: assert False@@ -1687,8 +1717,12 @@ class QAPISchema(object): def visit(self, visitor): visitor.visit_begin(self) + module = None for entity in self._entity_list: if visitor.visit_needed(entity): + if entity.module != module: + module = entity.module + visitor.visit_module(module) entity.visit(visitor) visitor.visit_end()tests/qapi-schema/comments.out +1
-0@@ -1,4 +1,5 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module comments.json enum Status ['good', 'bad', 'ugly']tests/qapi-schema/doc-bad-section.out +1
-0@@ -1,6 +1,7 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module doc-bad-section.json enum Enum ['one', 'two'] doc symbol=Enum body=tests/qapi-schema/doc-good.out +1
-0@@ -1,6 +1,7 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module doc-good.json enum Enum ['one', 'two'] object Base member base1: Enum optional=Falsetests/qapi-schema/event-case.out +1
-0@@ -1,5 +1,6 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module event-case.json event oops None boxed=Falsetests/qapi-schema/ident-with-escape.out +1
-0@@ -1,6 +1,7 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module ident-with-escape.json object q_obj_fooA-arg member bar1: str optional=False command fooA q_obj_fooA-arg -> Nonetests/qapi-schema/include-relpath.out +5
-0@@ -1,4 +1,9 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module include-relpath.json +include include/relpath.json +module include/relpath.json +include include-relpath-sub.json +module include-relpath-sub.json enum Status ['good', 'bad', 'ugly']tests/qapi-schema/include-repetition.out +10
-0@@ -1,4 +1,14 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module include-repetition.json +include comments.json +module comments.json enum Status ['good', 'bad', 'ugly'] +module include-repetition.json +include include-repetition-sub.json +module include-repetition-sub.json +include comments.json +include comments.json +module include-repetition.json +include comments.jsontests/qapi-schema/include-simple.out +3
-0@@ -1,4 +1,7 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module include-simple.json +include include-simple-sub.json +module include-simple-sub.json enum Status ['good', 'bad', 'ugly']tests/qapi-schema/indented-expr.out +1
-0@@ -1,6 +1,7 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module indented-expr.json command eins None -> None gen=True success_response=True boxed=False command zwei None -> Nonetests/qapi-schema/qapi-schema-test.out +1
-0@@ -1,6 +1,7 @@ object q_empty enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool'] prefix QTYPE +module qapi-schema-test.json object TestStruct member integer: int optional=False member boolean: bool optional=Falsetests/qapi-schema/test-qapi.py +7
-1@@ -17,6 +17,13 @@ import sys class QAPISchemaTestVisitor(QAPISchemaVisitor): + + def visit_module(self, name): + print 'module %s' % name + + def visit_include(self, name, info): + print 'include %s' % name + def visit_enum_type(self, name, info, values, prefix): print 'enum %s %s' % (name, values) if prefix: --2.13.6
RFC PATCH 16/21 qapi/types qapi/visit: Make visitors use QAPIGen more
In reply to Markus ArmbrusterThe conversion is rather shallow so far: most of the output accumulation is not converted. Take the next step: convert output accumulation in QAPISchemaGenTypeVisitor and QAPISchemaGenVisitVisitor. Helper functions outside these classes are not converted.
--- scripts/qapi/types.py | 83 +++++++++++++++++++++++---------------------------- scripts/qapi/visit.py | 83 +++++++++++++++++++++++---------------------------- 2 files changed, 74 insertions(+), 92 deletions(-)
scripts/qapi/types.py +37 -46
@@ -168,35 +168,44 @@ void qapi_free_%(c_name)s(%(c_name)s *obj) class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): - def __init__(self, opt_builtins): + def __init__(self, opt_builtins, prefix): self._opt_builtins = opt_builtins - self.decl = None - self.defn = None - self._fwdecl = None - self._btin = None + self._prefix = prefix + blurb = ' * Schema-defined QAPI types' + self._genc = QAPIGenC(blurb, __doc__) + self._genh = QAPIGenH(blurb, __doc__) + self._genc.preamble(mcgen(''' +#include "qemu/osdep.h" +#include "qapi/dealloc-visitor.h" +#include "%(prefix)sqapi-types.h" +#include "%(prefix)sqapi-visit.h" +''', + prefix=prefix)) + self._genh.preamble(mcgen(''' +#include "qapi/util.h" +''')) + self._btin = '\n' + guardstart('QAPI_TYPES_BUILTIN') + + def write(self, output_dir): + self._genc.write(output_dir, self._prefix + 'qapi-types.c') + self._genh.write(output_dir, self._prefix + 'qapi-types.h') def visit_begin(self, schema): # gen_object() is recursive, ensure it doesn't visit the empty type objects_seen.add(schema.the_empty_object_type.name) - self.decl = '' - self.defn = '' - self._fwdecl = '' - self._btin = '\n' + guardstart('QAPI_TYPES_BUILTIN') def visit_end(self): - self.decl = self._fwdecl + self.decl - self._fwdecl = None # To avoid header dependency hell, we always generate # declarations for built-in types in our header files and # simply guard them. See also opt_builtins (command line # option -b). self._btin += guardend('QAPI_TYPES_BUILTIN') - self.decl = self._btin + self.decl + self._genh.preamble(self._btin) self._btin = None def _gen_type_cleanup(self, name): - self.decl += gen_type_cleanup_decl(name) - self.defn += gen_type_cleanup(name) + self._genh.body(gen_type_cleanup_decl(name)) + self._genc.body(gen_type_cleanup(name)) def visit_enum_type(self, name, info, values, prefix): # Special case for our lone builtin enum type
@@ -204,10 +213,10 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): if not info: self._btin += gen_enum(name, values, prefix) if self._opt_builtins: - self.defn += gen_enum_lookup(name, values, prefix) + self._genc.body(gen_enum_lookup(name, values, prefix)) else: - self._fwdecl += gen_enum(name, values, prefix) - self.defn += gen_enum_lookup(name, values, prefix) + self._genh.preamble(gen_enum(name, values, prefix)) + self._genc.body(gen_enum_lookup(name, values, prefix)) def visit_array_type(self, name, info, element_type): if isinstance(element_type, QAPISchemaBuiltinType):
@@ -215,20 +224,20 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self._btin += gen_array(name, element_type) self._btin += gen_type_cleanup_decl(name) if self._opt_builtins: - self.defn += gen_type_cleanup(name) + self._genc.body(gen_type_cleanup(name)) else: - self._fwdecl += gen_fwd_object_or_array(name) - self.decl += gen_array(name, element_type) + self._genh.preamble(gen_fwd_object_or_array(name)) + self._genh.body(gen_array(name, element_type)) self._gen_type_cleanup(name) def visit_object_type(self, name, info, base, members, variants): # Nothing to do for the special empty builtin if name == 'q_empty': return - self._fwdecl += gen_fwd_object_or_array(name) - self.decl += gen_object(name, base, members, variants) + self._genh.preamble(gen_fwd_object_or_array(name)) + self._genh.body(gen_object(name, base, members, variants)) if base and not base.is_implicit(): - self.decl += gen_upcast(name, base) + self._genh.body(gen_upcast(name, base)) # TODO Worth changing the visitor signature, so we could # directly use rather than repeat type.is_implicit()? if not name.startswith('q_'):
@@ -236,31 +245,13 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self._gen_type_cleanup(name) def visit_alternate_type(self, name, info, variants): - self._fwdecl += gen_fwd_object_or_array(name) - self.decl += gen_object(name, None, [variants.tag_member], variants) + self._genh.preamble(gen_fwd_object_or_array(name)) + self._genh.body(gen_object(name, None, + [variants.tag_member], variants)) self._gen_type_cleanup(name) def gen_types(schema, output_dir, prefix, opt_builtins): - blurb = ' * Schema-defined QAPI types' - genc = QAPIGenC(blurb, __doc__) - genh = QAPIGenH(blurb, __doc__) - - genc.body(mcgen(''' -#include "qemu/osdep.h" -#include "qapi/dealloc-visitor.h" -#include "%(prefix)sqapi-types.h" -#include "%(prefix)sqapi-visit.h" -''', - prefix=prefix)) - - genh.body(mcgen(''' -#include "qapi/util.h" -''')) - - vis = QAPISchemaGenTypeVisitor(opt_builtins) + vis = QAPISchemaGenTypeVisitor(opt_builtins, prefix) schema.visit(vis) - genc.body(vis.defn) - genh.body(vis.decl) - genc.write(output_dir, prefix + 'qapi-types.c') - genh.write(output_dir, prefix + 'qapi-types.h') + vis.write(output_dir)
scripts/qapi/visit.py +37 -46
124 lines not on this page · raw message
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 16/21] qapi/types qapi/visit: Make visitors use QAPIGen more
Quoted 7 lines · Markus Armbruster
The conversion is rather shallow so far: most of the output accumulation is not converted. Take the next step: convert output accumulation in QAPISchemaGenTypeVisitor and QAPISchemaGenVisitVisitor. Helper functions outside these classes are not converted.
Signed-off-by Markus Armbruster <[email protected]>
Quoted 280 lines
--- scripts/qapi/types.py | 83 +++++++++++++++++++++++---------------------------- scripts/qapi/visit.py | 83 +++++++++++++++++++++++---------------------------- 2 files changed, 74 insertions(+), 92 deletions(-)scripts/qapi/types.py +37
-46@@ -168,35 +168,44 @@ void qapi_free_%(c_name)s(%(c_name)s *obj) class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): - def __init__(self, opt_builtins): + def __init__(self, opt_builtins, prefix): self._opt_builtins = opt_builtins - self.decl = None - self.defn = None - self._fwdecl = None - self._btin = None + self._prefix = prefix + blurb = ' * Schema-defined QAPI types' + self._genc = QAPIGenC(blurb, __doc__) + self._genh = QAPIGenH(blurb, __doc__) + self._genc.preamble(mcgen(''' +#include "qemu/osdep.h" +#include "qapi/dealloc-visitor.h" +#include "%(prefix)sqapi-types.h" +#include "%(prefix)sqapi-visit.h" +''', + prefix=prefix)) + self._genh.preamble(mcgen(''' +#include "qapi/util.h" +''')) + self._btin = '\n' + guardstart('QAPI_TYPES_BUILTIN') + + def write(self, output_dir): + self._genc.write(output_dir, self._prefix + 'qapi-types.c') + self._genh.write(output_dir, self._prefix + 'qapi-types.h') def visit_begin(self, schema): # gen_object() is recursive, ensure it doesn't visit the empty type objects_seen.add(schema.the_empty_object_type.name) - self.decl = '' - self.defn = '' - self._fwdecl = '' - self._btin = '\n' + guardstart('QAPI_TYPES_BUILTIN') def visit_end(self): - self.decl = self._fwdecl + self.decl - self._fwdecl = None # To avoid header dependency hell, we always generate # declarations for built-in types in our header files and # simply guard them. See also opt_builtins (command line # option -b). self._btin += guardend('QAPI_TYPES_BUILTIN') - self.decl = self._btin + self.decl + self._genh.preamble(self._btin) self._btin = None def _gen_type_cleanup(self, name): - self.decl += gen_type_cleanup_decl(name) - self.defn += gen_type_cleanup(name) + self._genh.body(gen_type_cleanup_decl(name)) + self._genc.body(gen_type_cleanup(name)) def visit_enum_type(self, name, info, values, prefix): # Special case for our lone builtin enum type@@ -204,10 +213,10 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): if not info: self._btin += gen_enum(name, values, prefix) if self._opt_builtins: - self.defn += gen_enum_lookup(name, values, prefix) + self._genc.body(gen_enum_lookup(name, values, prefix)) else: - self._fwdecl += gen_enum(name, values, prefix) - self.defn += gen_enum_lookup(name, values, prefix) + self._genh.preamble(gen_enum(name, values, prefix)) + self._genc.body(gen_enum_lookup(name, values, prefix)) def visit_array_type(self, name, info, element_type): if isinstance(element_type, QAPISchemaBuiltinType):@@ -215,20 +224,20 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self._btin += gen_array(name, element_type) self._btin += gen_type_cleanup_decl(name) if self._opt_builtins: - self.defn += gen_type_cleanup(name) + self._genc.body(gen_type_cleanup(name)) else: - self._fwdecl += gen_fwd_object_or_array(name) - self.decl += gen_array(name, element_type) + self._genh.preamble(gen_fwd_object_or_array(name)) + self._genh.body(gen_array(name, element_type)) self._gen_type_cleanup(name) def visit_object_type(self, name, info, base, members, variants): # Nothing to do for the special empty builtin if name == 'q_empty': return - self._fwdecl += gen_fwd_object_or_array(name) - self.decl += gen_object(name, base, members, variants) + self._genh.preamble(gen_fwd_object_or_array(name)) + self._genh.body(gen_object(name, base, members, variants)) if base and not base.is_implicit(): - self.decl += gen_upcast(name, base) + self._genh.body(gen_upcast(name, base)) # TODO Worth changing the visitor signature, so we could # directly use rather than repeat type.is_implicit()? if not name.startswith('q_'):@@ -236,31 +245,13 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self._gen_type_cleanup(name) def visit_alternate_type(self, name, info, variants): - self._fwdecl += gen_fwd_object_or_array(name) - self.decl += gen_object(name, None, [variants.tag_member], variants) + self._genh.preamble(gen_fwd_object_or_array(name)) + self._genh.body(gen_object(name, None, + [variants.tag_member], variants)) self._gen_type_cleanup(name) def gen_types(schema, output_dir, prefix, opt_builtins): - blurb = ' * Schema-defined QAPI types' - genc = QAPIGenC(blurb, __doc__) - genh = QAPIGenH(blurb, __doc__) - - genc.body(mcgen(''' -#include "qemu/osdep.h" -#include "qapi/dealloc-visitor.h" -#include "%(prefix)sqapi-types.h" -#include "%(prefix)sqapi-visit.h" -''', - prefix=prefix)) - - genh.body(mcgen(''' -#include "qapi/util.h" -''')) - - vis = QAPISchemaGenTypeVisitor(opt_builtins) + vis = QAPISchemaGenTypeVisitor(opt_builtins, prefix) schema.visit(vis) - genc.body(vis.defn) - genh.body(vis.decl) - genc.write(output_dir, prefix + 'qapi-types.c') - genh.write(output_dir, prefix + 'qapi-types.h') + vis.write(output_dir)scripts/qapi/visit.py +37
-47@@ -264,24 +264,38 @@ out: class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): - def __init__(self, opt_builtins): + def __init__(self, opt_builtins, prefix): self._opt_builtins = opt_builtins - self.decl = None - self.defn = None - self._btin = None - - def visit_begin(self, schema): - self.decl = '' - self.defn = '' + self._prefix = prefix + blurb = ' * Schema-defined QAPI visitors' + self._genc = QAPIGenC(blurb, __doc__) + self._genh = QAPIGenH(blurb, __doc__) + self._genc.preamble(mcgen(''' +#include "qemu/osdep.h" +#include "qemu-common.h" +#include "qapi/error.h" +#include "%(prefix)sqapi-visit.h" +''', + prefix=prefix)) + self._genh.preamble(mcgen(''' +#include "qapi/visitor.h" +#include "qapi/qmp/qerror.h" +#include "%(prefix)sqapi-types.h" +''', + prefix=prefix)) self._btin = guardstart('QAPI_VISIT_BUILTIN') + def write(self, output_dir): + self._genc.write(output_dir, self._prefix + 'qapi-visit.c') + self._genh.write(output_dir, self._prefix + 'qapi-visit.h') + def visit_end(self): # To avoid header dependency hell, we always generate # declarations for built-in types in our header files and # simply guard them. See also opt_builtins (command line # option -b). self._btin += guardend('QAPI_VISIT_BUILTIN') - self.decl = self._btin + self.decl + self._genh.preamble(self._btin) self._btin = None def visit_enum_type(self, name, info, values, prefix):@@ -290,10 +304,10 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): if not info: self._btin += gen_visit_decl(name, scalar=True) if self._opt_builtins: - self.defn += gen_visit_enum(name) + self._genc.body(gen_visit_enum(name)) else: - self.decl += gen_visit_decl(name, scalar=True) - self.defn += gen_visit_enum(name) + self._genh.body(gen_visit_decl(name, scalar=True)) + self._genc.body(gen_visit_enum(name)) def visit_array_type(self, name, info, element_type): decl = gen_visit_decl(name)@@ -301,53 +315,30 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): if isinstance(element_type, QAPISchemaBuiltinType): self._btin += decl if self._opt_builtins: - self.defn += defn + self._genc.body(defn) else: - self.decl += decl - self.defn += defn + self._genh.body(decl) + self._genc.body(defn) def visit_object_type(self, name, info, base, members, variants): # Nothing to do for the special empty builtin if name == 'q_empty': return - self.decl += gen_visit_members_decl(name) - self.defn += gen_visit_object_members(name, base, members, variants) + self._genh.body(gen_visit_members_decl(name)) + self._genc.body(gen_visit_object_members(name, base, members, variants)) # TODO Worth changing the visitor signature, so we could # directly use rather than repeat type.is_implicit()? if not name.startswith('q_'): # only explicit types need an allocating visit - self.decl += gen_visit_decl(name) - self.defn += gen_visit_object(name, base, members, variants) + self._genh.body(gen_visit_decl(name)) + self._genc.body(gen_visit_object(name, base, members, variants)) def visit_alternate_type(self, name, info, variants): - self.decl += gen_visit_decl(name) - self.defn += gen_visit_alternate(name, variants) + self._genh.body(gen_visit_decl(name)) + self._genc.body(gen_visit_alternate(name, variants)) def gen_visit(schema, output_dir, prefix, opt_builtins): - blurb = ' * Schema-defined QAPI visitors' - genc = QAPIGenC(blurb, __doc__) - genh = QAPIGenH(blurb, __doc__) - - genc.body(mcgen(''' -#include "qemu/osdep.h" -#include "qemu-common.h" -#include "qapi/error.h" -#include "%(prefix)sqapi-visit.h" -''', - prefix=prefix)) - - genh.body(mcgen(''' -#include "qapi/visitor.h" -#include "qapi/qmp/qerror.h" -#include "%(prefix)sqapi-types.h" - -''', - prefix=prefix)) - - vis = QAPISchemaGenVisitVisitor(opt_builtins) + vis = QAPISchemaGenVisitVisitor(opt_builtins, prefix) schema.visit(vis) - genc.body(vis.defn) - genh.body(vis.decl) - genc.write(output_dir, prefix + 'qapi-visit.c') - genh.write(output_dir, prefix + 'qapi-visit.h') + vis.write(output_dir) --2.13.6
RFC PATCH 17/21 qapi/types qapi/visit: Generate built-in stuff into separate files
In reply to Markus ArmbrusterLinking code from multiple separate QAPI schemata into the same program is possible, but involves some weirdness around built-in types:
* We generate code for built-in types into .c only with option --builtins. The user is responsible to generate code for exactly one QAPI schema per program with --builtins.
* We generate code for them it into .h regardless of --builtins, guarded by #ifndef QAPI_VISIT_BUILTIN. Because the code for built-in types is exactly the same in all of them, including any combination of these headers works.
Replace this contraption by something more conventional: generate code for built-in types into their very own files: qapi-builtin-types.c, qapi-builtin-visit.c, qapi-builtin-types.h, qapi-builtin-visit.h, but only with --builtins. Obey --output-dir, but ignore --prefix for them.
Make qapi-types.h include qapi-builtin-types.h. With multiple schemata you now have multiple qapi-types.[ch], but only one qapi-builtin-types.[ch]. Same for qapi-visit.[ch] and qapi-builtin-visit.[ch].
Bonus: if all you need is built-in stuff, you can include a much smaller header. To be exploited shortly.
--- Makefile | 13 +++++--- Makefile.objs | 1 + scripts/qapi/common.py | 18 +++++------ scripts/qapi/types.py | 82 ++++++++++++++++++++++++++---------------------- scripts/qapi/visit.py | 84 ++++++++++++++++++++++++++++---------------------- 5 files changed, 111 insertions(+), 87 deletions(-)
Makefile +9 -4
@@ -88,10 +88,13 @@ endif include $(SRC_PATH)/rules.mak GENERATED_FILES = qemu-version.h config-host.h qemu-options.def -GENERATED_FILES += qmp-commands.h qapi-types.h qapi-visit.h qapi-event.h -GENERATED_FILES += qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c -GENERATED_FILES += qmp-introspect.h -GENERATED_FILES += qmp-introspect.c +GENERATED_FILES += qmp-commands.h qmp-marshal.c +GENERATED_FILES += qapi-builtin-types.h qapi-builtin-types.c +GENERATED_FILES += qapi-types.h qapi-types.c +GENERATED_FILES += qapi-builtin-visit.h qapi-builtin-visit.c +GENERATED_FILES += qapi-visit.h qapi-visit.c +GENERATED_FILES += qapi-event.h qapi-event.c +GENERATED_FILES += qmp-introspect.c qmp-introspect.h GENERATED_FILES += qapi.texi GENERATED_FILES += trace/generated-tcg-tracers.h
@@ -514,7 +517,9 @@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \ $(SRC_PATH)/qapi/transaction.json \ $(SRC_PATH)/qapi/ui.json +qapi-builtin-types.c qapi-builtin-types.h \ qapi-types.c qapi-types.h \ +qapi-builtin-visit.c qapi-builtin-visit.h \ qapi-visit.c qapi-visit.h \ qmp-commands.h qmp-marshal.c \ qapi-event.c qapi-event.h \
Makefile.objs +1 -0
@@ -2,6 +2,7 @@ # Common libraries for tools and emulators stub-obj-y = stubs/ crypto/ util-obj-y = util/ qobject/ qapi/ +util-obj-y += qapi-builtin-types.o qapi-builtin-visit.o util-obj-y += qmp-introspect.o qapi-types.o qapi-visit.o qapi-event.o chardev-obj-y = chardev/
scripts/qapi/common.py +9 -9
@@ -1527,11 +1527,10 @@ class QAPISchema(object): def _def_builtin_type(self, name, json_type, c_type): self._def_entity(QAPISchemaBuiltinType(name, json_type, c_type)) - # TODO As long as we have QAPI_TYPES_BUILTIN to share multiple - # qapi-types.h from a single .c, all arrays of builtins must be - # declared in the first file whether or not they are used. Nicer - # would be to use lazy instantiation, while figuring out how to - # avoid compilation issues with multiple qapi-types.h. + # Instantiating only the arrays that are actually used would + # be nice, but we can't as long as their generated code + # (qapi-builtin-types.[ch]) may be shared by some other + # schema. self._make_array_type(name, None) def _def_predefineds(self):
@@ -1985,14 +1984,15 @@ class QAPIGen(object): return '' def write(self, output_dir, fname): - if output_dir: + pathname = os.path.join(output_dir, fname) + dir = os.path.dirname(pathname) + if dir: try: - os.makedirs(output_dir) + os.makedirs(dir) except os.error as e: if e.errno != errno.EEXIST: raise - fd = os.open(os.path.join(output_dir, fname), - os.O_RDWR | os.O_CREAT, 0666) + fd = os.open(pathname, os.O_RDWR | os.O_CREAT, 0666) f = os.fdopen(fd, 'r+') text = (self.top(fname) + self._preamble + self._body + self.bottom(fname))
scripts/qapi/types.py +45 -37
109 lines not on this page · raw message
scripts/qapi/visit.py +47 -37
105 lines not on this page · raw message
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 17/21] qapi/types qapi/visit: Generate built-in stuff into separate files
Quoted 24 lines · Markus Armbruster
Linking code from multiple separate QAPI schemata into the same program is possible, but involves some weirdness around built-in types:
* We generate code for built-in types into .c only with option --builtins. The user is responsible to generate code for exactly one QAPI schema per program with --builtins.
* We generate code for them it into .h regardless of --builtins, guarded by #ifndef
QAPI_VISIT_BUILTIN. Because the code for built-in types is exactly the same in all of them, including any combination of these headers works.Replace this contraption by something more conventional: generate code for built-in types into their very own files:
qapi-builtin-types.c,qapi-builtin-visit.c,qapi-builtin-types.h,qapi-builtin-visit.h, but only with --builtins. Obey --output-dir, but ignore --prefix for them.Make
qapi-types.hincludeqapi-builtin-types.h. With multiple schemata you now have multiple qapi-types.[ch], but only one qapi-builtin-types.[ch]. Same for qapi-visit.[ch] and qapi-builtin-visit.[ch].
Quoted 4 lines
Bonus: if all you need is built-in stuff, you can include a much smaller header. To be exploited shortly.
Signed-off-by Markus Armbruster <[email protected]>
Quoted 319 lines
--- Makefile | 13 +++++--- Makefile.objs | 1 + scripts/qapi/common.py | 18 +++++------ scripts/qapi/types.py | 82 ++++++++++++++++++++++++++---------------------- scripts/qapi/visit.py | 84 ++++++++++++++++++++++++++++---------------------- 5 files changed, 111 insertions(+), 87 deletions(-)Makefile +9
-4@@ -88,10 +88,13 @@ endif include $(SRC_PATH)/rules.mak GENERATED_FILES = qemu-version.h config-host.h qemu-options.def -GENERATED_FILES += qmp-commands.h qapi-types.h qapi-visit.h qapi-event.h -GENERATED_FILES += qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c -GENERATED_FILES += qmp-introspect.h -GENERATED_FILES += qmp-introspect.c +GENERATED_FILES += qmp-commands.h qmp-marshal.c +GENERATED_FILES += qapi-builtin-types.h qapi-builtin-types.c +GENERATED_FILES += qapi-types.h qapi-types.c +GENERATED_FILES += qapi-builtin-visit.h qapi-builtin-visit.c +GENERATED_FILES += qapi-visit.h qapi-visit.c +GENERATED_FILES += qapi-event.h qapi-event.c +GENERATED_FILES += qmp-introspect.c qmp-introspect.h GENERATED_FILES += qapi.texi GENERATED_FILES += trace/generated-tcg-tracers.h@@ -514,7 +517,9 @@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \ $(SRC_PATH)/qapi/transaction.json \ $(SRC_PATH)/qapi/ui.json +qapi-builtin-types.c qapi-builtin-types.h \ qapi-types.c qapi-types.h \ +qapi-builtin-visit.c qapi-builtin-visit.h \ qapi-visit.c qapi-visit.h \ qmp-commands.h qmp-marshal.c \ qapi-event.c qapi-event.h \Makefile.objs +1
-0@@ -2,6 +2,7 @@ # Common libraries for tools and emulators stub-obj-y = stubs/ crypto/ util-obj-y = util/ qobject/ qapi/ +util-obj-y += qapi-builtin-types.o qapi-builtin-visit.o util-obj-y += qmp-introspect.o qapi-types.o qapi-visit.o qapi-event.o chardev-obj-y = chardev/scripts/qapi/common.py +9
-9@@ -1527,11 +1527,10 @@ class QAPISchema(object): def _def_builtin_type(self, name, json_type, c_type): self._def_entity(QAPISchemaBuiltinType(name, json_type, c_type)) - # TODO As long as we have QAPI_TYPES_BUILTIN to share multiple - # qapi-types.h from a single .c, all arrays of builtins must be - # declared in the first file whether or not they are used. Nicer - # would be to use lazy instantiation, while figuring out how to - # avoid compilation issues with multiple qapi-types.h. + # Instantiating only the arrays that are actually used would + # be nice, but we can't as long as their generated code + # (qapi-builtin-types.[ch]) may be shared by some other + # schema. self._make_array_type(name, None) def _def_predefineds(self):@@ -1985,14 +1984,15 @@ class QAPIGen(object): return '' def write(self, output_dir, fname): - if output_dir: + pathname = os.path.join(output_dir, fname) + dir = os.path.dirname(pathname) + if dir: try: - os.makedirs(output_dir) + os.makedirs(dir) except os.error as e: if e.errno != errno.EEXIST: raise - fd = os.open(os.path.join(output_dir, fname), - os.O_RDWR | os.O_CREAT, 0666) + fd = os.open(pathname, os.O_RDWR | os.O_CREAT, 0666) f = os.fdopen(fd, 'r+') text = (self.top(fname) + self._preamble + self._body + self.bottom(fname))scripts/qapi/types.py +45
-37@@ -171,64 +171,72 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): def __init__(self, opt_builtins, prefix): self._opt_builtins = opt_builtins self._prefix = prefix - blurb = ' * Schema-defined QAPI types' - self._genc = QAPIGenC(blurb, __doc__) - self._genh = QAPIGenH(blurb, __doc__) + self._module = {} + self._add_module(None, ' * Built-in QAPI types') self._genc.preamble(mcgen(''' #include "qemu/osdep.h" #include "qapi/dealloc-visitor.h" -#include "%(prefix)sqapi-types.h" -#include "%(prefix)sqapi-visit.h" -''', - prefix=prefix)) +#include "qapi-builtin-types.h" +#include "qapi-builtin-visit.h" +''')) self._genh.preamble(mcgen(''' #include "qapi/util.h" ''')) - self._btin = '\n' + guardstart('QAPI_TYPES_BUILTIN') + + def _module_basename(self, name): + if name is None: + return 'qapi-builtin-types' + return self._prefix + 'qapi-types' + + def _add_module(self, name, blurb): + genc = QAPIGenC(blurb, __doc__) + genh = QAPIGenH(blurb, __doc__) + self._module[name] = (genc, genh) + self._set_module(name) + + def _set_module(self, name): + self._genc, self._genh = self._module[name] def write(self, output_dir): - self._genc.write(output_dir, self._prefix + 'qapi-types.c') - self._genh.write(output_dir, self._prefix + 'qapi-types.h') + for name in self._module: + if name is None and not self._opt_builtins: + continue + basename = self._module_basename(name) + (genc, genh) = self._module[name] + genc.write(output_dir, basename + '.c') + genh.write(output_dir, basename + '.h') def visit_begin(self, schema): # gen_object() is recursive, ensure it doesn't visit the empty type objects_seen.add(schema.the_empty_object_type.name) - def visit_end(self): - # To avoid header dependency hell, we always generate - # declarations for built-in types in our header files and - # simply guard them. See also opt_builtins (command line - # option -b). - self._btin += guardend('QAPI_TYPES_BUILTIN') - self._genh.preamble(self._btin) - self._btin = None + def visit_module(self, name): + if len(self._module) != 1: + return + self._add_module(name, ' * Schema-defined QAPI types') + self._genc.preamble(mcgen(''' +#include "qemu/osdep.h" +#include "qapi/dealloc-visitor.h" +#include "%(prefix)sqapi-types.h" +#include "%(prefix)sqapi-visit.h" +''', + prefix=self._prefix)) + self._genh.preamble(mcgen(''' +#include "qapi-builtin-types.h" +''')) def _gen_type_cleanup(self, name): self._genh.body(gen_type_cleanup_decl(name)) self._genc.body(gen_type_cleanup(name)) def visit_enum_type(self, name, info, values, prefix): - # Special case for our lone builtin enum type - # TODO use something cleaner than existence of info - if not info: - self._btin += gen_enum(name, values, prefix) - if self._opt_builtins: - self._genc.body(gen_enum_lookup(name, values, prefix)) - else: - self._genh.preamble(gen_enum(name, values, prefix)) - self._genc.body(gen_enum_lookup(name, values, prefix)) + self._genh.preamble(gen_enum(name, values, prefix)) + self._genc.body(gen_enum_lookup(name, values, prefix)) def visit_array_type(self, name, info, element_type): - if isinstance(element_type, QAPISchemaBuiltinType): - self._btin += gen_fwd_object_or_array(name) - self._btin += gen_array(name, element_type) - self._btin += gen_type_cleanup_decl(name) - if self._opt_builtins: - self._genc.body(gen_type_cleanup(name)) - else: - self._genh.preamble(gen_fwd_object_or_array(name)) - self._genh.body(gen_array(name, element_type)) - self._gen_type_cleanup(name) + self._genh.preamble(gen_fwd_object_or_array(name)) + self._genh.body(gen_array(name, element_type)) + self._gen_type_cleanup(name) def visit_object_type(self, name, info, base, members, variants): # Nothing to do for the special empty builtinscripts/qapi/visit.py +47
-38@@ -267,58 +267,68 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): def __init__(self, opt_builtins, prefix): self._opt_builtins = opt_builtins self._prefix = prefix - blurb = ' * Schema-defined QAPI visitors' - self._genc = QAPIGenC(blurb, __doc__) - self._genh = QAPIGenH(blurb, __doc__) + self._module = {} + self._add_module(None, ' * Built-in QAPI visitors') self._genc.preamble(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" #include "qapi/error.h" -#include "%(prefix)sqapi-visit.h" -''', - prefix=prefix)) +#include "qapi-builtin-visit.h" +''')) self._genh.preamble(mcgen(''' #include "qapi/visitor.h" #include "qapi/qmp/qerror.h" -#include "%(prefix)sqapi-types.h" +#include "qapi-builtin-types.h" ''', - prefix=prefix)) - self._btin = guardstart('QAPI_VISIT_BUILTIN') + prefix=prefix)) + + def _module_basename(self, name): + if name is None: + return 'qapi-builtin-visit' + return self._prefix + 'qapi-visit' + + def _add_module(self, name, blurb): + genc = QAPIGenC(blurb, __doc__) + genh = QAPIGenH(blurb, __doc__) + self._module[name] = (genc, genh) + self._set_module(name) + + def _set_module(self, name): + self._genc, self._genh = self._module[name] def write(self, output_dir): - self._genc.write(output_dir, self._prefix + 'qapi-visit.c') - self._genh.write(output_dir, self._prefix + 'qapi-visit.h') + for name in self._module: + if name is None and not self._opt_builtins: + continue + basename = self._module_basename(name) + (genc, genh) = self._module[name] + genc.write(output_dir, basename + '.c') + genh.write(output_dir, basename + '.h') - def visit_end(self): - # To avoid header dependency hell, we always generate - # declarations for built-in types in our header files and - # simply guard them. See also opt_builtins (command line - # option -b). - self._btin += guardend('QAPI_VISIT_BUILTIN') - self._genh.preamble(self._btin) - self._btin = None + def visit_module(self, name): + if len(self._module) != 1: + return + self._add_module(name, ' * Schema-defined QAPI visitors') + self._genc.preamble(mcgen(''' +#include "qemu/osdep.h" +#include "qemu-common.h" +#include "qapi/error.h" +#include "%(prefix)sqapi-visit.h" +''', + prefix=self._prefix)) + self._genh.preamble(mcgen(''' +#include "qapi-builtin-visit.h" +#include "%(prefix)sqapi-types.h" +''', + prefix=self._prefix)) def visit_enum_type(self, name, info, values, prefix): - # Special case for our lone builtin enum type - # TODO use something cleaner than existence of info - if not info: - self._btin += gen_visit_decl(name, scalar=True) - if self._opt_builtins: - self._genc.body(gen_visit_enum(name)) - else: - self._genh.body(gen_visit_decl(name, scalar=True)) - self._genc.body(gen_visit_enum(name)) + self._genh.body(gen_visit_decl(name, scalar=True)) + self._genc.body(gen_visit_enum(name)) def visit_array_type(self, name, info, element_type): - decl = gen_visit_decl(name) - defn = gen_visit_list(name, element_type) - if isinstance(element_type, QAPISchemaBuiltinType): - self._btin += decl - if self._opt_builtins: - self._genc.body(defn) - else: - self._genh.body(decl) - self._genc.body(defn) + self._genh.body(gen_visit_decl(name)) + self._genc.body(gen_visit_list(name, element_type)) def visit_object_type(self, name, info, base, members, variants): # Nothing to do for the special empty builtin --2.13.6
Re: [Qemu-devel] [PATCH RFC 17/21] qapi/types qapi/visit: Generate built-in stuff into separate files
In reply to Markus ArmbrusterQuoted 9 lines · Markus Armbruster
Linking code from multiple separate QAPI schemata into the same program is possible, but involves some weirdness around built-in types:
* We generate code for built-in types into .c only with option --builtins. The user is responsible to generate code for exactly one QAPI schema per program with --builtins.
* We generate code for them it into .h regardless of --builtins,
s/them it/them/
Quoted 20 lines
guarded by #ifndef
QAPI_VISIT_BUILTIN. Because the code for built-in types is exactly the same in all of them, including any combination of these headers works.Replace this contraption by something more conventional: generate code for built-in types into their very own files:
qapi-builtin-types.c,qapi-builtin-visit.c,qapi-builtin-types.h,qapi-builtin-visit.h, but only with --builtins. Obey --output-dir, but ignore --prefix for them.Make
qapi-types.hincludeqapi-builtin-types.h. With multiple schemata you now have multiple qapi-types.[ch], but only one qapi-builtin-types.[ch]. Same for qapi-visit.[ch] and qapi-builtin-visit.[ch].Bonus: if all you need is built-in stuff, you can include a much smaller header. To be exploited shortly.
Signed-off-by Markus Armbruster <[email protected]>---
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
RFC PATCH 18/21 qapi/common: Fix guardname() for funny filenames
In reply to Markus Armbrusterguardname() fails to return a valid C identifier for arguments containing anything but [A-Za-z0-9_.-']. Fix that.
--- scripts/qapi/common.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/qapi/common.py +1 -1
@@ -1860,7 +1860,7 @@ def mcgen(code, **kwds): def guardname(filename): - return c_name(filename, protect=False).upper() + return re.sub(r'[^A-Za-z0-9_]', '_', filename).upper() def guardstart(name):
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 18/21] qapi/common: Fix guardname() for funny filenames
Quoted 4 lines · Markus Armbruster
guardname()fails to return a valid C identifier for arguments containing anything but [A-Za-z0-9_.-']. Fix that.Signed-off-by Markus Armbruster <[email protected]>
Quoted 20 lines
--- scripts/qapi/common.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)scripts/qapi/common.py +1
-2@@ -1860,7 +1860,7 @@ def mcgen(code, **kwds): def guardname(filename): - return c_name(filename, protect=False).upper() + return re.sub(r'[^A-Za-z0-9_]', '_', filename).upper() def guardstart(name): --2.13.6
Re: [Qemu-devel] [PATCH RFC 18/21] qapi/common: Fix guardname() for funny filenames
In reply to Markus ArmbrusterQuoted 13 lines · Markus Armbruster
guardname()fails to return a valid C identifier for arguments containing anything but [A-Za-z0-9_.-']. Fix that.Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)scripts/qapi/common.py +0
-0
scripts/qapi/common.py def mcgen(code, **kwds): · line 1863
Quoted 5 lines
scripts/qapi/common.py +1
-1@@ -1860,7 +1860,7 @@ def mcgen(code, **kwds): def guardname(filename): - return c_name(filename, protect=False).upper() + return re.sub(r'[^A-Za-z0-9_]', '_', filename).upper()
For some choices of filename, the old code prefixes a q_ (via c_name) which gets turned into Q_ in the final guard name. The new code does not. Then again, all of the names protected by c_name() all contain lower case, while guard names are all upper case; so we aren't protecting ourselves from defining a reserved word; the main use for c_name() is to protect ourselves where we are not changing case (for example, _BOOL is no better than Q__BOOL as a guard name for a file named _Bool).
Might be worth mentioning this design consideration in the commit message, but the change itself is reasonable.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
RFC PATCH 19/21 qapi/types: Generate separate .h, .c for each module
In reply to Markus ArmbrusterOur qapi-schema.json is composed of modules connected by include directives, but the generated code is monolithic all the same: one qapi-types.h with all the types, one qapi-visit.h with all the visitors, and so forth. These monolithic headers get included all over the place. In my "build everything" tree, adding a QAPI type recompiles about 4500 out of 4800 objects.
Nobody would write such monolithic headers by hand. It stands to reason that one shouldn't generate them, either.
Split up generated qapi-types.h to mirror the schema's modular
structure: one header per module. Name the main module's header qapi-types.h, and sub-module D/B.json's header D/qapi-types-B.h.
Mirror the schema's includes in the headers, so that qapi-types.h gets you everything exactly as before. If you need less, you can include one or more of the sub-module headers. To be exploited shortly.
Split up qapi-types.c similarly.
--- Makefile | 30 ++++++++++++++++++++++++++++++ Makefile.objs | 18 +++++++++++++++++- scripts/qapi/types.py | 18 ++++++++++++++++-- 3 files changed, 63 insertions(+), 3 deletions(-)
Makefile +30 -0
@@ -91,6 +91,21 @@ GENERATED_FILES = qemu-version.h config-host.h qemu-options.def GENERATED_FILES += qmp-commands.h qmp-marshal.c GENERATED_FILES += qapi-builtin-types.h qapi-builtin-types.c GENERATED_FILES += qapi-types.h qapi-types.c +GENERATED_FILES += qapi/qapi-types-block-core.h qapi/qapi-types-block-core.c +GENERATED_FILES += qapi/qapi-types-block.h qapi/qapi-types-block.c +GENERATED_FILES += qapi/qapi-types-char.h qapi/qapi-types-char.c +GENERATED_FILES += qapi/qapi-types-common.h qapi/qapi-types-common.c +GENERATED_FILES += qapi/qapi-types-crypto.h qapi/qapi-types-crypto.c +GENERATED_FILES += qapi/qapi-types-introspect.h qapi/qapi-types-introspect.c +GENERATED_FILES += qapi/qapi-types-migration.h qapi/qapi-types-migration.c +GENERATED_FILES += qapi/qapi-types-net.h qapi/qapi-types-net.c +GENERATED_FILES += qapi/qapi-types-rocker.h qapi/qapi-types-rocker.c +GENERATED_FILES += qapi/qapi-types-run-state.h qapi/qapi-types-run-state.c +GENERATED_FILES += qapi/qapi-types-sockets.h qapi/qapi-types-sockets.c +GENERATED_FILES += qapi/qapi-types-tpm.h qapi/qapi-types-tpm.c +GENERATED_FILES += qapi/qapi-types-trace.h qapi/qapi-types-trace.c +GENERATED_FILES += qapi/qapi-types-transaction.h qapi/qapi-types-transaction.c +GENERATED_FILES += qapi/qapi-types-ui.h qapi/qapi-types-ui.c GENERATED_FILES += qapi-builtin-visit.h qapi-builtin-visit.c GENERATED_FILES += qapi-visit.h qapi-visit.c GENERATED_FILES += qapi-event.h qapi-event.c
@@ -519,6 +534,21 @@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \ qapi-builtin-types.c qapi-builtin-types.h \ qapi-types.c qapi-types.h \ +qapi/qapi-types-block-core.c qapi/qapi-types-block-core.h \ +qapi/qapi-types-block.c qapi/qapi-types-block.h \ +qapi/qapi-types-char.c qapi/qapi-types-char.h \ +qapi/qapi-types-common.c qapi/qapi-types-common.h \ +qapi/qapi-types-crypto.c qapi/qapi-types-crypto.h \ +qapi/qapi-types-introspect.c qapi/qapi-types-introspect.h \ +qapi/qapi-types-migration.c qapi/qapi-types-migration.h \ +qapi/qapi-types-net.c qapi/qapi-types-net.h \ +qapi/qapi-types-rocker.c qapi/qapi-types-rocker.h \ +qapi/qapi-types-run-state.c qapi/qapi-types-run-state.h \ +qapi/qapi-types-sockets.c qapi/qapi-types-sockets.h \ +qapi/qapi-types-tpm.c qapi/qapi-types-tpm.h \ +qapi/qapi-types-trace.c qapi/qapi-types-trace.h \ +qapi/qapi-types-transaction.c qapi/qapi-types-transaction.h \ +qapi/qapi-types-ui.c qapi/qapi-types-ui.h \ qapi-builtin-visit.c qapi-builtin-visit.h \ qapi-visit.c qapi-visit.h \ qmp-commands.h qmp-marshal.c \
Makefile.objs +17 -1
@@ -3,7 +3,23 @@ stub-obj-y = stubs/ crypto/ util-obj-y = util/ qobject/ qapi/ util-obj-y += qapi-builtin-types.o qapi-builtin-visit.o -util-obj-y += qmp-introspect.o qapi-types.o qapi-visit.o qapi-event.o +util-obj-y += qapi-types.o +util-obj-y += qapi/qapi-types-block-core.o +util-obj-y += qapi/qapi-types-block.o +util-obj-y += qapi/qapi-types-char.o +util-obj-y += qapi/qapi-types-common.o +util-obj-y += qapi/qapi-types-crypto.o +util-obj-y += qapi/qapi-types-introspect.o +util-obj-y += qapi/qapi-types-migration.o +util-obj-y += qapi/qapi-types-net.o +util-obj-y += qapi/qapi-types-rocker.o +util-obj-y += qapi/qapi-types-run-state.o +util-obj-y += qapi/qapi-types-sockets.o +util-obj-y += qapi/qapi-types-tpm.o +util-obj-y += qapi/qapi-types-trace.o +util-obj-y += qapi/qapi-types-transaction.o +util-obj-y += qapi/qapi-types-ui.o +util-obj-y += qmp-introspect.o qapi-visit.o qapi-event.o chardev-obj-y = chardev/
scripts/qapi/types.py +16 -2
@@ -172,6 +172,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self._opt_builtins = opt_builtins self._prefix = prefix self._module = {} + self._main_module = None self._add_module(None, ' * Built-in QAPI types') self._genc.preamble(mcgen(''' #include "qemu/osdep.h"
@@ -186,7 +187,11 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): def _module_basename(self, name): if name is None: return 'qapi-builtin-types' - return self._prefix + 'qapi-types' + basename = os.path.join(os.path.dirname(name), + self._prefix + 'qapi-types') + if name == self._main_module: + return basename + return basename + '-' + os.path.splitext(os.path.basename(name))[0] def _add_module(self, name, blurb): genc = QAPIGenC(blurb, __doc__)
@@ -211,7 +216,10 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): objects_seen.add(schema.the_empty_object_type.name) def visit_module(self, name): - if len(self._module) != 1: + if self._main_module is None: + self._main_module = name + if name in self._module: + self._set_module(name) return self._add_module(name, ' * Schema-defined QAPI types') self._genc.preamble(mcgen('''
@@ -225,6 +233,12 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): #include "qapi-builtin-types.h" ''')) + def visit_include(self, name, info): + self._genh.preamble(mcgen(''' +#include "%(basename)s.h" +''', + basename=self._module_basename(name))) + def _gen_type_cleanup(self, name): self._genh.body(gen_type_cleanup_decl(name)) self._genc.body(gen_type_cleanup(name))
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 19/21] qapi/types: Generate separate .h, .c for each module
Quoted 14 lines · Markus Armbruster
Our qapi-schema.json is composed of modules connected by include directives, but the generated code is monolithic all the same: one
qapi-types.hwith all the types, oneqapi-visit.hwith all the visitors, and so forth. These monolithic headers get included all over the place. In my "build everything" tree, adding a QAPI type recompiles about 4500 out of 4800 objects.Nobody would write such monolithic headers by hand. It stands to reason that one shouldn't generate them, either.
Split up generated
qapi-types.hto mirror the schema's modular
structure: one header per module. Name the main module's headerqapi-types.h, and sub-module D/B.json's headerD/qapi-types-B.h.
Quoted 7 lines
Mirror the schema's includes in the headers, so that
qapi-types.hgets you everything exactly as before. If you need less, you can include one or more of the sub-module headers. To be exploited shortly.Split up
qapi-types.csimilarly.Signed-off-by Markus Armbruster <[email protected]>
Most of the necessary split work is done in previous patch, this enables it:
Quoted 136 lines
--- Makefile | 30 ++++++++++++++++++++++++++++++ Makefile.objs | 18 +++++++++++++++++- scripts/qapi/types.py | 18 ++++++++++++++++-- 3 files changed, 63 insertions(+), 3 deletions(-)Makefile +30
-0@@ -91,6 +91,21 @@ GENERATED_FILES = qemu-version.h config-host.h qemu-options.def GENERATED_FILES += qmp-commands.h qmp-marshal.c GENERATED_FILES += qapi-builtin-types.h qapi-builtin-types.c GENERATED_FILES += qapi-types.h qapi-types.c +GENERATED_FILES += qapi/qapi-types-block-core.h qapi/qapi-types-block-core.c +GENERATED_FILES += qapi/qapi-types-block.h qapi/qapi-types-block.c +GENERATED_FILES += qapi/qapi-types-char.h qapi/qapi-types-char.c +GENERATED_FILES += qapi/qapi-types-common.h qapi/qapi-types-common.c +GENERATED_FILES += qapi/qapi-types-crypto.h qapi/qapi-types-crypto.c +GENERATED_FILES += qapi/qapi-types-introspect.h qapi/qapi-types-introspect.c +GENERATED_FILES += qapi/qapi-types-migration.h qapi/qapi-types-migration.c +GENERATED_FILES += qapi/qapi-types-net.h qapi/qapi-types-net.c +GENERATED_FILES += qapi/qapi-types-rocker.h qapi/qapi-types-rocker.c +GENERATED_FILES += qapi/qapi-types-run-state.h qapi/qapi-types-run-state.c +GENERATED_FILES += qapi/qapi-types-sockets.h qapi/qapi-types-sockets.c +GENERATED_FILES += qapi/qapi-types-tpm.h qapi/qapi-types-tpm.c +GENERATED_FILES += qapi/qapi-types-trace.h qapi/qapi-types-trace.c +GENERATED_FILES += qapi/qapi-types-transaction.h qapi/qapi-types-transaction.c +GENERATED_FILES += qapi/qapi-types-ui.h qapi/qapi-types-ui.c GENERATED_FILES += qapi-builtin-visit.h qapi-builtin-visit.c GENERATED_FILES += qapi-visit.h qapi-visit.c GENERATED_FILES += qapi-event.h qapi-event.c@@ -519,6 +534,21 @@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \ qapi-builtin-types.c qapi-builtin-types.h \ qapi-types.c qapi-types.h \ +qapi/qapi-types-block-core.c qapi/qapi-types-block-core.h \ +qapi/qapi-types-block.c qapi/qapi-types-block.h \ +qapi/qapi-types-char.c qapi/qapi-types-char.h \ +qapi/qapi-types-common.c qapi/qapi-types-common.h \ +qapi/qapi-types-crypto.c qapi/qapi-types-crypto.h \ +qapi/qapi-types-introspect.c qapi/qapi-types-introspect.h \ +qapi/qapi-types-migration.c qapi/qapi-types-migration.h \ +qapi/qapi-types-net.c qapi/qapi-types-net.h \ +qapi/qapi-types-rocker.c qapi/qapi-types-rocker.h \ +qapi/qapi-types-run-state.c qapi/qapi-types-run-state.h \ +qapi/qapi-types-sockets.c qapi/qapi-types-sockets.h \ +qapi/qapi-types-tpm.c qapi/qapi-types-tpm.h \ +qapi/qapi-types-trace.c qapi/qapi-types-trace.h \ +qapi/qapi-types-transaction.c qapi/qapi-types-transaction.h \ +qapi/qapi-types-ui.c qapi/qapi-types-ui.h \ qapi-builtin-visit.c qapi-builtin-visit.h \ qapi-visit.c qapi-visit.h \ qmp-commands.h qmp-marshal.c \Makefile.objs +17
-1@@ -3,7 +3,23 @@ stub-obj-y = stubs/ crypto/ util-obj-y = util/ qobject/ qapi/ util-obj-y += qapi-builtin-types.o qapi-builtin-visit.o -util-obj-y += qmp-introspect.o qapi-types.o qapi-visit.o qapi-event.o +util-obj-y += qapi-types.o +util-obj-y += qapi/qapi-types-block-core.o +util-obj-y += qapi/qapi-types-block.o +util-obj-y += qapi/qapi-types-char.o +util-obj-y += qapi/qapi-types-common.o +util-obj-y += qapi/qapi-types-crypto.o +util-obj-y += qapi/qapi-types-introspect.o +util-obj-y += qapi/qapi-types-migration.o +util-obj-y += qapi/qapi-types-net.o +util-obj-y += qapi/qapi-types-rocker.o +util-obj-y += qapi/qapi-types-run-state.o +util-obj-y += qapi/qapi-types-sockets.o +util-obj-y += qapi/qapi-types-tpm.o +util-obj-y += qapi/qapi-types-trace.o +util-obj-y += qapi/qapi-types-transaction.o +util-obj-y += qapi/qapi-types-ui.o +util-obj-y += qmp-introspect.o qapi-visit.o qapi-event.o chardev-obj-y = chardev/scripts/qapi/types.py +16
-3@@ -172,6 +172,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self._opt_builtins = opt_builtins self._prefix = prefix self._module = {} + self._main_module = None self._add_module(None, ' * Built-in QAPI types') self._genc.preamble(mcgen(''' #include "qemu/osdep.h"@@ -186,7 +187,11 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): def _module_basename(self, name): if name is None: return 'qapi-builtin-types' - return self._prefix + 'qapi-types' + basename = os.path.join(os.path.dirname(name), + self._prefix + 'qapi-types') + if name == self._main_module: + return basename + return basename + '-' + os.path.splitext(os.path.basename(name))[0] def _add_module(self, name, blurb): genc = QAPIGenC(blurb, __doc__)@@ -211,7 +216,10 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): objects_seen.add(schema.the_empty_object_type.name) def visit_module(self, name): - if len(self._module) != 1: + if self._main_module is None: + self._main_module = name + if name in self._module: + self._set_module(name) return self._add_module(name, ' * Schema-defined QAPI types') self._genc.preamble(mcgen('''@@ -225,6 +233,12 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): #include "qapi-builtin-types.h" ''')) + def visit_include(self, name, info): + self._genh.preamble(mcgen(''' +#include "%(basename)s.h" +''', + basename=self._module_basename(name))) + def _gen_type_cleanup(self, name): self._genh.body(gen_type_cleanup_decl(name)) self._genc.body(gen_type_cleanup(name)) --2.13.6
RFC PATCH 20/21 Include less of qapi-types.h
In reply to Markus ArmbrusterIn my "build everything" tree, a change to the types in qapi-schema.json triggers a recompile of about 4500 out of 4800 objects.
The previous commit split up the generated qapi-types.h. Replace includes of qapi-types.h (i.e. all types) by includes of parts where possible.
To illustrate the benefits: adding a type to qapi/migration.json now recompiles some 2300 instead of 4500 objects. The next commit will improve it further.
--- crypto/cipherpriv.h | 2 +- hw/block/block.c | 1 + hw/block/hd-geometry.c | 1 + hw/net/rocker/rocker_fp.c | 2 +- include/block/block.h | 2 +- include/block/dirty-bitmap.h | 2 +- include/chardev/char.h | 1 + include/crypto/cipher.h | 2 +- include/crypto/hash.h | 2 +- include/crypto/hmac.h | 2 +- include/crypto/secret.h | 1 + include/crypto/tlscreds.h | 1 + include/hw/block/block.h | 2 +- include/hw/block/fdc.h | 2 +- include/hw/ppc/spapr_drc.h | 1 + include/hw/qdev-properties.h | 1 + include/io/dns-resolver.h | 1 + include/migration/colo.h | 2 +- include/migration/failover.h | 2 +- include/migration/global_state.h | 1 + include/monitor/monitor.h | 1 + include/net/filter.h | 1 + include/net/net.h | 2 +- include/qapi/error.h | 2 +- include/qapi/qmp/qobject.h | 2 +- include/qapi/visitor.h | 2 +- include/qemu/sockets.h | 2 +- include/qemu/throttle.h | 2 +- include/qom/cpu.h | 1 + include/qom/object.h | 2 +- include/sysemu/dump.h | 2 ++ include/sysemu/hostmem.h | 1 + include/sysemu/replay.h | 1 + include/sysemu/sysemu.h | 1 + include/sysemu/tpm.h | 1 + include/sysemu/watchdog.h | 2 +- include/ui/input.h | 2 +- migration/migration.h | 1 + migration/ram.h | 2 +- net/tap_int.h | 2 +- replication.h | 1 + ui/vnc.h | 1 + 42 files changed, 43 insertions(+), 22 deletions(-)
crypto/cipherpriv.h +1 -1
@@ -15,7 +15,7 @@#ifndef QCRYPTO_CIPHERPRIV_H #define QCRYPTO_CIPHERPRIV_H -#include "qapi-types.h" +#include "qapi/qapi-types-crypto.h" typedef struct QCryptoCipherDriver QCryptoCipherDriver ;
hw/block/block.c +1 -0
@@ -12,6 +12,7 @@#include "sysemu/block-backend.h" #include "hw/block/block.h" #include "qapi/error.h" +#include "qapi/qapi-types-block.h" #include "qemu/error-report.h" void blkconf_serial (BlockConf *conf,char **serial)
hw/block/hd-geometry.c +1 -0
@@ -32,6 +32,7 @@#include "qemu/osdep.h" #include "sysemu/block-backend.h" +#include "qapi/qapi-types-block.h" #include "qemu/bswap.h" #include "hw/block/block.h" #include "trace.h"
hw/net/rocker/rocker_fp.c +1 -1
@@ -16,7 +16,7 @@#include "qemu/osdep.h" #include "net/clients.h" - +#include "qapi/qapi-types-rocker.h" #include "rocker.h" #include "rocker_hw.h" #include "rocker_fp.h"
include/block/block.h +1 -1
@@ -2,7 +2,7 @@#define BLOCK_H #include "block/aio.h" -#include "qapi-types.h" +#include "qapi/qapi-types-block-core.h" #include "qemu/iov.h" #include "qemu/coroutine.h" #include "block/accounting.h"
include/block/dirty-bitmap.h +1 -1
@@ -2,7 +2,7 @@#define BLOCK_DIRTY_BITMAP_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-block-core.h" #include "qemu/hbitmap.h" BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
include/chardev/char.h +1 -0
@@ -1,6 +1,7 @@#ifndef QEMU_CHAR_H #define QEMU_CHAR_H +#include "qapi/qapi-types-char.h" #include "qemu/main-loop.h" #include "qemu/bitmap.h" #include "qom/object.h"
include/crypto/cipher.h +1 -1
@@ -21,7 +21,7 @@#ifndef QCRYPTO_CIPHER_H #define QCRYPTO_CIPHER_H -#include "qapi-types.h" +#include "qapi/qapi-types-crypto.h" typedef struct QCryptoCipher QCryptoCipher ;
include/crypto/hash.h +1 -1
@@ -21,7 +21,7 @@#ifndef QCRYPTO_HASH_H #define QCRYPTO_HASH_H -#include "qapi-types.h" +#include "qapi/qapi-types-crypto.h" /* See also "QCryptoHashAlgorithm" defined in qapi/crypto.json */
include/crypto/hmac.h +1 -1
@@ -12,7 +12,7 @@#ifndef QCRYPTO_HMAC_H #define QCRYPTO_HMAC_H -#include "qapi-types.h" +#include "qapi/qapi-types-crypto.h" typedef struct QCryptoHmac QCryptoHmac ; struct QCryptoHmac {
include/crypto/secret.h +1 -0
@@ -21,6 +21,7 @@#ifndef QCRYPTO_SECRET_H #define QCRYPTO_SECRET_H +#include "qapi/qapi-types-crypto.h" #include "qom/object.h" #define TYPE_QCRYPTO_SECRET "secret"
include/crypto/tlscreds.h +1 -0
@@ -21,6 +21,7 @@#ifndef QCRYPTO_TLSCREDS_H #define QCRYPTO_TLSCREDS_H +#include "qapi/qapi-types-crypto.h" #include "qom/object.h" #ifdef CONFIG_GNUTLS
include/hw/block/block.h +1 -1
@@ -12,7 +12,7 @@#define HW_BLOCK_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-block-core.h" /* Configuration */
include/hw/block/fdc.h +1 -1
@@ -2,7 +2,7 @@#define HW_FDC_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-block.h" /* fdc.c */ #define MAX_FD 2
include/hw/ppc/spapr_drc.h +1 -0
@@ -14,6 +14,7 @@#define HW_SPAPR_DRC_H #include <libfdt.h> +#include "qapi/qapi-types-run-state.h" #include "qom/object.h" #include "sysemu/sysemu.h" #include "hw/qdev.h"
include/hw/qdev-properties.h +1 -0
@@ -1,6 +1,7 @@#ifndef QEMU_QDEV_PROPERTIES_H #define QEMU_QDEV_PROPERTIES_H +#include "qapi-types.h" #include "hw/qdev-core.h" /*** qdev-properties.c ***/
include/io/dns-resolver.h +1 -0
@@ -22,6 +22,7 @@#define QIO_DNS_RESOLVER_H #include "qemu-common.h" +#include "qapi/qapi-types-sockets.h" #include "qom/object.h" #include "io/task.h"
include/migration/colo.h +1 -1
@@ -14,7 +14,7 @@#define QEMU_COLO_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-migration.h" void colo_info_init (void );
include/migration/failover.h +1 -1
@@ -14,7 +14,7 @@#define QEMU_FAILOVER_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-migration.h" void failover_init_state (void ); FailoverStatus failover_set_state(FailoverStatus old_state,
include/migration/global_state.h +1 -0
@@ -13,6 +13,7 @@#ifndef QEMU_MIGRATION_GLOBAL_STATE_H #define QEMU_MIGRATION_GLOBAL_STATE_H +#include "qapi/qapi-types-run-state.h" #include "sysemu/sysemu.h" void register_global_state (void );
include/monitor/monitor.h +1 -0
@@ -3,6 +3,7 @@#include "qemu-common.h" #include "block/block.h" +#include "qapi-types.h" #include "qemu/readline.h" extern Monitor *cur_mon;
include/net/filter.h +1 -0
@@ -9,6 +9,7 @@#ifndef QEMU_NET_FILTER_H #define QEMU_NET_FILTER_H +#include "qapi/qapi-types-net.h" #include "qom/object.h" #include "qemu-common.h" #include "net/queue.h"
include/net/net.h +1 -1
@@ -2,7 +2,7 @@#define QEMU_NET_H #include "qemu/queue.h" -#include "qapi-types.h" +#include "qapi/qapi-types-net.h" #include "net/queue.h" #include "migration/vmstate.h"
include/qapi/error.h +1 -1
@@ -115,7 +115,7 @@#ifndef ERROR_H #define ERROR_H -#include "qapi-types.h" +#include "qapi/qapi-types-common.h" /* * Overall category of an error.
include/qapi/qmp/qobject.h +1 -1
@@ -32,7 +32,7 @@ #ifndef QOBJECT_H #define QOBJECT_H -#include "qapi-types.h" +#include "qapi-builtin-types.h" struct QObject { QType type;
include/qapi/visitor.h +1 -1
@@ -15,7 +15,7 @@#ifndef QAPI_VISITOR_H #define QAPI_VISITOR_H -#include "qapi-types.h" +#include "qapi-builtin-types.h" /* * The QAPI schema defines both a set of C data types, and a QMP wire
include/qemu/sockets.h +1 -1
@@ -9,7 +9,7 @@ int inet_aton(const char *cp, struct in_addr *ia);#endif /* !_WIN32 */ -#include "qapi-types.h" +#include "qapi/qapi-types-sockets.h" /* misc helpers */ int qemu_socket (int domain,int type,int protocol);
include/qemu/throttle.h +1 -1
@@ -26,7 +26,7 @@#define THROTTLE_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-block-core.h" #include "qemu/timer.h" #define THROTTLE_VALUE_MAX 1000000000000000LL
include/qom/cpu.h +1 -0
@@ -24,6 +24,7 @@#include "disas/bfd.h" #include "exec/hwaddr.h" #include "exec/memattrs.h" +#include "qapi/qapi-types-run-state.h" #include "qemu/bitmap.h" #include "qemu/queue.h" #include "qemu/thread.h"
include/qom/object.h +1 -1
@@ -14,7 +14,7 @@#ifndef QEMU_OBJECT_H #define QEMU_OBJECT_H -#include "qapi-types.h" +#include "qapi-builtin-types.h" #include "qemu/queue.h" struct TypeImpl ;
include/sysemu/dump.h +2 -0
@@ -14,6 +14,8 @@#ifndef DUMP_H #define DUMP_H +#include "qapi-types.h" +#define MAKEDUMPFILE_SIGNATURE "makedumpfile"#define MAX_SIZE_MDF_HEADER (4096)/* max size of makedumpfile_header */ #define TYPE_FLAT_HEADER (1)/* type of flattened format */
include/sysemu/hostmem.h +1 -0
@@ -14,6 +14,7 @@#define SYSEMU_HOSTMEM_H #include "sysemu/sysemu.h" /* for MAX_NODES */ +#include "qapi-types.h" #include "qom/object.h" #include "exec/memory.h" #include "qemu/bitmap.h"
include/sysemu/replay.h +1 -0
@@ -13,6 +13,7 @@*/ #include "sysemu.h" +#include "qapi-types.h" /* replay clock kinds */ enum ReplayClockKind {
include/sysemu/sysemu.h +1 -0
@@ -2,6 +2,7 @@#define SYSEMU_H /* Misc. things related to the system emulator. */ +#include "qapi/qapi-types-run-state.h" #include "qemu/queue.h" #include "qemu/timer.h" #include "qemu/notify.h"
include/sysemu/tpm.h +1 -0
@@ -12,6 +12,7 @@#ifndef QEMU_TPM_H #define QEMU_TPM_H +#include "qapi/qapi-types-tpm.h" #include "qom/object.h" int tpm_config_parse (QemuOptsList *opts_list,const char *optarg);
include/sysemu/watchdog.h +1 -1
@@ -23,7 +23,7 @@#define QEMU_WATCHDOG_H #include "qemu/queue.h" -#include "qapi-types.h" +#include "qapi/qapi-types-run-state.h" struct WatchdogTimerModel {QLIST_ENTRY (WatchdogTimerModel) entry;
include/ui/input.h +1 -1
@@ -1,7 +1,7 @@#ifndef INPUT_H #define INPUT_H -#include "qapi-types.h" +#include "qapi/qapi-types-ui.h" #define INPUT_EVENT_MASK_KEY (1<<INPUT_EVENT_KIND_KEY)#define INPUT_EVENT_MASK_BTN (1<<INPUT_EVENT_KIND_BTN)
migration/migration.h +1 -0
@@ -15,6 +15,7 @@#define QEMU_MIGRATION_H #include "qemu-common.h" +#include "qapi/qapi-types-migration.h" #include "qemu/thread.h" #include "exec/cpu-common.h" #include "qemu/coroutine_int.h"
migration/ram.h +1 -1
@@ -30,7 +30,7 @@#define QEMU_MIGRATION_RAM_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-migration.h" #include "exec/cpu-common.h" extern MigrationStats ram_counters;
net/tap_int.h +1 -1
@@ -27,7 +27,7 @@#define NET_TAP_INT_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-net.h" int tap_open (char *ifname,int ifname_size,int *vnet_hdr,int vnet_hdr_required,int mq_required,Error **errp);
replication.h +1 -0
@@ -15,6 +15,7 @@#ifndef REPLICATION_H #define REPLICATION_H +#include "qapi/qapi-types-block-core.h" #include "qemu/queue.h" typedef struct ReplicationOps ReplicationOps ;
ui/vnc.h +1 -0
@@ -28,6 +28,7 @@#define QEMU_VNC_H #include "qemu-common.h" +#include "qapi/qapi-types-ui.h" #include "qemu/queue.h" #include "qemu/thread.h" #include "ui/console.h"
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH RFC 20/21] Include less of qapi-types.h
Quoted 8 lines · Markus Armbruster
In my "build everything" tree, a change to the types in qapi-schema.json triggers a recompile of about 4500 out of 4800 objects.
The previous commit split up the generated
qapi-types.h. Replace includes ofqapi-types.h(i.e. all types) by includes of parts where possible.
Quoted 5 lines
To illustrate the benefits: adding a type to
qapi/migration.jsonnow recompiles some 2300 instead of 4500 objects. The next commit will improve it further.Signed-off-by Markus Armbruster <[email protected]>
for the record, what was your methodology?
Quoted 575 lines
--- crypto/cipherpriv.h | 2 +- hw/block/block.c | 1 + hw/block/hd-geometry.c | 1 + hw/net/rocker/rocker_fp.c | 2 +- include/block/block.h | 2 +- include/block/dirty-bitmap.h | 2 +- include/chardev/char.h | 1 + include/crypto/cipher.h | 2 +- include/crypto/hash.h | 2 +- include/crypto/hmac.h | 2 +- include/crypto/secret.h | 1 + include/crypto/tlscreds.h | 1 + include/hw/block/block.h | 2 +- include/hw/block/fdc.h | 2 +- include/hw/ppc/spapr_drc.h | 1 + include/hw/qdev-properties.h | 1 + include/io/dns-resolver.h | 1 + include/migration/colo.h | 2 +- include/migration/failover.h | 2 +- include/migration/global_state.h | 1 + include/monitor/monitor.h | 1 + include/net/filter.h | 1 + include/net/net.h | 2 +- include/qapi/error.h | 2 +- include/qapi/qmp/qobject.h | 2 +- include/qapi/visitor.h | 2 +- include/qemu/sockets.h | 2 +- include/qemu/throttle.h | 2 +- include/qom/cpu.h | 1 + include/qom/object.h | 2 +- include/sysemu/dump.h | 2 ++ include/sysemu/hostmem.h | 1 + include/sysemu/replay.h | 1 + include/sysemu/sysemu.h | 1 + include/sysemu/tpm.h | 1 + include/sysemu/watchdog.h | 2 +- include/ui/input.h | 2 +- migration/migration.h | 1 + migration/ram.h | 2 +- net/tap_int.h | 2 +- replication.h | 1 + ui/vnc.h | 1 + 42 files changed, 43 insertions(+), 22 deletions(-)crypto/cipherpriv.h +1
-1@@ -15,7 +15,7 @@#ifndef QCRYPTO_CIPHERPRIV_H #define QCRYPTO_CIPHERPRIV_H -#include "qapi-types.h" +#include "qapi/qapi-types-crypto.h" typedef struct QCryptoCipherDriver QCryptoCipherDriver ;hw/block/block.c +1
-0@@ -12,6 +12,7 @@#include "sysemu/block-backend.h" #include "hw/block/block.h" #include "qapi/error.h" +#include "qapi/qapi-types-block.h" #include "qemu/error-report.h" void blkconf_serial (BlockConf *conf,char **serial)hw/block/hd-geometry.c +1
-0@@ -32,6 +32,7 @@#include "qemu/osdep.h" #include "sysemu/block-backend.h" +#include "qapi/qapi-types-block.h" #include "qemu/bswap.h" #include "hw/block/block.h" #include "trace.h" hw/net/rocker/rocker_fp.c +1
-1@@ -16,7 +16,7 @@#include "qemu/osdep.h" #include "net/clients.h" - +#include "qapi/qapi-types-rocker.h" #include "rocker.h" #include "rocker_hw.h" #include "rocker_fp.h" include/block/block.h +1
-1@@ -2,7 +2,7 @@#define BLOCK_H #include "block/aio.h" -#include "qapi-types.h" +#include "qapi/qapi-types-block-core.h" #include "qemu/iov.h" #include "qemu/coroutine.h" #include "block/accounting.h" include/block/dirty-bitmap.h +1
-1@@ -2,7 +2,7 @@#define BLOCK_DIRTY_BITMAP_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-block-core.h" #include "qemu/hbitmap.h" BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,include/chardev/char.h +1
-0@@ -1,6 +1,7 @@#ifndef QEMU_CHAR_H #define QEMU_CHAR_H +#include "qapi/qapi-types-char.h" #include "qemu/main-loop.h" #include "qemu/bitmap.h" #include "qom/object.h" include/crypto/cipher.h +1
-1@@ -21,7 +21,7 @@#ifndef QCRYPTO_CIPHER_H #define QCRYPTO_CIPHER_H -#include "qapi-types.h" +#include "qapi/qapi-types-crypto.h" typedef struct QCryptoCipher QCryptoCipher ;include/crypto/hash.h +1
-1@@ -21,7 +21,7 @@#ifndef QCRYPTO_HASH_H #define QCRYPTO_HASH_H -#include "qapi-types.h" +#include "qapi/qapi-types-crypto.h" /* See also "QCryptoHashAlgorithm" defined in qapi/crypto.json */ include/crypto/hmac.h +1
-1@@ -12,7 +12,7 @@#ifndef QCRYPTO_HMAC_H #define QCRYPTO_HMAC_H -#include "qapi-types.h" +#include "qapi/qapi-types-crypto.h" typedef struct QCryptoHmac QCryptoHmac ; struct QCryptoHmac {include/crypto/secret.h +1
-0@@ -21,6 +21,7 @@#ifndef QCRYPTO_SECRET_H #define QCRYPTO_SECRET_H +#include "qapi/qapi-types-crypto.h" #include "qom/object.h" #define TYPE_QCRYPTO_SECRET "secret"include/crypto/tlscreds.h +1
-0@@ -21,6 +21,7 @@#ifndef QCRYPTO_TLSCREDS_H #define QCRYPTO_TLSCREDS_H +#include "qapi/qapi-types-crypto.h" #include "qom/object.h" #ifdef CONFIG_GNUTLS include/hw/block/block.h +1
-1@@ -12,7 +12,7 @@#define HW_BLOCK_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-block-core.h" /* Configuration */ include/hw/block/fdc.h +1
-1@@ -2,7 +2,7 @@#define HW_FDC_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-block.h" /* fdc.c */ #define MAX_FD 2include/hw/ppc/spapr_drc.h +1
-0@@ -14,6 +14,7 @@#define HW_SPAPR_DRC_H #include <libfdt.h> +#include "qapi/qapi-types-run-state.h" #include "qom/object.h" #include "sysemu/sysemu.h" #include "hw/qdev.h" include/hw/qdev-properties.h +1
-0@@ -1,6 +1,7 @@#ifndef QEMU_QDEV_PROPERTIES_H #define QEMU_QDEV_PROPERTIES_H +#include "qapi-types.h" #include "hw/qdev-core.h" /*** qdev-properties.c ***/ include/io/dns-resolver.h +1
-0@@ -22,6 +22,7 @@#define QIO_DNS_RESOLVER_H #include "qemu-common.h" +#include "qapi/qapi-types-sockets.h" #include "qom/object.h" #include "io/task.h" include/migration/colo.h +1
-1@@ -14,7 +14,7 @@#define QEMU_COLO_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-migration.h" void colo_info_init (void );include/migration/failover.h +1
-1@@ -14,7 +14,7 @@#define QEMU_FAILOVER_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-migration.h" void failover_init_state (void ); FailoverStatus failover_set_state(FailoverStatus old_state,include/migration/global_state.h +1
-0@@ -13,6 +13,7 @@#ifndef QEMU_MIGRATION_GLOBAL_STATE_H #define QEMU_MIGRATION_GLOBAL_STATE_H +#include "qapi/qapi-types-run-state.h" #include "sysemu/sysemu.h" void register_global_state (void );include/monitor/monitor.h +1
-0@@ -3,6 +3,7 @@#include "qemu-common.h" #include "block/block.h" +#include "qapi-types.h" #include "qemu/readline.h" extern Monitor *cur_mon;include/net/filter.h +1
-0@@ -9,6 +9,7 @@#ifndef QEMU_NET_FILTER_H #define QEMU_NET_FILTER_H +#include "qapi/qapi-types-net.h" #include "qom/object.h" #include "qemu-common.h" #include "net/queue.h" include/net/net.h +1
-1@@ -2,7 +2,7 @@#define QEMU_NET_H #include "qemu/queue.h" -#include "qapi-types.h" +#include "qapi/qapi-types-net.h" #include "net/queue.h" #include "migration/vmstate.h" include/qapi/error.h +1
-1@@ -115,7 +115,7 @@#ifndef ERROR_H #define ERROR_H -#include "qapi-types.h" +#include "qapi/qapi-types-common.h" /* * Overall category of an error. include/qapi/qmp/qobject.h +1
-1@@ -32,7 +32,7 @@ #ifndef QOBJECT_H #define QOBJECT_H -#include "qapi-types.h" +#include "qapi-builtin-types.h" struct QObject { QType type;include/qapi/visitor.h +1
-1@@ -15,7 +15,7 @@#ifndef QAPI_VISITOR_H #define QAPI_VISITOR_H -#include "qapi-types.h" +#include "qapi-builtin-types.h" /* * The QAPI schema defines both a set of C data types, and a QMP wire include/qemu/sockets.h +1
-1@@ -9,7 +9,7 @@ int inet_aton(const char *cp, struct in_addr *ia);#endif /* !_WIN32 */ -#include "qapi-types.h" +#include "qapi/qapi-types-sockets.h" /* misc helpers */ int qemu_socket (int domain,int type,int protocol);include/qemu/throttle.h +1
-1@@ -26,7 +26,7 @@#define THROTTLE_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-block-core.h" #include "qemu/timer.h" #define THROTTLE_VALUE_MAX 1000000000000000LLinclude/qom/cpu.h +1
-0@@ -24,6 +24,7 @@#include "disas/bfd.h" #include "exec/hwaddr.h" #include "exec/memattrs.h" +#include "qapi/qapi-types-run-state.h" #include "qemu/bitmap.h" #include "qemu/queue.h" #include "qemu/thread.h" include/qom/object.h +1
-1@@ -14,7 +14,7 @@#ifndef QEMU_OBJECT_H #define QEMU_OBJECT_H -#include "qapi-types.h" +#include "qapi-builtin-types.h" #include "qemu/queue.h" struct TypeImpl ;include/sysemu/dump.h +2
-0@@ -14,6 +14,8 @@#ifndef DUMP_H #define DUMP_H +#include "qapi-types.h" +#define MAKEDUMPFILE_SIGNATURE "makedumpfile"#define MAX_SIZE_MDF_HEADER (4096)/* max size of makedumpfile_header */ #define TYPE_FLAT_HEADER (1)/* type of flattened format */ include/sysemu/hostmem.h +1
-0@@ -14,6 +14,7 @@#define SYSEMU_HOSTMEM_H #include "sysemu/sysemu.h" /* for MAX_NODES */ +#include "qapi-types.h" #include "qom/object.h" #include "exec/memory.h" #include "qemu/bitmap.h" include/sysemu/replay.h +1
-0@@ -13,6 +13,7 @@*/ #include "sysemu.h" +#include "qapi-types.h" /* replay clock kinds */ enum ReplayClockKind {include/sysemu/sysemu.h +1
-0@@ -2,6 +2,7 @@#define SYSEMU_H /* Misc. things related to the system emulator. */ +#include "qapi/qapi-types-run-state.h" #include "qemu/queue.h" #include "qemu/timer.h" #include "qemu/notify.h" include/sysemu/tpm.h +1
-0@@ -12,6 +12,7 @@#ifndef QEMU_TPM_H #define QEMU_TPM_H +#include "qapi/qapi-types-tpm.h" #include "qom/object.h" int tpm_config_parse (QemuOptsList *opts_list,const char *optarg);include/sysemu/watchdog.h +1
-1@@ -23,7 +23,7 @@#define QEMU_WATCHDOG_H #include "qemu/queue.h" -#include "qapi-types.h" +#include "qapi/qapi-types-run-state.h" struct WatchdogTimerModel {QLIST_ENTRY (WatchdogTimerModel) entry;include/ui/input.h +1
-1@@ -1,7 +1,7 @@#ifndef INPUT_H #define INPUT_H -#include "qapi-types.h" +#include "qapi/qapi-types-ui.h" #define INPUT_EVENT_MASK_KEY (1<<INPUT_EVENT_KIND_KEY)#define INPUT_EVENT_MASK_BTN (1<<INPUT_EVENT_KIND_BTN)migration/migration.h +1
-0@@ -15,6 +15,7 @@#define QEMU_MIGRATION_H #include "qemu-common.h" +#include "qapi/qapi-types-migration.h" #include "qemu/thread.h" #include "exec/cpu-common.h" #include "qemu/coroutine_int.h" migration/ram.h +1
-1@@ -30,7 +30,7 @@#define QEMU_MIGRATION_RAM_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-migration.h" #include "exec/cpu-common.h" extern MigrationStats ram_counters;net/tap_int.h +1
-1@@ -27,7 +27,7 @@#define NET_TAP_INT_H #include "qemu-common.h" -#include "qapi-types.h" +#include "qapi/qapi-types-net.h" int tap_open (char *ifname,int ifname_size,int *vnet_hdr,int vnet_hdr_required,int mq_required,Error **errp);replication.h +1
-0@@ -15,6 +15,7 @@#ifndef REPLICATION_H #define REPLICATION_H +#include "qapi/qapi-types-block-core.h" #include "qemu/queue.h" typedef struct ReplicationOps ReplicationOps ;ui/vnc.h +1
-1@@ -28,6 +28,7 @@#define QEMU_VNC_H #include "qemu-common.h" +#include "qapi/qapi-types-ui.h" #include "qemu/queue.h" #include "qemu/thread.h" #include "ui/console.h" --2.13.6
Re: [Qemu-devel] [PATCH RFC 20/21] Include less of qapi-types.h
Quoted 9 lines · Marc-Andre Lureau
Quoted 8 lines · Markus Armbruster
In my "build everything" tree, a change to the types in qapi-schema.json triggers a recompile of about 4500 out of 4800 objects.
The previous commit split up the generated
qapi-types.h. Replace includes ofqapi-types.h(i.e. all types) by includes of parts where possible.
Quoted 7 lines
Quoted 5 lines
To illustrate the benefits: adding a type to
qapi/migration.jsonnow recompiles some 2300 instead of 4500 objects. The next commit will improve it further.Signed-off-by Markus Armbruster <[email protected]>for the record, what was your methodology?
I counted compiles like this:
$ make 2>&1 | awk '/^ [A-Z]/ { h[$1]++ } END { n = asorti(h, x); for (i = 1; i <=n; i++) print x[i], h[x[i]] }'Quoted 1 line
Reviewed-by Marc-André Lureau <[email protected]>
Thanks!
RFC PATCH 21/21 qapi: Empty out qapi-schema.json
In reply to Markus ArmbrusterTypes defined directly in qapi-schema.json end up in qapi-types.h, and including that pulls in everything else.
Move everything but include directives from qapi-schema.json to new sub-module qapi/misc.json, and replace the remaining includes of qapi-types.h.
Adding a type to qapi/migration.json now recompiles less than 200 instead of 2300 out of 4500 objects.
--- Makefile | 2 + Makefile.objs | 1 + include/hw/qdev-properties.h | 3 +- include/monitor/monitor.h | 2 +- include/sysemu/arch_init.h | 2 +- include/sysemu/balloon.h | 2 +- include/sysemu/dump.h | 2 +- include/sysemu/hostmem.h | 2 +- include/sysemu/replay.h | 3 +- qapi-schema.json | 3098 +----------------------------------------- qapi/misc.json | 3090 +++++++++++++++++++++++++++++++++++++++++ qapi/run-state.json | 10 + 12 files changed, 3113 insertions(+), 3104 deletions(-) create mode 100644 qapi/misc.json
Makefile +2 -0
@@ -98,6 +98,7 @@ GENERATED_FILES += qapi/qapi-types-common.h qapi/qapi-types-common.c GENERATED_FILES += qapi/qapi-types-crypto.h qapi/qapi-types-crypto.c GENERATED_FILES += qapi/qapi-types-introspect.h qapi/qapi-types-introspect.c GENERATED_FILES += qapi/qapi-types-migration.h qapi/qapi-types-migration.c +GENERATED_FILES += qapi/qapi-types-misc.h qapi/qapi-types-misc.c GENERATED_FILES += qapi/qapi-types-net.h qapi/qapi-types-net.c GENERATED_FILES += qapi/qapi-types-rocker.h qapi/qapi-types-rocker.c GENERATED_FILES += qapi/qapi-types-run-state.h qapi/qapi-types-run-state.c
@@ -541,6 +542,7 @@ qapi/qapi-types-common.c qapi/qapi-types-common.h \ qapi/qapi-types-crypto.c qapi/qapi-types-crypto.h \ qapi/qapi-types-introspect.c qapi/qapi-types-introspect.h \ qapi/qapi-types-migration.c qapi/qapi-types-migration.h \ +qapi/qapi-types-misc.c qapi/qapi-types-misc.h \ qapi/qapi-types-net.c qapi/qapi-types-net.h \ qapi/qapi-types-rocker.c qapi/qapi-types-rocker.h \ qapi/qapi-types-run-state.c qapi/qapi-types-run-state.h \
Makefile.objs +1 -0
@@ -11,6 +11,7 @@ util-obj-y += qapi/qapi-types-common.o util-obj-y += qapi/qapi-types-crypto.o util-obj-y += qapi/qapi-types-introspect.o util-obj-y += qapi/qapi-types-migration.o +util-obj-y += qapi/qapi-types-misc.o util-obj-y += qapi/qapi-types-net.o util-obj-y += qapi/qapi-types-rocker.o util-obj-y += qapi/qapi-types-run-state.o
include/hw/qdev-properties.h +2 -1
@@ -1,7 +1,8 @@#ifndef QEMU_QDEV_PROPERTIES_H #define QEMU_QDEV_PROPERTIES_H -#include "qapi-types.h" +#include "qapi/qapi-types-block.h" +#include "qapi/qapi-types-misc.h" #include "hw/qdev-core.h" /*** qdev-properties.c ***/
include/monitor/monitor.h +1 -1
@@ -3,7 +3,7 @@#include "qemu-common.h" #include "block/block.h" -#include "qapi-types.h" +#include "qapi/qapi-types-misc.h" #include "qemu/readline.h" extern Monitor *cur_mon;
include/sysemu/arch_init.h +1 -1
@@ -1,7 +1,7 @@#ifndef QEMU_ARCH_INIT_H #define QEMU_ARCH_INIT_H -#include "qapi-types.h" +#include "qapi/qapi-types-misc.h" enum { QEMU_ARCH_ALL = -1,
include/sysemu/balloon.h +1 -1
@@ -14,7 +14,7 @@#ifndef QEMU_BALLOON_H #define QEMU_BALLOON_H -#include "qapi-types.h" +#include "qapi/qapi-types-misc.h" typedef void (QEMUBalloonEvent )(void *opaque,ram_addr_t target);typedef void (QEMUBalloonStatus )(void *opaque,BalloonInfo *info);
include/sysemu/dump.h +1 -1
@@ -14,7 +14,7 @@#ifndef DUMP_H #define DUMP_H -#include "qapi-types.h" +#include "qapi/qapi-types-misc.h" #define MAKEDUMPFILE_SIGNATURE "makedumpfile"#define MAX_SIZE_MDF_HEADER (4096)/* max size of makedumpfile_header */
include/sysemu/hostmem.h +1 -1
@@ -14,7 +14,7 @@#define SYSEMU_HOSTMEM_H #include "sysemu/sysemu.h" /* for MAX_NODES */ -#include "qapi-types.h" +#include "qapi/qapi-types-misc.h" #include "qom/object.h" #include "exec/memory.h" #include "qemu/bitmap.h"
include/sysemu/replay.h +2 -1
@@ -13,7 +13,8 @@*/ #include "sysemu.h" -#include "qapi-types.h" +#include "qapi/qapi-types-misc.h" +#include "qapi/qapi-types-ui.h" /* replay clock kinds */ enum ReplayClockKind {
qapi-schema.json +1 -3097
2,921 lines not on this page · raw message
qapi/misc.json new file +3090 -0
3,090 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.13.6
Part of the diff is not shown here. View the raw message.