qapi: Add detection for the 'savevm' fix for blockdev
cover qapi: Add detection for the 'savevm' fix for blockdev
Add 'features' field in the schema for commands and add a feature flag to advertise that the fix for savevm [1] is present.
[1] https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03487.html
Peter Krempa (2):
qapi: Add feature flags to commands in qapi introspection
qapi: Allow introspecting fix for savevm's cooperation with blockdev
docs/devel/qapi-code-gen.txt | 4 ++-- qapi/introspect.json | 6 ++++- qapi/misc.json | 8 ++++++- scripts/qapi/commands.py | 3 ++- scripts/qapi/common.py | 40 +++++++++++++++++++++++++++++----- scripts/qapi/doc.py | 3 ++- scripts/qapi/introspect.py | 7 +++++- tests/qapi-schema/test-qapi.py | 7 +++++- 8 files changed, 65 insertions(+), 13 deletions(-)
Signature · 1 line
2.21.0
PATCH 1/2 qapi: Add feature flags to commands in qapi introspection
Similarly to features for struct types introduce the feature flags also for commands. This will allow notifying management layers of fixes and compatible changes in the behaviour of an command which may not be detectable any other way.
The changes were heavily inspired by commit 6a8c0b51025.
--- docs/devel/qapi-code-gen.txt | 4 ++-- qapi/introspect.json | 6 ++++- scripts/qapi/commands.py | 3 ++- scripts/qapi/common.py | 40 +++++++++++++++++++++++++++++----- scripts/qapi/doc.py | 3 ++- scripts/qapi/introspect.py | 7 +++++- tests/qapi-schema/test-qapi.py | 7 +++++- 7 files changed, 58 insertions(+), 12 deletions(-)
docs/devel/qapi-code-gen.txt +2 -2
@@ -726,8 +726,8 @@ change in the QMP syntax (usually by allowing values or operations that previously resulted in an error). QMP clients may still need to know whether the extension is available. -For this purpose, a list of features can be specified for a struct type. -This is exposed to the client as a list of string, where each string +For this purpose, a list of features can be specified for a command or struct +type. This is exposed to the client as a list of string, where each string signals that this build of QEMU shows a certain behaviour. In the schema, features can be specified as simple strings, for example:
qapi/introspect.json +5 -1
@@ -266,13 +266,17 @@ # @allow-oob: whether the command allows out-of-band execution, # defaults to false (Since: 2.12) # +# @features: names of features associated with the command, in no particular +# order. (since 4.2) +# # TODO: @success-response (currently irrelevant, because it's QGA, not QMP) # # Since: 2.5 ## { 'struct': 'SchemaInfoCommand', 'data': { 'arg-type': 'str', 'ret-type': 'str', - '*allow-oob': 'bool' } } + '*allow-oob': 'bool', + '*features': [ 'str' ] } } ## # @SchemaInfoEvent:
scripts/qapi/commands.py +2 -1
@@ -276,7 +276,8 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds); genc.add(gen_registry(self._regy.get_content(), self._prefix)) def visit_command(self, name, info, ifcond, arg_type, ret_type, gen, - success_response, boxed, allow_oob, allow_preconfig): + success_response, boxed, allow_oob, allow_preconfig, + features): if not gen: return # FIXME: If T is a user-defined type, the user is responsible
scripts/qapi/common.py +35 -5
@@ -838,6 +838,7 @@ def check_type(info, source, value, allow_array=False, def check_command(expr, info): name = expr['command'] boxed = expr.get('boxed', False) + features = expr.get('features') args_meta = ['struct'] if boxed:
@@ -852,6 +853,19 @@ def check_command(expr, info): expr.get('returns'), allow_array=True, allow_optional=True, allow_metas=returns_meta) + if features: + if not isinstance(features, list): + raise QAPISemError(info, + "Command '%s' requires an array for 'features'" % + name) + for f in features: + assert isinstance(f, dict) + check_known_keys(info, "feature of command %s" % name, f, + ['name'], ['if']) + + check_if(f, info) + check_name(info, "Feature of command %s" % name, f['name']) + def check_event(expr, info): name = expr['event']
@@ -1138,8 +1152,10 @@ def check_exprs(exprs): meta = 'command' check_keys(expr_elem, 'command', [], ['data', 'returns', 'gen', 'success-response', - 'boxed', 'allow-oob', 'allow-preconfig', 'if']) + 'boxed', 'allow-oob', 'allow-preconfig', 'if', + 'features']) normalize_members(expr.get('data')) + normalize_features(expr.get('features')) elif 'event' in expr: meta = 'event' check_keys(expr_elem, 'event', [], ['data', 'boxed', 'if'])
@@ -1283,7 +1299,8 @@ class QAPISchemaVisitor(object): pass def visit_command(self, name, info, ifcond, arg_type, ret_type, gen, - success_response, boxed, allow_oob, allow_preconfig): + success_response, boxed, allow_oob, allow_preconfig, + features): pass def visit_event(self, name, info, ifcond, arg_type, boxed):
@@ -1697,10 +1714,14 @@ class QAPISchemaAlternateType(QAPISchemaType): class QAPISchemaCommand(QAPISchemaEntity): def __init__(self, name, info, doc, ifcond, arg_type, ret_type, - gen, success_response, boxed, allow_oob, allow_preconfig): + gen, success_response, boxed, allow_oob, allow_preconfig, + features): QAPISchemaEntity.__init__(self, name, info, doc, ifcond) assert not arg_type or isinstance(arg_type, str) assert not ret_type or isinstance(ret_type, str) + for f in features: + assert isinstance(f, QAPISchemaFeature) + f.set_owner(name) self._arg_type_name = arg_type self.arg_type = None self._ret_type_name = ret_type
@@ -1710,6 +1731,7 @@ class QAPISchemaCommand(QAPISchemaEntity): self.boxed = boxed self.allow_oob = allow_oob self.allow_preconfig = allow_preconfig + self.features = features def check(self, schema): QAPISchemaEntity.check(self, schema)
@@ -1731,12 +1753,18 @@ class QAPISchemaCommand(QAPISchemaEntity): self.ret_type = schema.lookup_type(self._ret_type_name) assert isinstance(self.ret_type, QAPISchemaType) + # Features are in a name space separate from members + seen = {} + for f in self.features: + f.check_clash(self.info, seen) + def visit(self, visitor): visitor.visit_command(self.name, self.info, self.ifcond, self.arg_type, self.ret_type, self.gen, self.success_response, self.boxed, self.allow_oob, - self.allow_preconfig) + self.allow_preconfig, + self.features) class QAPISchemaEvent(QAPISchemaEntity):
@@ -1989,6 +2017,7 @@ class QAPISchema(object): allow_oob = expr.get('allow-oob', False) allow_preconfig = expr.get('allow-preconfig', False) ifcond = expr.get('if') + features = expr.get('features', []) if isinstance(data, OrderedDict): data = self._make_implicit_object_type( name, info, doc, ifcond, 'arg', self._make_members(data, info))
@@ -1997,7 +2026,8 @@ class QAPISchema(object): rets = self._make_array_type(rets[0], info) self._def_entity(QAPISchemaCommand(name, info, doc, ifcond, data, rets, gen, success_response, - boxed, allow_oob, allow_preconfig)) + boxed, allow_oob, allow_preconfig, + self._make_features(features))) def _def_event(self, expr, info, doc): name = expr['event']
scripts/qapi/doc.py +2 -1
@@ -249,7 +249,8 @@ class QAPISchemaGenDocVisitor(qapi.common.QAPISchemaVisitor): body=texi_entity(doc, 'Members', ifcond))) def visit_command(self, name, info, ifcond, arg_type, ret_type, gen, - success_response, boxed, allow_oob, allow_preconfig): + success_response, boxed, allow_oob, allow_preconfig, + features): doc = self.cur_doc if boxed: body = texi_body(doc)
scripts/qapi/introspect.py +6 -1
@@ -206,13 +206,18 @@ const QLitObject %(c_name)s = %(c_string)s; for m in variants.variants]}, ifcond) def visit_command(self, name, info, ifcond, arg_type, ret_type, gen, - success_response, boxed, allow_oob, allow_preconfig): + success_response, boxed, allow_oob, allow_preconfig, + features): arg_type = arg_type or self._schema.the_empty_object_type ret_type = ret_type or self._schema.the_empty_object_type obj = {'arg-type': self._use_type(arg_type), 'ret-type': self._use_type(ret_type)} if allow_oob: obj['allow-oob'] = allow_oob + + if features: + obj['features'] = [(f.name, {'if': f.ifcond}) for f in features] + self._gen_qlit(name, 'command', obj, ifcond) def visit_event(self, name, info, ifcond, arg_type, boxed):
tests/qapi-schema/test-qapi.py +6 -1
@@ -60,13 +60,18 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor): self._print_if(ifcond) def visit_command(self, name, info, ifcond, arg_type, ret_type, gen, - success_response, boxed, allow_oob, allow_preconfig): + success_response, boxed, allow_oob, allow_preconfig, + features): print('command %s %s -> %s' % (name, arg_type and arg_type.name, ret_type and ret_type.name)) print(' gen=%s success_response=%s boxed=%s oob=%s preconfig=%s' % (gen, success_response, boxed, allow_oob, allow_preconfig)) self._print_if(ifcond) + if features: + for f in features: + print(' feature %s' % f.name) + self._print_if(f.ifcond, 8) def visit_event(self, name, info, ifcond, arg_type, boxed): print('event %s %s' % (name, arg_type and arg_type.name))
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 1/2] qapi: Add feature flags to commands in qapi introspection
Quoted 3 lines · Peter Krempa
Similarly to features for struct types introduce the feature flags also for commands. This will allow notifying management layers of fixes and compatible changes in the behaviour of an command which may not be
s/ an / a /
Quoted 7 lines
detectable any other way.
The changes were heavily inspired by commit
6a8c0b51025.Signed-off-by Peter Krempa <[email protected]>--- docs/devel/qapi-code-gen.txt | 4 ++--
May be some rebase churn needed here as Markus has been reworking that file:
https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg02959.html
Quoted 13 lines
qapi/introspect.json | 6 ++++- scripts/qapi/commands.py | 3 ++- scripts/qapi/common.py | 40 +++++++++++++++++++++++++++++----- scripts/qapi/doc.py | 3 ++- scripts/qapi/introspect.py | 7 +++++- tests/qapi-schema/test-qapi.py | 7 +++++- 7 files changed, 58 insertions(+), 12 deletions(-)docs/devel/qapi-code-gen.txt +0
-0
docs/devel/qapi-code-gen.txt change in the QMP syntax (usually by allowing values or operations that · line 730
Quoted 7 lines
docs/devel/qapi-code-gen.txt +2
-2@@ -726,8 +726,8 @@ change in the QMP syntax (usually by allowing values or operations that previously resulted in an error). QMP clients may still need to know whether the extension is available. -For this purpose, a list of features can be specified for a struct type. -This is exposed to the client as a list of string, where each string +For this purpose, a list of features can be specified for a command or struct +type. This is exposed to the client as a list of string, where each string
Pre-existing, but "list of strings" or "list of string entries"
Quoted 2 lines
docs/devel/qapi-code-gen.txt +0
-0signals that this build of QEMU shows a certain behaviour.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 2/2 qapi: Allow introspecting fix for savevm's cooperation with blockdev
In reply to Peter Krempasavevm was buggy as it considered all monitor owned block device nodes for snapshot. With introduction of -blockdev the common usage made all nodes including protocol nodes monitor owned and thus considered for snapshot. This was fixed but clients need to be able to detect whether this fix is present.
Since savevm does not have an QMP alternative add the feature for the 'human-monitor-command' backdoor which is used to call this command in modern use.
--- qapi/misc.json | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
qapi/misc.json +7 -1
@@ -1020,6 +1020,11 @@ # # @cpu-index: The CPU to use for commands that require an implicit CPU # +# Features: +# @savevm-blockdev-monitor-nodes: If present, the savevm monitor command +# correctly handles monitor owned block nodes +# when taking a snapshot. +# # Returns: the output of the command as a string # # Since: 0.14.0
@@ -1047,7 +1052,8 @@ ## { 'command': 'human-monitor-command', 'data': {'command-line': 'str', '*cpu-index': 'int'}, - 'returns': 'str' } + 'returns': 'str', + 'features' : [ { 'name': 'savevm-blockdev-monitor-nodes' } ] } ## # @change:
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 2/2] qapi: Allow introspecting fix for savevm's cooperation with blockdev
Quoted 20 lines · Peter Krempa
savevm was buggy as it considered all monitor owned block device nodes for snapshot. With introduction of -blockdev the common usage made all nodes including protocol nodes monitor owned and thus considered for snapshot. This was fixed but clients need to be able to detect whether this fix is present.
Since savevm does not have an QMP alternative add the feature for the 'human-monitor-command' backdoor which is used to call this command in modern use.
Signed-off-by Peter Krempa <[email protected]>--- qapi/misc.json | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)qapi/misc.json +0
-0
qapi/misc.json line 1026
Quoted 7 lines
qapi/misc.json +4
-0@@ -1020,6 +1020,11 @@ # # @cpu-index: The CPU to use for commands that require an implicit CPU # +# Features: +# @savevm-blockdev-monitor-nodes: If present, the savevm monitor command +# correctly handles monitor owned block nodes +# when taking a snapshot.
Is it worth adding a '(since 4.2)' on when features are added?
qapi/misc.json line 1056
Quoted 11 lines
qapi/misc.json +3
-1+# # Returns: the output of the command as a string # # Since: 0.14.0@@ -1047,7 +1052,8 @@ ## { 'command': 'human-monitor-command', 'data': {'command-line': 'str', '*cpu-index': 'int'}, - 'returns': 'str' } + 'returns': 'str', + 'features' : [ { 'name': 'savevm-blockdev-monitor-nodes' } ] }
We could, of course, actually implement a QMP 'savevm' and use _that_ as the introspection. But that's a bigger can of worms, so this is reasonable enough for the 4.2 timeframe.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH 2/2] qapi: Allow introspecting fix for savevm's cooperation with blockdev
Am 17.09.2019 um 18:33 hat Eric Blake geschrieben:
Quoted 16 lines · Peter Krempa
Quoted 15 lines
savevm was buggy as it considered all monitor owned block device nodes for snapshot. With introduction of -blockdev the common usage made all nodes including protocol nodes monitor owned and thus considered for snapshot. This was fixed but clients need to be able to detect whether this fix is present.
Since savevm does not have an QMP alternative add the feature for the 'human-monitor-command' backdoor which is used to call this command in modern use.
Signed-off-by Peter Krempa <[email protected]>--- qapi/misc.json | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
Quoted 14 lines
qapi/misc.json line 1026
Quoted 12 lines
qapi/misc.json +4
-0@@ -1020,6 +1020,11 @@ # # @cpu-index: The CPU to use for commands that require an implicit CPU # +# Features: +# @savevm-blockdev-monitor-nodes: If present, the savevm monitor command +# correctly handles monitor owned block nodes +# when taking a snapshot.Is it worth adding a '(since 4.2)' on when features are added?
I would also rather describe the change in behaviour ("only includes monitor owned block nodes if they have no parents") than saying that the behaviour is now correct.
(Not the least because we know that the current way still isn't quite correct, just hopefully correct enough: Block job BlockBackends are currently snapshotted, which is unintended and will be changed in the future. However, in practice people probably won't use block jobs on non-root nodes and internal snapshots together.)
Quoted 7 lines
Quoted 7 lines
qapi/misc.json +1
-0+# # Returns: the output of the command as a string # # Since: 0.14.0@@ -1047,7 +1052,8 @@ ## { 'command': 'human-monitor-command',
Quoted 8 lines
qapi/misc.json line 1056
Quoted 4 lines
qapi/misc.json +2
-1'data': {'command-line': 'str', '*cpu-index': 'int'}, - 'returns': 'str' } + 'returns': 'str', + 'features' : [ { 'name': 'savevm-blockdev-monitor-nodes' } ] }We could, of course, actually implement a QMP 'savevm' and use _that_ as the introspection. But that's a bigger can of worms, so this is reasonable enough for the 4.2 timeframe.
I think a QMP savevm would sidestep the whole issue by taking an explicit list of nodes to snapshot, and an explicit option to tell which node to store the vmstate on.
Kevin
Re: [Qemu-devel] [PATCH 2/2] qapi: Allow introspecting fix for savevm's cooperation with blockdev
Quoted 32 lines · Kevin Wolf
Am 17.09.2019 um 18:33 hat Eric Blake geschrieben:
Quoted 16 lines · Peter Krempa
Quoted 15 lines
savevm was buggy as it considered all monitor owned block device nodes for snapshot. With introduction of -blockdev the common usage made all nodes including protocol nodes monitor owned and thus considered for snapshot. This was fixed but clients need to be able to detect whether this fix is present.
Since savevm does not have an QMP alternative add the feature for the 'human-monitor-command' backdoor which is used to call this command in modern use.
Signed-off-by Peter Krempa <[email protected]>--- qapi/misc.json | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)Quoted 14 lines
qapi/misc.json line 1026
Quoted 12 lines
qapi/misc.json +4
-0@@ -1020,6 +1020,11 @@ # # @cpu-index: The CPU to use for commands that require an implicit CPU # +# Features: +# @savevm-blockdev-monitor-nodes: If present, the savevm monitor command +# correctly handles monitor owned block nodes +# when taking a snapshot.Is it worth adding a '(since 4.2)' on when features are added?
Quoted 3 lines
I would also rather describe the change in behaviour ("only includes monitor owned block nodes if they have no parents") than saying that the behaviour is now correct.
That's a good idea. I'll post it in a v2 if required.
Quoted 22 lines
(Not the least because we know that the current way still isn't quite correct, just hopefully correct enough: Block job BlockBackends are currently snapshotted, which is unintended and will be changed in the future. However, in practice people probably won't use block jobs on non-root nodes and internal snapshots together.)
Quoted 7 lines
Quoted 7 lines
qapi/misc.json +1
-0+# # Returns: the output of the command as a string # # Since: 0.14.0@@ -1047,7 +1052,8 @@ ## { 'command': 'human-monitor-command',Quoted 8 lines
qapi/misc.json line 1056
Quoted 4 lines
qapi/misc.json +2
-1'data': {'command-line': 'str', '*cpu-index': 'int'}, - 'returns': 'str' } + 'returns': 'str', + 'features' : [ { 'name': 'savevm-blockdev-monitor-nodes' } ] }We could, of course, actually implement a QMP 'savevm' and use _that_ as the introspection. But that's a bigger can of worms, so this is reasonable enough for the 4.2 timeframe.
Quoted 3 lines
I think a QMP savevm would sidestep the whole issue by taking an explicit list of nodes to snapshot, and an explicit option to tell which node to store the vmstate on.
A straight replacement for savevm would be quite pointless and also such discussion took already place multiple times in the past. The result always was that we need something better than just a qmp version of savevm.
I'll be happy to implement the libvirt support for the "new" savevm if it ever appears.