migration: add static analysis tool to check vmstate compat between versions
cover migration: add static analysis tool to check vmstate compat between versions
Hello,
This series adds a static vmstate checker to check for breakage of live migration by analyzing the vmstate information between different QEMU versions.
QEMU is modified to add a -dump-vmstate commandline option, which takes a filename as the argument. When invoked, QEMU dumps the vmstate info in JSON format for the current machine type to the file.
This file is then fed into the python script introduced in patch 2. The script takes 'src' and 'dest' arguments, indicating the direction of migration. The script then performs a series of checks and spews out information on inconsistencies it finds in the data.
Two stripped-down versions of JSON dumps are included in this patchset (patch 3).
Patches 4 - 18 contain modifications to those dumps, to show the checks that are carried out by the script. The result of running the script against the final version of those files is appended at the end.
The checks are to be performed for a particular machine type, and comparing different machine types is bound to turn up false-positives:
e.g.
(in a qemu 2.0 tree):
./x86_64-softmmu/qemu-system-x86_64 -dump-vmstate qemu-2.0.json
(in a qemu 2.2 tree:)
./x86_64-softmmu/qemu-system-x86_64 -dump-vmstate -M pc-2.0 qemu-2.2-m2.0.json
./scripts/vmstate-static-checker.py -s qemu-2.0.json -d qemu-2.2-m2.0.json
should not show any output.
The idea is to include this script in 'make check', ensuring new commits don't break migration.
Later, this script can also be baked into the release process: vmstate dumps for released versions of qemu for various machine types can be stored in a directory in the tree. At the time of freezing the tree for releases, (like -rc2), the dump for the current release can be updated. A policy that says "No more live migration-breaking changes can be accepted post -rc2" can be put in place. Also, for checks for older machine types on the current tree with the saved dumps should not indicate any regressions.
I expect there to be a few false positives at the start (though there aren't any right now). The script will keep evolving, though, to include new scenarios, to make it more helpful.
A later project would be to also store other guest-visible information and use that output for more checks (including lspci output from inside guests).
As mentioned earlier, this is the result of running the script against the sample json data included in this patchset:
$ ./scripts/vmstate-static-checker.py --src ./tests/vmstate-static-checker-data/dump1.json --dest ./tests/vmstate-static-checker-data/dump2.json Section "usb-kbd" Description "usb-kbd" Field "kbd.keycodes" size mismatch: 4 , 2 Section "PIIX3-xen" Description "PIIX3": minimum version error: 1 < 2 Section "PIIX3-xen" Description "PIIX3": Entry "Subsections" missing Section "tpci200": Description "tpci200" missing, got "tpci2002" instead; skipping Section "megasas", Description "PCIDevice": expected field "irq_state", while dest has no further fields Section "SUNW,fdtwo" Description "fdc": version error: 2 > 1 Section "SUNW,fdtwo", Description "fdrive": Subsection "fdrive/media_rate" not found Section "fusbh200-ehci-usb" version error: 2 > 1 Section "fusbh200-ehci-usb", Description "ehci-core": expected field "usbsts", got "usbsts_pending"; skipping rest Section "intel-hda-generic", Description "intel-hda", Field "pci": missing description Section "cfi.pflash01": Entry "Description" missing Section "pci-serial-4x" Description "pci-serial-multi": Entry "Fields" missing Warning: checking incompatible machine types: "pc-i440fx-2.1", "pc-i440fx-2.2" Section "fw_cfg" does not exist in dest
Amit Shah (18):
migration: dump vmstate info as a json file for static analysis
vmstate-static-checker: script to validate vmstate changes
tests: vmstate static checker: add dump1 and dump2 files
tests: vmstate static checker: incompat machine types
tests: vmstate static checker: add version error in main section
tests: vmstate static checker: version mismatch inside a Description
tests: vmstate static checker: minimum_version_id check
tests: vmstate static checker: remove a section
tests: vmstate static checker: remove a field
tests: vmstate static checker: remove last field in a struct
tests: vmstate static checker: change description name
tests: vmstate static checker: remove Fields
tests: vmstate static checker: remove Description
tests: vmstate static checker: remove Description inside Fields
tests: vmstate static checker: remove a subsection
tests: vmstate static checker: remove Subsections
tests: vmstate static checker: add substructure for usb-kbd for hid
section
tests: vmstate static checker: add size mismatch inside substructure
include/migration/vmstate.h | 2 + qemu-options.hx | 9 + savevm.c | 134 +++ scripts/vmstate-static-checker.py | 336 ++++++++ tests/vmstate-static-checker-data/dump1.json | 1163 ++++++++++++++++++++++++++ tests/vmstate-static-checker-data/dump2.json | 968 +++++++++++++++++++++ vl.c | 14 + 7 files changed, 2626 insertions(+) create mode 100755 scripts/vmstate-static-checker.py create mode 100644 tests/vmstate-static-checker-data/dump1.json create mode 100644 tests/vmstate-static-checker-data/dump2.json
Signature · 1 line
1.9.0
PATCH 1/18 migration: dump vmstate info as a json file for static analysis
This commit adds a new command, '-dump-vmstate', that takes a filename as a parameter. When executed, QEMU will dump the vmstate information for the machine type it's invoked with to the file, and quit.
The JSON-format output can then be used to compare the vmstate info for different QEMU versions, specifically to test whether live migration would break due to changes in the vmstate data.
This is based on a version from Andreas Färber posted here:
https://lists.gnu.org/archive/html/qemu-devel/2013-10/msg03095.html
A Python script that compares the output of such JSON dumps is included in the following commit.
--- include/migration/vmstate.h | 2 + qemu-options.hx | 9 +++ savevm.c | 134 ++++++++++++++++++++++++++++++++++++++++++++ vl.c | 14 +++++ 4 files changed, 159 insertions(+)
include/migration/vmstate.h +2 -0
@@ -778,4 +778,6 @@ void vmstate_register_ram(struct MemoryRegion *memory, DeviceState *dev);void vmstate_unregister_ram (struct MemoryRegion *memory,DeviceState *dev);void vmstate_register_ram_global (struct MemoryRegion *memory); +void dump_vmstate_json_to_file (FILE *out_fp); +#endif
qemu-options.hx +9 -0
@@ -3146,6 +3146,15 @@ STEXI prepend a timestamp to each log message.(default:on) ETEXI +DEF("dump-vmstate", HAS_ARG, QEMU_OPTION_dump_vmstate, + "-dump-vmstate <file>\n" "", QEMU_ARCH_ALL) +STEXI +@item -dump-vmstate @var{file} +@findex -dump-vmstate +Dump json-encoded vmstate information for current machine type to file +in @var{file} +ETEXI + HXCOMM This is the last statement. Insert new options before this line! STEXI @end table
savevm.c +134 -0
146 lines not on this page · raw message
vl.c +14 -0
@@ -2988,6 +2988,7 @@ int main(int argc, char **argv, char **envp)const char *trace_file =NULL ;const ram_addr_t default_ram_size = (ram_addr_t )DEFAULT_RAM_SIZE *1024 *1024 ; +FILE *vmstate_dump_file =NULL ;atexit (qemu_run_exit_notifiers);error_set_progname (argv[0 ]);
@@ -3956,6 +3957,13 @@ int main(int argc, char **argv, char **envp) }configure_msg (opts);break ; +case QEMU_OPTION_dump_vmstate: + vmstate_dump_file =fopen (optarg,"w" ); +if (vmstate_dump_file ==NULL ) { +fprintf (stderr,"open %s: %s\n" , optarg,strerror (errno)); +exit (1 ); + } +break ;default :os_parse_cmd_args (popt->index, optarg); }
@@ -4533,6 +4541,12 @@ int main(int argc, char **argv, char **envp) } } +if (vmstate_dump_file) { +/* dump and exit */ +dump_vmstate_json_to_file (vmstate_dump_file); +return 0 ; + } +if (incoming) {Error *local_err =NULL ;qemu_start_incoming_migration (incoming, &local_err);
Signature · 1 line
1.9.0
Re: [Qemu-devel] [PATCH 01/18] migration: dump vmstate info as a json file for static analysis
Quoted 7 lines · Amit Shah
This commit adds a new command, '-dump-vmstate', that takes a filename as a parameter. When executed, QEMU will dump the vmstate information for the machine type it's invoked with to the file, and quit.
The JSON-format output can then be used to compare the vmstate info for different QEMU versions, specifically to test whether live migration would break due to changes in the vmstate data.
Are we going to document that JSON format anywhere? Is it worth making it part of qapi-schema.json, whether to also expose the dump via a QMP command, or at a bare minimum to take advantage of some code generation from the qapi engine rather than doing it all by hand?
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH 01/18] migration: dump vmstate info as a json file for static analysis
Hi,
Quoted 10 lines · Eric Blake
Quoted 7 lines · Amit Shah
This commit adds a new command, '-dump-vmstate', that takes a filename as a parameter. When executed, QEMU will dump the vmstate information for the machine type it's invoked with to the file, and quit.
The JSON-format output can then be used to compare the vmstate info for different QEMU versions, specifically to test whether live migration would break due to changes in the vmstate data.
Are we going to document that JSON format anywhere?
I suppose we should, I thought I should wait for comments here on any extra fields that people want.
I suppose documenting would just be coming up with a schema, though? ...
Quoted 2 lines
Is it worth making
it part of qapi-schema.json,
but documenting the entire schema is difficult, as each device will have its own schema text (due to the differing fields). At least that's how I understand it; please correct me if I'm wrong.
Quoted 3 lines
whether to also expose the dump via a QMP
command, or at a bare minimum to take advantage of some code generation from the qapi engine rather than doing it all by hand?
Surely -- I did give it some thought at the initial stages, but since I've not had to modify this much since I first wrote it, and it does work well, I'm just thinking it's alright if we do that later. Focus right now is on the tool itself.
Amit
Re: [Qemu-devel] [PATCH 01/18] migration: dump vmstate info as a json file for static analysis
Quoted 17 lines · Amit Shah
Hi,
Quoted 10 lines · Eric Blake
Quoted 7 lines · Amit Shah
This commit adds a new command, '-dump-vmstate', that takes a filename as a parameter. When executed, QEMU will dump the vmstate information for the machine type it's invoked with to the file, and quit.
The JSON-format output can then be used to compare the vmstate info for different QEMU versions, specifically to test whether live migration would break due to changes in the vmstate data.
Are we going to document that JSON format anywhere?
I suppose we should, I thought I should wait for comments here on any extra fields that people want.
Quoted 8 lines
I suppose documenting would just be coming up with a schema, though? ...
Quoted 2 lines
Is it worth making
it part of qapi-schema.json,but documenting the entire schema is difficult, as each device will have its own schema text (due to the differing fields). At least that's how I understand it; please correct me if I'm wrong.
If field names form JSON keys, then yes, documenting each device will be its own schema. But if you write things generic enough, you can have a single schema that covers all devices, by ensuring that device-specific field names are supplied only as values, not keys.
That is,
'device': { 'name': 'foo',
'fields': { 'field1': 'int32', 'field2': 'int64' }
}
requires a schema for each device, while
'device': { 'name': 'foo',
'fields': [
{ 'name': 'field1', 'type': 'int32' },
{ 'name': 'field2', 'type': 'int64' } ]
}
is generic. It is more verbose and requires more structure, but by isolating device details into values rather than keys, you get rid of the need for per-device schemas.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH 01/18] migration: dump vmstate info as a json file for static analysis
In reply to Amit ShahQuoted 66 lines · * Amit Shah ([email protected]) wrote:
This commit adds a new command, '-dump-vmstate', that takes a filename as a parameter. When executed, QEMU will dump the vmstate information for the machine type it's invoked with to the file, and quit.
The JSON-format output can then be used to compare the vmstate info for different QEMU versions, specifically to test whether live migration would break due to changes in the vmstate data.
This is based on a version from Andreas Färber posted here:
https://lists.gnu.org/archive/html/qemu-devel/2013-10/msg03095.htmlA Python script that compares the output of such JSON dumps is included in the following commit.
Signed-off-by Amit Shah <[email protected]>--- include/migration/vmstate.h | 2 + qemu-options.hx | 9 +++ savevm.c | 134 ++++++++++++++++++++++++++++++++++++++++++++ vl.c | 14 +++++ 4 files changed, 159 insertions(+)include/migration/vmstate.h +2
-0@@ -778,4 +778,6 @@ void vmstate_register_ram(struct MemoryRegion *memory, DeviceState *dev);void vmstate_unregister_ram (struct MemoryRegion *memory,DeviceState *dev);void vmstate_register_ram_global (struct MemoryRegion *memory); +void dump_vmstate_json_to_file (FILE *out_fp); +#endif qemu-options.hx +9
-0@@ -3146,6 +3146,15 @@ STEXI prepend a timestamp to each log message.(default:on) ETEXI +DEF("dump-vmstate", HAS_ARG, QEMU_OPTION_dump_vmstate, + "-dump-vmstate <file>\n" "", QEMU_ARCH_ALL) +STEXI +@item -dump-vmstate @var{file} +@findex -dump-vmstate +Dump json-encoded vmstate information for current machine type to file +in @var{file} +ETEXI + HXCOMM This is the last statement. Insert new options before this line! STEXI @end tablesavevm.c +1
-0@@ -24,6 +24,7 @@#include "config-host.h" #include "qemu-common.h" +#include "hw/boards.h" #include "hw/hw.h" #include "hw/qdev.h" #include "net/net.h"
savevm.c static QTAILQ_HEAD(savevm_handlers, SaveStateEntry) savevm_handlers = · line 250
Quoted 9 lines
savevm.c +6
-0@@ -241,6 +242,139 @@ static QTAILQ_HEAD(savevm_handlers, SaveStateEntry) savevm_handlers =QTAILQ_HEAD_INITIALIZER (savevm_handlers);static int global_section_id; +static void dump_vmstate_vmsd (FILE *out_file, +const VMStateDescription *vmsd,int indent, +bool is_subsection); + +static void dump_vmstate_vmsf (FILE *out_file,const VMStateField *field, +int indent)
checkpatch points out that some tabs managed to get into that indent line.
Generally I think this patch is OK and quite useful; two thoughts:
1) I was surprised it dumped every object type, rather than just those that are instantiated; I think the latter would be more use in some circumstances, since there's a load of weird and wonderful objects that exist and are very rarely used.
2) 'fields_exists' is a weird naming to put in the json file - it's a function pointer for determining if the field is going to be present; maybe renaming as 'conditional' would make sense.
Dave
--
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Re: [Qemu-devel] [PATCH 01/18] migration: dump vmstate info as a json file for static analysis
Quoted 70 lines · Dr. David Alan Gilbert
Quoted 69 lines · * Amit Shah ([email protected]) wrote:
This commit adds a new command, '-dump-vmstate', that takes a filename as a parameter. When executed, QEMU will dump the vmstate information for the machine type it's invoked with to the file, and quit.
The JSON-format output can then be used to compare the vmstate info for different QEMU versions, specifically to test whether live migration would break due to changes in the vmstate data.
This is based on a version from Andreas Färber posted here:
https://lists.gnu.org/archive/html/qemu-devel/2013-10/msg03095.htmlA Python script that compares the output of such JSON dumps is included in the following commit.
Signed-off-by Amit Shah <[email protected]>--- include/migration/vmstate.h | 2 + qemu-options.hx | 9 +++ savevm.c | 134 ++++++++++++++++++++++++++++++++++++++++++++ vl.c | 14 +++++ 4 files changed, 159 insertions(+)include/migration/vmstate.h +2
-0@@ -778,4 +778,6 @@ void vmstate_register_ram(struct MemoryRegion *memory, DeviceState *dev);void vmstate_unregister_ram (struct MemoryRegion *memory,DeviceState *dev);void vmstate_register_ram_global (struct MemoryRegion *memory); +void dump_vmstate_json_to_file (FILE *out_fp); +#endif qemu-options.hx +9
-0@@ -3146,6 +3146,15 @@ STEXI prepend a timestamp to each log message.(default:on) ETEXI +DEF("dump-vmstate", HAS_ARG, QEMU_OPTION_dump_vmstate, + "-dump-vmstate <file>\n" "", QEMU_ARCH_ALL) +STEXI +@item -dump-vmstate @var{file} +@findex -dump-vmstate +Dump json-encoded vmstate information for current machine type to file +in @var{file} +ETEXI + HXCOMM This is the last statement. Insert new options before this line! STEXI @end tablesavevm.c +1
-0@@ -24,6 +24,7 @@#include "config-host.h" #include "qemu-common.h" +#include "hw/boards.h" #include "hw/hw.h" #include "hw/qdev.h" #include "net/net.h" @@ -241,6 +242,139 @@ static QTAILQ_HEAD(savevm_handlers, SaveStateEntry) savevm_handlers =QTAILQ_HEAD_INITIALIZER (savevm_handlers);static int global_section_id;
Quoted 8 lines
savevm.c static QTAILQ_HEAD(savevm_handlers, SaveStateEntry) savevm_handlers = · line 250
Quoted 6 lines
savevm.c +6
-0+static void dump_vmstate_vmsd (FILE *out_file, +const VMStateDescription *vmsd,int indent, +bool is_subsection); + +static void dump_vmstate_vmsf (FILE *out_file,const VMStateField *field, +int indent)checkpatch points out that some tabs managed to get into that indent line.
Will fix.
Quoted 5 lines
Generally I think this patch is OK and quite useful; two thoughts:
1) I was surprised it dumped every object type, rather than just those that are instantiated; I think the latter would be more use in some circumstances, since there's a load of weird and wonderful objects that exist and are very rarely used.
The idea is to be able to take a qemu binary and compare with another binary; if only fields that are instantiated are used, various invocations will have to be tried to find devices that may have broken.
An alternative way of checking only devices which have been added to the running machine can be done via a monitor command (or a parameter to the existing cmdline option). But I'm not sure if that'll be more useful than the current one.
Quoted 3 lines
2) '
fields_exists' is a weird naming to put in the json file - it's a function pointer for determining if the field is going to be present; maybe renaming as 'conditional' would make sense.
Yea; I don't know if field_exists is going to be useful anyway. It's runtime info rather than static, so perhaps can just be dropped. Right now, anyway, the checker doesn't make use of this field at all.
Thanks for taking a look!
Amit
Re: [Qemu-devel] [PATCH 01/18] migration: dump vmstate info as a json file for static analysis
Quoted 93 lines · * Amit Shah ([email protected]) wrote:
Quoted 70 lines · Dr. David Alan Gilbert
Quoted 69 lines · * Amit Shah ([email protected]) wrote:
This commit adds a new command, '-dump-vmstate', that takes a filename as a parameter. When executed, QEMU will dump the vmstate information for the machine type it's invoked with to the file, and quit.
The JSON-format output can then be used to compare the vmstate info for different QEMU versions, specifically to test whether live migration would break due to changes in the vmstate data.
This is based on a version from Andreas Färber posted here:
https://lists.gnu.org/archive/html/qemu-devel/2013-10/msg03095.htmlA Python script that compares the output of such JSON dumps is included in the following commit.
Signed-off-by Amit Shah <[email protected]>--- include/migration/vmstate.h | 2 + qemu-options.hx | 9 +++ savevm.c | 134 ++++++++++++++++++++++++++++++++++++++++++++ vl.c | 14 +++++ 4 files changed, 159 insertions(+)include/migration/vmstate.h +2
-0@@ -778,4 +778,6 @@ void vmstate_register_ram(struct MemoryRegion *memory, DeviceState *dev);void vmstate_unregister_ram (struct MemoryRegion *memory,DeviceState *dev);void vmstate_register_ram_global (struct MemoryRegion *memory); +void dump_vmstate_json_to_file (FILE *out_fp); +#endif qemu-options.hx +9
-0@@ -3146,6 +3146,15 @@ STEXI prepend a timestamp to each log message.(default:on) ETEXI +DEF("dump-vmstate", HAS_ARG, QEMU_OPTION_dump_vmstate, + "-dump-vmstate <file>\n" "", QEMU_ARCH_ALL) +STEXI +@item -dump-vmstate @var{file} +@findex -dump-vmstate +Dump json-encoded vmstate information for current machine type to file +in @var{file} +ETEXI + HXCOMM This is the last statement. Insert new options before this line! STEXI @end tablesavevm.c +1
-0@@ -24,6 +24,7 @@#include "config-host.h" #include "qemu-common.h" +#include "hw/boards.h" #include "hw/hw.h" #include "hw/qdev.h" #include "net/net.h" @@ -241,6 +242,139 @@ static QTAILQ_HEAD(savevm_handlers, SaveStateEntry) savevm_handlers =QTAILQ_HEAD_INITIALIZER (savevm_handlers);static int global_section_id;Quoted 8 lines
savevm.c static QTAILQ_HEAD(savevm_handlers, SaveStateEntry) savevm_handlers = · line 250
Quoted 6 lines
savevm.c +6
-0+static void dump_vmstate_vmsd (FILE *out_file, +const VMStateDescription *vmsd,int indent, +bool is_subsection); + +static void dump_vmstate_vmsf (FILE *out_file,const VMStateField *field, +int indent)checkpatch points out that some tabs managed to get into that indent line.
Will fix.
Quoted 5 lines
Generally I think this patch is OK and quite useful; two thoughts:
1) I was surprised it dumped every object type, rather than just those that are instantiated; I think the latter would be more use in some circumstances, since there's a load of weird and wonderful objects that exist and are very rarely used.The idea is to be able to take a qemu binary and compare with another binary; if only fields that are instantiated are used, various invocations will have to be tried to find devices that may have broken.
Quoted 4 lines
An alternative way of checking only devices which have been added to the running machine can be done via a monitor command (or a parameter to the existing cmdline option). But I'm not sure if that'll be more useful than the current one.
Or perhaps a way to dump that info and mask your checker with it if wanted?
Quoted 7 lines
Quoted 3 lines
2) '
fields_exists' is a weird naming to put in the json file - it's a function pointer for determining if the field is going to be present; maybe renaming as 'conditional' would make sense.Yea; I don't know if
field_existsis going to be useful anyway. It's runtime info rather than static, so perhaps can just be dropped. Right now, anyway, the checker doesn't make use of this field at all.
I think it's useful to have field_exists because it lets you know that it's conditional, I just think it's weird naming it like that in the json, since an entry in the json that says 'fields_exists: true' sounds like the field always exists, which is the opposite of what it means. It's just a naming thing here.
Dave
--
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Re: [Qemu-devel] [PATCH 01/18] migration: dump vmstate info as a json file for static analysis
Quoted 7 lines · Dr. David Alan Gilbert
Quoted 5 lines · * Amit Shah ([email protected]) wrote:
The idea is to be able to take a qemu binary and compare with another binary; if only fields that are instantiated are used, various invocations will have to be tried to find devices that may have broken.
Quoted 6 lines
Quoted 4 lines
An alternative way of checking only devices which have been added to the running machine can be done via a monitor command (or a parameter to the existing cmdline option). But I'm not sure if that'll be more useful than the current one.
Or perhaps a way to dump that info and mask your checker with it if wanted?
A 'blacklist' file, which stores names of sections that you're not interested in?
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
2) '
fields_exists' is a weird naming to put in the json file - it's a function pointer for determining if the field is going to be present; maybe renaming as 'conditional' would make sense.Yea; I don't know if
field_existsis going to be useful anyway. It's runtime info rather than static, so perhaps can just be dropped. Right now, anyway, the checker doesn't make use of this field at all.
Quoted 5 lines
I think it's useful to have
field_existsbecause it lets you know that it's conditional, I just think it's weird naming it like that in the json, since an entry in the json that says 'fields_exists: true' sounds like the field always exists, which is the opposite of what it means. It's just a naming thing here.
On the name of the field, I doubt anyone will read the json file itself to get confused by it. Also, it stays true to what the field is called in the actual vmstate structs in qemu.
On the usability of the field: it's like subsections: they may exist or not, but we should check them nevertheless on src and dest, and any difference should be flagged.
Amit
Re: [Qemu-devel] [PATCH 01/18] migration: dump vmstate info as a json file for static analysis
Quoted 13 lines · Amit Shah
Quoted 12 lines · Dr. David Alan Gilbert
Quoted 9 lines · * Amit Shah ([email protected]) wrote:
The idea is to be able to take a qemu binary and compare with another binary; if only fields that are instantiated are used, various invocations will have to be tried to find devices that may have broken.
An alternative way of checking only devices which have been added to the running machine can be done via a monitor command (or a parameter to the existing cmdline option). But I'm not sure if that'll be more useful than the current one.
Quoted 4 lines
Quoted 1 line
Or perhaps a way to dump that info and mask your checker with it if wanted?
A 'blacklist' file, which stores names of sections that you're not interested in?
An error message format that lets me grep -v for sections I'm not interested in? Stupidest solution that could possibly work...
Re: [Qemu-devel] [PATCH 01/18] migration: dump vmstate info as a json file for static analysis
In reply to Dr. David Alan GilbertQuoted 80 lines · Dr. David Alan Gilbert
Quoted 66 lines · * Amit Shah ([email protected]) wrote:
This commit adds a new command, '-dump-vmstate', that takes a filename as a parameter. When executed, QEMU will dump the vmstate information for the machine type it's invoked with to the file, and quit.
The JSON-format output can then be used to compare the vmstate info for different QEMU versions, specifically to test whether live migration would break due to changes in the vmstate data.
This is based on a version from Andreas Färber posted here:
https://lists.gnu.org/archive/html/qemu-devel/2013-10/msg03095.htmlA Python script that compares the output of such JSON dumps is included in the following commit.
Signed-off-by Amit Shah <[email protected]>--- include/migration/vmstate.h | 2 + qemu-options.hx | 9 +++ savevm.c | 134 ++++++++++++++++++++++++++++++++++++++++++++ vl.c | 14 +++++ 4 files changed, 159 insertions(+)include/migration/vmstate.h +2
-0@@ -778,4 +778,6 @@ void vmstate_register_ram(struct MemoryRegion *memory, DeviceState *dev);void vmstate_unregister_ram (struct MemoryRegion *memory,DeviceState *dev);void vmstate_register_ram_global (struct MemoryRegion *memory); +void dump_vmstate_json_to_file (FILE *out_fp); +#endif qemu-options.hx +9
-0@@ -3146,6 +3146,15 @@ STEXI prepend a timestamp to each log message.(default:on) ETEXI +DEF("dump-vmstate", HAS_ARG, QEMU_OPTION_dump_vmstate, + "-dump-vmstate <file>\n" "", QEMU_ARCH_ALL) +STEXI +@item -dump-vmstate @var{file} +@findex -dump-vmstate +Dump json-encoded vmstate information for current machine type to file +in @var{file} +ETEXI + HXCOMM This is the last statement. Insert new options before this line! STEXI @end tablesavevm.c +1
-0@@ -24,6 +24,7 @@#include "config-host.h" #include "qemu-common.h" +#include "hw/boards.h" #include "hw/hw.h" #include "hw/qdev.h" #include "net/net.h" savevm.c static QTAILQ_HEAD(savevm_handlers, SaveStateEntry) savevm_handlers = · line 250
Quoted 9 lines
savevm.c +6
-0@@ -241,6 +242,139 @@ static QTAILQ_HEAD(savevm_handlers, SaveStateEntry) savevm_handlers =QTAILQ_HEAD_INITIALIZER (savevm_handlers);static int global_section_id; +static void dump_vmstate_vmsd (FILE *out_file, +const VMStateDescription *vmsd,int indent, +bool is_subsection); + +static void dump_vmstate_vmsf (FILE *out_file,const VMStateField *field, +int indent)checkpatch points out that some tabs managed to get into that indent line.
Quoted 5 lines
Generally I think this patch is OK and quite useful; two thoughts:
1) I was surprised it dumped every object type, rather than just those that are instantiated; I think the latter would be more use in some circumstances, since there's a load of weird and wonderful objects that exist and are very rarely used.
Dumping everything lets you reason about what *could* happen at runtime. Which is the point of static checking, isn't it?
Optionally dumping only instantiated stuff shouldn't be hard, if it turns out to be useful.
[...]
PATCH 2/18 vmstate-static-checker: script to validate vmstate changes
In reply to Amit ShahThis script compares the vmstate dumps in JSON format as output by QEMU with the -dump-vmstate option.
It flags various errors, like version mismatch, sections going away, size mismatches, etc.
This script is tolerant of a few changes that do not change the on-wire format, like embedding a few fields within substructs.
The script takes -s/--src and -d/--dest parameters, to which filenames are given as arguments.
--- scripts/vmstate-static-checker.py | 336 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 336 insertions(+) create mode 100755 scripts/vmstate-static-checker.py
scripts/vmstate-static-checker.py new file +336 -0
@@ -0,0 +1,336 @@ +#!/usr/bin/python +# +# Compares vmstate information stored in JSON format, obtained from +# the -dump-vmstate QEMU command. +# +# Copyright 2014 Amit Shah <[email protected]> +# Copyright 2014 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, see <http://www.gnu.org/licenses/>. + +import json +import argparse + +def check_fields_match(s_field, d_field): + # Some fields changed names between qemu versions. This list + # is used to whitelist such changes. + changed_names = ['dev', 'pcidev', 'pci_dev', 'd', \ + 'parent_obj', 'parent_obj.parent_obj', \ + 'parent_obj.parent_obj.parent_obj', \ + 'bridge.dev', 'bridge.dev.shpc', 'shpc', \ + 'card', 'port.br.dev', 'port.br.dev.exp.aer_log', \ + 'br.dev', 'br.dev.exp.aer_log', \ + 'parent_obj.parent_obj.parent_obj.exp.aer_log', \ + 'parent_obj.parent_obj.exp.aer_log', \ + 'pci0_status', \ + 'acpi_pci_hotplug.acpi_pcihp_pci_status[0x0]', \ + 'pci_irq_levels', 'pci_irq_levels_vmstate', \ + 'num_surfaces', 'ssd.num_surfaces', \ + 'timer', 'timer_expiry', \ + 'usb-ptr-queue', 'HIDPointerEventQueue' ] + + if s_field == d_field: + return True + + if s_field in changed_names and d_field in changed_names: + return True + + return False + + +def exists_in_substruct(fields, item): + # Some QEMU versions moved a few fields inside a substruct. This + # kept the on-wire format the same. This function checks if + # something got shifted inside a substruct. For example, the + # change in commit 1f42d22233b4f3d1a2933ff30e8d6a6d9ee2d08f + + try: + desc = fields["Description"] + except KeyError: + return False + + try: + substruct_fields = desc["Fields"] + except KeyError: + return False + + d_iter = iter(substruct_fields) + + try: + d_item = d_iter.next() + except StopIteration: + return False + + return check_fields_match(d_item["field"], item) + + +def forward_to_substruct(fields): + # We don't need to check for failure here as exists_in_substruct() + # would have already done that. This is called only when a + # substruct exists. + + desc = fields["Description"] + substruct_fields = desc["Fields"] + + return substruct_fields + + +def check_fields(src_fields, dest_fields, desc, sec): + # This function checks for all the fields in a section. If some + # fields got embedded into a substruct, this function will also + # attempt to check inside the substruct. + + d_iter = iter(dest_fields) + s_iter = iter(src_fields) + + # Using these lists as stacks to store previous value of s_iter + # and d_iter, so that when time comes to exit out of a substruct, + # we can go back one level up and continue from where we left off. + + s_iter_list = [] + d_iter_list = [] + + advance_src = True + advance_dest = True + + while True: + if advance_src: + try: + s_item = s_iter.next() + except StopIteration: + if not s_iter_list: + break + + s_iter = s_iter_list.pop() + continue + else: + # We want to avoid advancing just once -- when entering a + # substruct, or when exiting one. + advance_src = True + + if advance_dest: + try: + d_item = d_iter.next() + except StopIteration: + if not d_iter_list: + # We were not in a substruct + print "Section \"" + sec + "\",", + print "Description " + "\"" + desc + "\":", + print "expected field \"" + s_item["field"] + "\",", + print "while dest has no further fields" + break + + d_iter = d_iter_list.pop() + advance_src = False + continue + else: + advance_dest = True + + match_found = check_fields_match(s_item["field"], d_item["field"]) + + if not match_found: + in_dest_substruct = False + in_src_substruct = False + # Some fields were put in substructs, keeping the + # on-wire format the same, but breaking static tools + # like this one. + + # First, check if dest has a new substruct. + in_dest_substruct = exists_in_substruct(d_item, s_item["field"]) + if in_dest_substruct: + # listiterators don't have a prev() function, so we + # have to store our current location, descend into the + # substruct, and ensure we come out as if nothing + # happened when the substruct is over. + # + # Essentially we're opening the substructs that got + # added which didn't change the wire format. + d_iter_list.append(d_iter) + substruct_fields = forward_to_substruct(d_item) + d_iter = iter(substruct_fields) + advance_src = False + continue + else: + # Next, check if src has substruct that dest removed + # (can happen in backward migration: 2.0 -> 1.5) + in_src_substruct = exists_in_substruct(s_item, d_item["field"]) + if in_src_substruct: + s_iter_list.append(s_iter) + substruct_fields = forward_to_substruct(s_item) + s_iter = iter(substruct_fields) + advance_dest = False + continue + else: + print "Section \"" + sec + "\",", + print "Description \"" + desc + "\":", + print "expected field \"" + s_item["field"] + "\",", + print "got \"" + d_item["field"] + "\"; skipping rest" + break + + check_version(s_item, d_item, sec, desc) + + try: + s_item["Description"] + except KeyError: + # Check size of this field only if it's not a VMSTRUCT entry + check_size(s_item, d_item, sec, desc, s_item["field"]) + + check_description_in_list(s_item, d_item, sec, desc) + + +def check_subsections(src_sub, dest_sub, desc, sec): + for s_index, s_item in enumerate(src_sub): + found = False + for d_index, d_item in enumerate(dest_sub): + if s_item["name"] != d_item["name"]: + continue + + found = True + check_descriptions(s_item, d_item, sec) + + if found == False: + print "Section \"" + sec + "\", Description \"" + desc + "\":", + print "Subsection \"" + s_item["name"] + "\" not found" + + +def check_description_in_list(s_item, d_item, sec, desc): + try: + s_item["Description"] + except KeyError: + return + + try: + d_item["Description"] + except KeyError: + print "Section \"" + sec + "\", Description \"" + desc + "\",", + print "Field \"" + s_item["field"] + "\": missing description" + return + + check_descriptions(s_item["Description"], d_item["Description"], sec) + + +def check_descriptions(src_desc, dest_desc, sec): + check_version(src_desc, dest_desc, sec, src_desc["name"]) + + if not (check_fields_match(src_desc["name"], dest_desc["name"])): + print "Section \"" + sec + "\":", + print "Description \"" + src_desc["name"] + "\"", + print "missing, got \"" + dest_desc["name"] +"\" instead; skipping" + return + + for f in src_desc: + try: + dest_desc[f] + except KeyError: + print "Section \"" + sec + "\"", + print "Description \"" + src_desc["name"] + "\":", + print "Entry \"" + f + "\" missing" + continue + + if f == 'Fields': + check_fields(src_desc[f], dest_desc[f], src_desc["name"], sec) + + if f == 'Subsections': + check_subsections(src_desc[f], dest_desc[f], src_desc["name"], sec) + + +def check_version(s, d, sec, desc=None): + if s["version_id"] > d["version_id"]: + print "Section \"" + sec + "\"", + if desc: + print "Description \"" + desc + "\":", + print "version error:", s["version_id"], ">", d["version_id"] + + try: + d["minimum_version_id"] + except KeyError: + return + + if s["version_id"] < d["minimum_version_id"]: + print "Section \"" + sec + "\"", + if desc: + print "Description \"" + desc + "\":", + print "minimum version error:", s["version_id"], "<", + print d["minimum_version_id"] + + +def check_size(s, d, sec, desc=None, field=None): + if s["size"] != d["size"]: + print "Section \"" + sec + "\"", + if desc: + print "Description \"" + desc + "\"", + if field: + print "Field \"" + field + "\"", + print "size mismatch:", s["size"], ",", d["size"] + + +def check_machine_type(s, d, sec): + if s["Name"] != d["Name"]: + print "Warning: checking incompatible machine types:", + print "\"" + s["Name"] + "\", \"" + d["Name"] + "\"" + return + + +def main(): + parser = argparse.ArgumentParser(description= + 'Parse vmstate dumps from QEMU.') + parser.add_argument('-s', '--src', type=file, required=True, + help='json dump from src qemu') + parser.add_argument('-d', '--dest', type=file, required=True, + help='json dump from dest qemu') + parser.add_argument('--reverse', required=False, default=False, + action='store_true', + help='reverse the direction') + args = parser.parse_args() + + src_data = json.load(args.src) + dest_data = json.load(args.dest) + args.src.close() + args.dest.close() + + if args.reverse == True: + temp = src_data + src_data = dest_data + dest_data = temp + + for sec in src_data: + try: + dest_data[sec] + except KeyError: + print "Section \"" + sec + "\" does not exist in dest" + continue + + s = src_data[sec] + d = dest_data[sec] + + if sec == "vmschkmachine": + check_machine_type(s, d, sec); + continue; + + check_version(s, d, sec) + + for entry in s: + try: + d[entry] + except KeyError: + print "Section \"" + sec + "\": Entry \"" + entry + "\"", + print "missing" + continue + + if entry == "Description": + check_descriptions(s[entry], d[entry], sec) + + +if __name__ == '__main__': + main()
Signature · 1 line
1.9.0
PATCH 3/18 tests: vmstate static checker: add dump1 and dump2 files
In reply to Amit ShahThese are stripped-down JSON data as obtained from the -dump-vmstate option. The two files are identical in this commit, and will be modified in the later commits to show what the script does with the data.
--- tests/vmstate-static-checker-data/dump1.json | 1163 ++++++++++++++++++++++++++ tests/vmstate-static-checker-data/dump2.json | 1163 ++++++++++++++++++++++++++ 2 files changed, 2326 insertions(+) create mode 100644 tests/vmstate-static-checker-data/dump1.json create mode 100644 tests/vmstate-static-checker-data/dump2.json
tests/vmstate-static-checker-data/dump1.json new file +1163 -0
@@ -0,0 +1,1163 @@ +{ + "vmschkmachine": { + "Name": "pc-i440fx-2.1" + }, + "fw_cfg": { + "Name": "fw_cfg", + "version_id": 2, + "minimum_version_id": 1, + "Description": { + "name": "fw_cfg", + "version_id": 2, + "minimum_version_id": 1, + "Fields": [ + { + "field": "cur_entry", + "version_id": 0, + "field_exists": false, + "size": 2 + }, + { + "field": "cur_offset", + "version_id": 0, + "field_exists": true, + "size": 4 + }, + { + "field": "cur_offset", + "version_id": 2, + "field_exists": false, + "size": 4 + } + ] + } + }, + "fusbh200-ehci-usb": { + "Name": "fusbh200-ehci-usb", + "version_id": 2, + "minimum_version_id": 1, + "Description": { + "name": "ehci-sysbus", + "version_id": 2, + "minimum_version_id": 1, + "Fields": [ + { + "field": "ehci", + "version_id": 2, + "field_exists": false, + "size": 1880, + "Description": { + "name": "ehci-core", + "version_id": 2, + "minimum_version_id": 1, + "Fields": [ + { + "field": "usbcmd", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "usbsts", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "usbsts_pending", + "version_id": 2, + "field_exists": false, + "size": 4 + }, + { + "field": "usbsts_frindex", + "version_id": 2, + "field_exists": false, + "size": 4 + }, + { + "field": "usbintr", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "frindex", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "ctrldssegment", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "periodiclistbase", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "asynclistaddr", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "configflag", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "portsc[0]", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "portsc[1]", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "portsc[2]", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "portsc[3]", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "portsc[4]", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "portsc[5]", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "frame_timer", + "version_id": 0, + "field_exists": false, + "size": 8 + }, + { + "field": "last_run_ns", + "version_id": 0, + "field_exists": false, + "size": 8 + }, + { + "field": "async_stepdown", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "astate", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "pstate", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "a_fetch_addr", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "p_fetch_addr", + "version_id": 0, + "field_exists": false, + "size": 4 + } + ] + } + } + ] + } + }, + "pci-serial-4x": { + "Name": "pci-serial-4x", + "version_id": 1, + "minimum_version_id": 1, + "Description": { + "name": "pci-serial-multi", + "version_id": 1, + "minimum_version_id": 1, + "Fields": [ + { + "field": "dev", + "version_id": 0, + "field_exists": false, + "size": 1944, + "Description": { + "name": "PCIDevice", + "version_id": 2, + "minimum_version_id": 1, + "Fields": [ + { + "field": "version_id", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "config", + "version_id": 0, + "field_exists": false, + "size": 256 + }, + { + "field": "irq_state", + "version_id": 2, + "field_exists": false, + "size": 16 + } + ] + } + }, + { + "field": "state", + "version_id": 0, + "field_exists": false, + "size": 368, + "Description": { + "name": "serial", + "version_id": 3, + "minimum_version_id": 2, + "Fields": [ + { + "field": "divider", + "version_id": 2, + "field_exists": false, + "size": 2 + }, + { + "field": "rbr", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "ier", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "iir", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "lcr", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "mcr", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "lsr", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "msr", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "scr", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "fcr_vmstate", + "version_id": 3, + "field_exists": false, + "size": 1 + } + ] + } + }, + { + "field": "level", + "version_id": 0, + "field_exists": false, + "size": 4 + } + ] + } + }, + "intel-hda-generic": { + "Name": "intel-hda-generic", + "version_id": 1, + "minimum_version_id": 0, + "Description": { + "name": "intel-hda", + "version_id": 1, + "minimum_version_id": 0, + "Fields": [ + { + "field": "pci", + "version_id": 0, + "field_exists": false, + "size": 1944, + "Description": { + "name": "PCIDevice", + "version_id": 2, + "minimum_version_id": 1, + "Fields": [ + { + "field": "version_id", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "config", + "version_id": 0, + "field_exists": false, + "size": 256 + }, + { + "field": "irq_state", + "version_id": 2, + "field_exists": false, + "size": 16 + } + ] + } + }, + { + "field": "g_ctl", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "wake_en", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "state_sts", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "int_ctl", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "int_sts", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "wall_clk", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "corb_lbase", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "corb_ubase", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "corb_rp", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "corb_wp", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "corb_ctl", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "corb_sts", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "corb_size", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "rirb_lbase", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "rirb_ubase", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "rirb_wp", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "rirb_cnt", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "rirb_ctl", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "rirb_sts", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "rirb_size", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "dp_lbase", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "dp_ubase", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "icw", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "irr", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "ics", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "st", + "version_id": 0, + "field_exists": false, + "size": 56, + "Description": { + "name": "intel-hda-stream", + "version_id": 1, + "minimum_version_id": 0, + "Fields": [ + { + "field": "ctl", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "lpib", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "cbl", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "lvi", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "fmt", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "bdlp_lbase", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "bdlp_ubase", + "version_id": 0, + "field_exists": false, + "size": 4 + } + ] + } + }, + { + "field": "rirb_count", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "wall_base_ns", + "version_id": 0, + "field_exists": false, + "size": 8 + } + ] + } + }, + "cfi.pflash01": { + "Name": "cfi.pflash01", + "version_id": 1, + "minimum_version_id": 1, + "Description": { + "name": "pflash_cfi01", + "version_id": 1, + "minimum_version_id": 1, + "Fields": [ + { + "field": "wcycle", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "cmd", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "status", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "counter", + "version_id": 0, + "field_exists": false, + "size": 8 + } + ] + } + }, + "megasas": { + "Name": "megasas", + "version_id": 0, + "minimum_version_id": 0, + "Description": { + "name": "megasas", + "version_id": 0, + "minimum_version_id": 0, + "Fields": [ + { + "field": "parent_obj", + "version_id": 0, + "field_exists": false, + "size": 1944, + "Description": { + "name": "PCIDevice", + "version_id": 2, + "minimum_version_id": 1, + "Fields": [ + { + "field": "version_id", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "config", + "version_id": 0, + "field_exists": false, + "size": 256 + }, + { + "field": "irq_state", + "version_id": 2, + "field_exists": false, + "size": 16 + } + ] + } + }, + { + "field": "fw_state", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "intr_mask", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "doorbell", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "reply_queue_pa", + "version_id": 0, + "field_exists": false, + "size": 8 + }, + { + "field": "consumer_pa", + "version_id": 0, + "field_exists": false, + "size": 8 + }, + { + "field": "producer_pa", + "version_id": 0, + "field_exists": false, + "size": 8 + } + ] + } + }, + "PIIX3-xen": { + "Name": "PIIX3-xen", + "version_id": 3, + "minimum_version_id": 2, + "Description": { + "name": "PIIX3", + "version_id": 3, + "minimum_version_id": 2, + "Fields": [ + { + "field": "dev", + "version_id": 0, + "field_exists": false, + "size": 1944, + "Description": { + "name": "PCIDevice", + "version_id": 2, + "minimum_version_id": 1, + "Fields": [ + { + "field": "version_id", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "config", + "version_id": 0, + "field_exists": false, + "size": 256 + }, + { + "field": "irq_state", + "version_id": 2, + "field_exists": false, + "size": 16 + } + ] + } + }, + { + "field": "pci_irq_levels_vmstate", + "version_id": 3, + "field_exists": false, + "size": 4 + } + ], + "Subsections": [ + { + "name": "PIIX3/rcr", + "version_id": 1, + "minimum_version_id": 1, + "Fields": [ + { + "field": "rcr", + "version_id": 0, + "field_exists": false, + "size": 1 + } + ] + } + ] + } + }, + "tpci200": { + "Name": "tpci200", + "version_id": 1, + "minimum_version_id": 1, + "Description": { + "name": "tpci200", + "version_id": 1, + "minimum_version_id": 1, + "Fields": [ + { + "field": "dev", + "version_id": 0, + "field_exists": false, + "size": 1944, + "Description": { + "name": "PCIDevice", + "version_id": 2, + "minimum_version_id": 1, + "Fields": [ + { + "field": "version_id", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "config", + "version_id": 0, + "field_exists": false, + "size": 256 + }, + { + "field": "irq_state", + "version_id": 2, + "field_exists": false, + "size": 16 + } + ] + } + }, + { + "field": "big_endian", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "ctrl", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "status", + "version_id": 0, + "field_exists": false, + "size": 2 + }, + { + "field": "int_set", + "version_id": 0, + "field_exists": false, + "size": 1 + } + ] + } + }, + "SUNW,fdtwo": { + "Name": "SUNW,fdtwo", + "version_id": 2, + "minimum_version_id": 2, + "Description": { + "name": "fdc", + "version_id": 2, + "minimum_version_id": 2, + "Fields": [ + { + "field": "state", + "version_id": 0, + "field_exists": false, + "size": 360, + "Description": { + "name": "fdc", + "version_id": 2, + "minimum_version_id": 2, + "Fields": [ + { + "field": "sra", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "srb", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "dor_vmstate", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "tdr", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "dsr", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "msr", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "status0", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "status1", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "status2", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "fifo", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "data_pos", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "data_len", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "data_state", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "data_dir", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "eot", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "timer0", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "timer1", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "precomp_trk", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "config", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "lock", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "pwrd", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "num_floppies", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "drives", + "version_id": 1, + "field_exists": false, + "size": 40, + "Description": { + "name": "fdrive", + "version_id": 1, + "minimum_version_id": 1, + "Fields": [ + { + "field": "head", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "track", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "sect", + "version_id": 0, + "field_exists": false, + "size": 1 + } + ], + "Subsections": [ + { + "name": "fdrive/media_changed", + "version_id": 1, + "minimum_version_id": 1, + "Fields": [ + { + "field": "media_changed", + "version_id": 0, + "field_exists": false, + "size": 1 + } + ] + }, + { + "name": "fdrive/media_rate", + "version_id": 1, + "minimum_version_id": 1, + "Fields": [ + { + "field": "media_rate", + "version_id": 0, + "field_exists": false, + "size": 1 + } + ] + } + ] + } + } + ] + } + } + ] + } + }, + "usb-kbd": { + "Name": "usb-kbd", + "version_id": 1, + "minimum_version_id": 1, + "Description": { + "name": "usb-kbd", + "version_id": 1, + "minimum_version_id": 1, + "Fields": [ + { + "field": "dev", + "version_id": 0, + "field_exists": false, + "size": 4352, + "Description": { + "name": "USBDevice", + "version_id": 1, + "minimum_version_id": 1, + "Fields": [ + { + "field": "addr", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "state", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "remote_wakeup", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "setup_state", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "setup_len", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "setup_index", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "setup_buf", + "version_id": 0, + "field_exists": false, + "size": 1 + } + ] + } + }, + { + "field": "kbd.keycodes", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "head", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "n", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "kbd.modifiers", + "version_id": 0, + "field_exists": false, + "size": 2 + }, + { + "field": "kbd.leds", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "kbd.key", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "kbd.keys", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "protocol", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "idle", + "version_id": 0, + "field_exists": false, + "size": 1 + } + ] + } + } +}
tests/vmstate-static-checker-data/dump2.json new file +1163 -0
1,163 lines not on this page · raw message
Signature · 1 line
1.9.0
PATCH 4/18 tests: vmstate static checker: incompat machine types
In reply to Amit ShahThis commit modifies the dump2 data to flag incompatibilities in the machine types being compared.
--- tests/vmstate-static-checker-data/dump2.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/vmstate-static-checker-data/dump2.json +1 -1
@@ -1,6 +1,6 @@ { "vmschkmachine": { - "Name": "pc-i440fx-2.1" + "Name": "pc-i440fx-2.2" }, "fw_cfg": { "Name": "fw_cfg",
Signature · 1 line
1.9.0
PATCH 5/18 tests: vmstate static checker: add version error in main section
In reply to Amit Shah--- tests/vmstate-static-checker-data/dump2.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/vmstate-static-checker-data/dump2.json +1 -1
@@ -34,7 +34,7 @@ }, "fusbh200-ehci-usb": { "Name": "fusbh200-ehci-usb", - "version_id": 2, + "version_id": 1, "minimum_version_id": 1, "Description": { "name": "ehci-sysbus",
Signature · 1 line
1.9.0
PATCH 6/18 tests: vmstate static checker: version mismatch inside a Description
In reply to Amit Shah--- tests/vmstate-static-checker-data/dump2.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/vmstate-static-checker-data/dump2.json +1 -1
@@ -829,7 +829,7 @@ "minimum_version_id": 2, "Description": { "name": "fdc", - "version_id": 2, + "version_id": 1, "minimum_version_id": 2, "Fields": [ {
Signature · 1 line
1.9.0
PATCH 7/18 tests: vmstate static checker: minimum_version_id check
In reply to Amit Shah--- tests/vmstate-static-checker-data/dump1.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/vmstate-static-checker-data/dump1.json +1 -1
@@ -698,7 +698,7 @@ "minimum_version_id": 2, "Description": { "name": "PIIX3", - "version_id": 3, + "version_id": 1, "minimum_version_id": 2, "Fields": [ {
Signature · 1 line
1.9.0
PATCH 8/18 tests: vmstate static checker: remove a section
In reply to Amit Shah--- tests/vmstate-static-checker-data/dump2.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/vmstate-static-checker-data/dump2.json +1 -1
@@ -2,7 +2,7 @@ "vmschkmachine": { "Name": "pc-i440fx-2.2" }, - "fw_cfg": { + "fw_cfg2": { "Name": "fw_cfg", "version_id": 2, "minimum_version_id": 1,
Signature · 1 line
1.9.0
PATCH 9/18 tests: vmstate static checker: remove a field
In reply to Amit Shah--- tests/vmstate-static-checker-data/dump2.json | 6 ------ 1 file changed, 6 deletions(-)
tests/vmstate-static-checker-data/dump2.json +0 -6
@@ -58,12 +58,6 @@ "size": 4 }, { - "field": "usbsts", - "version_id": 0, - "field_exists": false, - "size": 4 - }, - { "field": "usbsts_pending", "version_id": 2, "field_exists": false,
Signature · 1 line
1.9.0
PATCH 10/18 tests: vmstate static checker: remove last field in a struct
In reply to Amit Shah--- tests/vmstate-static-checker-data/dump2.json | 6 ------ 1 file changed, 6 deletions(-)
tests/vmstate-static-checker-data/dump2.json +0 -6
@@ -637,12 +637,6 @@ "version_id": 0, "field_exists": false, "size": 256 - }, - { - "field": "irq_state", - "version_id": 2, - "field_exists": false, - "size": 16 } ] }
Signature · 1 line
1.9.0
PATCH 11/18 tests: vmstate static checker: change description name
In reply to Amit Shah--- tests/vmstate-static-checker-data/dump2.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/vmstate-static-checker-data/dump2.json +1 -1
@@ -749,7 +749,7 @@ "version_id": 1, "minimum_version_id": 1, "Description": { - "name": "tpci200", + "name": "tpci2002", "version_id": 1, "minimum_version_id": 1, "Fields": [
Signature · 1 line
1.9.0
PATCH 12/18 tests: vmstate static checker: remove Fields
In reply to Amit Shah--- tests/vmstate-static-checker-data/dump2.json | 114 +-------------------------- 1 file changed, 1 insertion(+), 113 deletions(-)
tests/vmstate-static-checker-data/dump2.json +1 -113
@@ -196,119 +196,7 @@ "Description": { "name": "pci-serial-multi", "version_id": 1, - "minimum_version_id": 1, - "Fields": [ - { - "field": "dev", - "version_id": 0, - "field_exists": false, - "size": 1944, - "Description": { - "name": "PCIDevice", - "version_id": 2, - "minimum_version_id": 1, - "Fields": [ - { - "field": "version_id", - "version_id": 0, - "field_exists": false, - "size": 4 - }, - { - "field": "config", - "version_id": 0, - "field_exists": false, - "size": 256 - }, - { - "field": "irq_state", - "version_id": 2, - "field_exists": false, - "size": 16 - } - ] - } - }, - { - "field": "state", - "version_id": 0, - "field_exists": false, - "size": 368, - "Description": { - "name": "serial", - "version_id": 3, - "minimum_version_id": 2, - "Fields": [ - { - "field": "divider", - "version_id": 2, - "field_exists": false, - "size": 2 - }, - { - "field": "rbr", - "version_id": 0, - "field_exists": false, - "size": 1 - }, - { - "field": "ier", - "version_id": 0, - "field_exists": false, - "size": 1 - }, - { - "field": "iir", - "version_id": 0, - "field_exists": false, - "size": 1 - }, - { - "field": "lcr", - "version_id": 0, - "field_exists": false, - "size": 1 - }, - { - "field": "mcr", - "version_id": 0, - "field_exists": false, - "size": 1 - }, - { - "field": "lsr", - "version_id": 0, - "field_exists": false, - "size": 1 - }, - { - "field": "msr", - "version_id": 0, - "field_exists": false, - "size": 1 - }, - { - "field": "scr", - "version_id": 0, - "field_exists": false, - "size": 1 - }, - { - "field": "fcr_vmstate", - "version_id": 3, - "field_exists": false, - "size": 1 - } - ] - } - }, - { - "field": "level", - "version_id": 0, - "field_exists": false, - "size": 4 - } - ] + "minimum_version_id": 1 } }, "intel-hda-generic": {
Signature · 1 line
1.9.0
PATCH 13/18 tests: vmstate static checker: remove Description
In reply to Amit Shah--- tests/vmstate-static-checker-data/dump2.json | 33 +--------------------------- 1 file changed, 1 insertion(+), 32 deletions(-)
tests/vmstate-static-checker-data/dump2.json +1 -32
@@ -462,38 +462,7 @@ "cfi.pflash01": { "Name": "cfi.pflash01", "version_id": 1, - "minimum_version_id": 1, - "Description": { - "name": "pflash_cfi01", - "version_id": 1, - "minimum_version_id": 1, - "Fields": [ - { - "field": "wcycle", - "version_id": 0, - "field_exists": false, - "size": 1 - }, - { - "field": "cmd", - "version_id": 0, - "field_exists": false, - "size": 1 - }, - { - "field": "status", - "version_id": 0, - "field_exists": false, - "size": 1 - }, - { - "field": "counter", - "version_id": 0, - "field_exists": false, - "size": 8 - } - ] - } + "minimum_version_id": 1 }, "megasas": { "Name": "megasas",
Signature · 1 line
1.9.0
PATCH 14/18 tests: vmstate static checker: remove Description inside Fields
In reply to Amit Shah--- tests/vmstate-static-checker-data/dump2.json | 27 +-------------------------- 1 file changed, 1 insertion(+), 26 deletions(-)
tests/vmstate-static-checker-data/dump2.json +1 -26
@@ -212,32 +212,7 @@ "field": "pci", "version_id": 0, "field_exists": false, - "size": 1944, - "Description": { - "name": "PCIDevice", - "version_id": 2, - "minimum_version_id": 1, - "Fields": [ - { - "field": "version_id", - "version_id": 0, - "field_exists": false, - "size": 4 - }, - { - "field": "config", - "version_id": 0, - "field_exists": false, - "size": 256 - }, - { - "field": "irq_state", - "version_id": 2, - "field_exists": false, - "size": 16 - } - ] - } + "size": 1944 }, { "field": "g_ctl",
Signature · 1 line
1.9.0
PATCH 15/18 tests: vmstate static checker: remove a subsection
In reply to Amit Shah--- tests/vmstate-static-checker-data/dump2.json | 13 ------------- 1 file changed, 13 deletions(-)
tests/vmstate-static-checker-data/dump2.json +0 -13
@@ -836,19 +836,6 @@ "size": 1 } ] - }, - { - "name": "fdrive/media_rate", - "version_id": 1, - "minimum_version_id": 1, - "Fields": [ - { - "field": "media_rate", - "version_id": 0, - "field_exists": false, - "size": 1 - } - ] } ] }
Signature · 1 line
1.9.0
PATCH 16/18 tests: vmstate static checker: remove Subsections
In reply to Amit Shah--- tests/vmstate-static-checker-data/dump2.json | 15 --------------- 1 file changed, 15 deletions(-)
tests/vmstate-static-checker-data/dump2.json +0 -15
@@ -558,21 +558,6 @@ "field_exists": false, "size": 4 } - ], - "Subsections": [ - { - "name": "PIIX3/rcr", - "version_id": 1, - "minimum_version_id": 1, - "Fields": [ - { - "field": "rcr", - "version_id": 0, - "field_exists": false, - "size": 1 - } - ] - } ] } },
Signature · 1 line
1.9.0
PATCH 17/18 tests: vmstate static checker: add substructure for usb-kbd for hid section
In reply to Amit ShahThis shows how the script deals with substructures added to vmstate descriptions that don't change the on-wire format.
--- tests/vmstate-static-checker-data/dump2.json | 115 +++++++++++++++------------ 1 file changed, 64 insertions(+), 51 deletions(-)
tests/vmstate-static-checker-data/dump2.json +64 -51
@@ -844,7 +844,7 @@ "field": "dev", "version_id": 0, "field_exists": false, - "size": 4352, + "size": 5832, "Description": { "name": "USBDevice", "version_id": 1,
@@ -896,58 +896,71 @@ } }, { - "field": "kbd.keycodes", + "field": "hid", "version_id": 0, "field_exists": false, - "size": 4 - }, - { - "field": "head", - "version_id": 0, - "field_exists": false, - "size": 4 - }, - { - "field": "n", - "version_id": 0, - "field_exists": false, - "size": 4 - }, - { - "field": "kbd.modifiers", - "version_id": 0, - "field_exists": false, - "size": 2 - }, - { - "field": "kbd.leds", - "version_id": 0, - "field_exists": false, - "size": 1 - }, - { - "field": "kbd.key", - "version_id": 0, - "field_exists": false, - "size": 1 - }, - { - "field": "kbd.keys", - "version_id": 0, - "field_exists": false, - "size": 4 - }, - { - "field": "protocol", - "version_id": 0, - "field_exists": false, - "size": 4 - }, - { - "field": "idle", - "version_id": 0, - "field_exists": false, - "size": 1 + "size": 312, + "Description": { + "name": "HIDKeyboardDevice", + "version_id": 1, + "minimum_version_id": 1, + "Fields": [ + { + "field": "kbd.keycodes", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "head", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "n", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "kbd.modifiers", + "version_id": 0, + "field_exists": false, + "size": 2 + }, + { + "field": "kbd.leds", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "kbd.key", + "version_id": 0, + "field_exists": false, + "size": 1 + }, + { + "field": "kbd.keys", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "protocol", + "version_id": 0, + "field_exists": false, + "size": 4 + }, + { + "field": "idle", + "version_id": 0, + "field_exists": false, + "size": 1 + } + ] + } } ] }
Signature · 1 line
1.9.0
PATCH 18/18 tests: vmstate static checker: add size mismatch inside substructure
In reply to Amit Shah--- tests/vmstate-static-checker-data/dump2.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/vmstate-static-checker-data/dump2.json +1 -1
@@ -909,7 +909,7 @@ "field": "kbd.keycodes", "version_id": 0, "field_exists": false, - "size": 4 + "size": 2 }, { "field": "head",
Signature · 1 line
1.9.0