migration: add static analysis tool to check vmstate compat

Hello,

v5:
- Add section / description name checking for matching names for changed fields. This has the benefit of also getting rid of the for loop in the lookup. However, for each section, like piix4_pm, all the changes have to be in one group, as against them being in different ones. That's still better than earlier where sections were not checked. If my Python gets better, I might come up with a way to improve on this as well. (Juan Quintela)
- Add more comments to the Python script
- Update help text to be more verbose - for qemu and for the script.
- Simplify the check_fields() function - lesser indentations

v4:
- Return value capped to 255 to prevent wrap-around (Eric Blake)

v3:
- Python script returns an error code: 0 if no errors, positive for the number of errors identified.

v2:
- Tabs->spaces (Dave Gilbert)
- Several changes to the python script to make it more python-like (Vitaly Kuznetsov)
- Don't store the empty fields created by VMSTATE_VALIDATE in the json output

This series adds a static vmstate checker to check for breakage of live migration by analyzing the vmstate information between different QEMU versions.

In patch 1, QEMU is modified to add a -dump-vmstate commandline option, which takes a filename as the argument. When invoked, QEMU dumps the vmstate info in JSON format for the current machine type to the file. This patch is loosely based on a version from Andreas Färber.

This JSON file is then fed into the Python script introduced in patch
2. The script takes 'src' and 'dest' arguments, indicating the direction of migration. The script then performs a series of checks and spews out information on inconsistencies it finds in the data.

Two stripped-down versions of JSON dumps are included in this patchset (patch 3).

Patches 4 - 18 contain modifications to those dumps, to show the checks that are performed by the script. The result of running the script against the final version of those files is appended below.

The checks are to be performed for a particular machine type, and comparing different machine types is bound to turn up false-positives:
e.g.

(in a qemu 2.0 tree):
./x86_64-softmmu/qemu-system-x86_64 -dump-vmstate qemu-2.0.json

(in a qemu 2.2 tree:)
./x86_64-softmmu/qemu-system-x86_64 -dump-vmstate -M pc-i440fx-2.0 \
qemu-2.2-m2.0.json

./scripts/vmstate-static-checker.py -s qemu-2.0.json -d qemu-2.2-m2.0.json

should not show any output.

The idea is to include this script in 'make check', ensuring new commits don't break migration.

Later, this script can also be baked into the release process: vmstate dumps for released versions of qemu for various machine types can be stored in a directory in the tree. At the time of freezing the tree for releases, (like -rc2), the dump for the current release can be updated. A policy that says "No more live migration-breaking changes can be accepted post -rc2" can be put in place. Also, for checks for older machine types on the current tree with the saved dumps should not indicate any regressions.

I expect there to be a few false positives at the start (though there aren't any right now). The script will keep evolving, though, to include new scenarios, to make it more helpful.

A later project would be to also store other guest-visible information and use that output for more checks (including lspci output from inside guests).

As mentioned earlier, this is the result of running the script against the sample json data included in this patchset:

$ ./scripts/vmstate-static-checker.py --src ./tests/vmstate-static-checker-data/dump1.json  --dest ./tests/vmstate-static-checker-data/dump2.json 
Section "usb-kbd" Description "usb-kbd" Field "kbd.keycodes" size mismatch: 4 , 2
Section "PIIX3-xen" Description "PIIX3": minimum version error: 1 < 2
Section "PIIX3-xen" Description "PIIX3": Entry "Subsections" missing
Section "tpci200": Description "tpci200" missing, got "tpci2002" instead; skipping
Section "megasas", Description "PCIDevice": expected field "irq_state", while dest has no further fields
Section "SUNW,fdtwo" Description "fdc": version error: 2 > 1
Section "SUNW,fdtwo", Description "fdrive": Subsection "fdrive/media_rate" not found
Section "fusbh200-ehci-usb" version error: 2 > 1
Section "fusbh200-ehci-usb", Description "ehci-core": expected field "usbsts", got "usbsts_pending"; skipping rest
Section "intel-hda-generic", Description "intel-hda", Field "pci": missing description
Section "cfi.pflash01": Entry "Description" missing
Section "pci-serial-4x" Description "pci-serial-multi": Entry "Fields" missing
Warning: checking incompatible machine types: "pc-i440fx-2.1", "pc-i440fx-2.2"
Section "fw_cfg" does not exist in dest

$ echo $?
13

Amit Shah (18):
migration: dump vmstate info as a json file for static analysis
vmstate-static-checker: script to validate vmstate changes
tests: vmstate static checker: add dump1 and dump2 files
tests: vmstate static checker: incompat machine types
tests: vmstate static checker: add version error in main section
tests: vmstate static checker: version mismatch inside a Description
tests: vmstate static checker: minimum_version_id check
tests: vmstate static checker: remove a section
tests: vmstate static checker: remove a field
tests: vmstate static checker: remove last field in a struct
tests: vmstate static checker: change description name
tests: vmstate static checker: remove Fields
tests: vmstate static checker: remove Description
tests: vmstate static checker: remove Description inside Fields
tests: vmstate static checker: remove a subsection
tests: vmstate static checker: remove Subsections
tests: vmstate static checker: add substructure for usb-kbd for hid
section
tests: vmstate static checker: add size mismatch inside substructure

 include/migration/vmstate.h                  |    2 +
 qemu-options.hx                              |   14 +
 savevm.c                                     |  139 +++
 scripts/vmstate-static-checker.py            |  345 ++++++++
 tests/vmstate-static-checker-data/dump1.json | 1163 ++++++++++++++++++++++++++
 tests/vmstate-static-checker-data/dump2.json |  968 +++++++++++++++++++++
 vl.c                                         |   13 +
 7 files changed, 2644 insertions(+)
 create mode 100755 scripts/vmstate-static-checker.py
 create mode 100644 tests/vmstate-static-checker-data/dump1.json
 create mode 100644 tests/vmstate-static-checker-data/dump2.json

1.9.3