Skip to contentSkip to conversation

vmsd checks

PATCH In review Dr. David Alan Gilbert (git) 4 participants 17 messages Jan 13, 2022 – Feb 2, 2022

lore
Share & tools
lore
#1 lore raw

PATCH v2 1/3 ppc: Fix vmstate_pbr403 subsection name

In reply to Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

The pbr403 subsection is part of the tlb6xx state, so I believe it's name needs to be:

    .name = "cpu/tlb6xx/pbr403",
---
 target/ppc/machine.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
target/ppc/machine.c +1 -1
index 756d8de5d8..e535edb7c4 100644
@@ -718,7 +718,7 @@ static bool pbr403_needed(void *opaque)
 }
 
 static const VMStateDescription vmstate_pbr403 = {
-    .name = "cpu/pbr403",
+    .name = "cpu/tlb6xx/pbr403",
     .version_id = 1,
     .minimum_version_id = 1,
     .needed = pbr403_needed,
Signature · 1 line
2.34.1
#9 lore raw

PATCH v2 3/3 migration: Perform vmsd structure check during tests

In reply to Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Perform a check on vmsd structures during test runs in the hope of catching any missing terminators and other simple screwups.

---
 migration/savevm.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
migration/savevm.c +39 -0
index 8077393d11..97a4471220 100644
@@ -66,6 +66,7 @@
 #include "net/announce.h"
 #include "qemu/yank.h"
 #include "yank_functions.h"
+#include "sysemu/qtest.h"
 
 const unsigned int postcopy_ram_discard_version;
 
@@ -839,6 +840,39 @@ void unregister_savevm(VMStateIf *obj, const char *idstr, void *opaque)
     }
 }
 
+/*
+ * Perform some basic checks on vmsd's at registration
+ * time.
+ */
+static void vmstate_check(const VMStateDescription *vmsd)
+{
+    const VMStateField *field = vmsd->fields;
+    const VMStateDescription **subsection = vmsd->subsections;
+
+    if (field) {
+        while (field->name) {
+            if (field->flags & (VMS_STRUCT | VMS_VSTRUCT)) {
+                /* Recurse to sub structures */
+                vmstate_check(field->vmsd);
+            }
+            /* Carry on */
+            field++;
+        }
+        /* Check for the end of field list canary */
+        assert(field->flags == VMS_END);
+    }
+
+    while (subsection && *subsection) {
+        /*
+         * The name of a subsection should start with the name of the
+         * current object.
+         */
+        assert(!strncmp(vmsd->name, (*subsection)->name, strlen(vmsd->name)));
+        vmstate_check(*subsection);
+        subsection++;
+    }
+}
+
 int vmstate_register_with_alias_id(VMStateIf *obj, uint32_t instance_id,
                                    const VMStateDescription *vmsd,
                                    void *opaque, int alias_id,
@@ -884,6 +918,11 @@ int vmstate_register_with_alias_id(VMStateIf *obj, uint32_t instance_id,
     } else {
         se->instance_id = instance_id;
     }
+
+    /* Perform a recursive sanity check during the test runs */
+    if (qtest_enabled()) {
+        vmstate_check(vmsd);
+    }
     assert(!se->compat || se->instance_id == 0);
     savevm_state_handler_insert(se);
     return 0;
Signature · 1 line
2.34.1
#0 lore raw

v2 cover vmsd checks

From: "Dr. David Alan Gilbert" <[email protected]>

Aftern the patch the other day where I added a missing END_OF_LIST, Peter suggested adding something more robust.

Here I:
add a check for a flag at the end of the list
add a check that's performed in vmstate_register_with_alias_id
only within qtest recursively for that canary and for subsection naming constraints.
Fix a ppc issue that the vmstate naming constraint caught
(Waiting for a reply from the PPC folk to check that).

The checks can't go in until I get the def into libslirp.

Dr. David Alan Gilbert (3):
ppc: Fix vmstate_pbr403 subsection name
migration: Add canary to VMSTATE_END_OF_LIST
migration: Perform vmsd structure check during tests

 include/migration/vmstate.h |  7 ++++++-
 migration/savevm.c          | 40 +++++++++++++++++++++++++++++++++++++
 migration/vmstate.c         |  2 ++
 target/ppc/machine.c        |  2 +-
 4 files changed, 49 insertions(+), 2 deletions(-)
Signature · 1 line
2.34.1
#5 lore raw

PATCH v2 2/3 migration: Add canary to VMSTATE_END_OF_LIST

From: "Dr. David Alan Gilbert" <[email protected]>

We fairly regularly forget VMSTATE_END_OF_LIST markers off descriptions; given that the current check is only for ->name being NULL, sometimes we get unlucky and the code apparently works and no one spots the error.

Explicitly add a flag, VMS_END that should be set, and assert it is set during the traversal.

Note: This can't go in until we update the copy of vmstate.h in slirp.

---
 include/migration/vmstate.h | 7 ++++++-
 migration/savevm.c          | 1 +
 migration/vmstate.c         | 2 ++
 3 files changed, 9 insertions(+), 1 deletion(-)
include/migration/vmstate.h +6 -1
index 017c03675c..b50708e57a 100644
@@ -147,6 +147,9 @@ enum VMStateFlags {
      * VMStateField.struct_version_id to tell which version of the
      * structure we are referencing to use. */
     VMS_VSTRUCT           = 0x8000,
+
+    /* Marker for end of list */
+    VMS_END = 0x10000
 };
 
 typedef enum {
@@ -1163,7 +1166,9 @@ extern const VMStateInfo vmstate_info_qlist;
     VMSTATE_UNUSED_BUFFER(_test, 0, _size)
 
 #define VMSTATE_END_OF_LIST()                                         \
-    {}
+    {                     \
+        .flags = VMS_END, \
+    }
 
 int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd,
                        void *opaque, int version_id);
migration/savevm.c +1 -0
index 0bef031acb..8077393d11 100644
@@ -620,6 +620,7 @@ static void dump_vmstate_vmsd(FILE *out_file,
             field++;
             first = false;
         }
+        assert(field->flags == VMS_END);
         fprintf(out_file, "\n%*s]", indent, "");
     }
     if (vmsd->subsections != NULL) {
migration/vmstate.c +2 -0
index 05f87cdddc..181ba08c7d 100644
@@ -160,6 +160,7 @@ int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd,
         }
         field++;
     }
+    assert(field->flags == VMS_END);
     ret = vmstate_subsection_load(f, vmsd, opaque);
     if (ret != 0) {
         return ret;
@@ -413,6 +414,7 @@ int vmstate_save_state_v(QEMUFile *f, const VMStateDescription *vmsd,
         }
         field++;
     }
+    assert(field->flags == VMS_END);
 
     if (vmdesc) {
         json_writer_end_array(vmdesc);
Signature · 1 line
2.34.1
Reviewed-by on v2 2/3
#6 lore raw

Re: [PATCH v2 2/3] migration: Add canary to VMSTATE_END_OF_LIST

Quoted 7 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

We fairly regularly forget VMSTATE_END_OF_LIST markers off descriptions; given that the current check is only for ->name being NULL, sometimes we get unlucky and the code apparently works and no one spots the error.

Quoted 8 lines

Explicitly add a flag, VMS_END that should be set, and assert it is set during the traversal.

Note: This can't go in until we update the copy of vmstate.h in slirp.

---

thanks
-- PMM

Reviewed-by on v2 1/3
#2 lore raw

Re: [PATCH v2 1/3] ppc: Fix vmstate_pbr403 subsection name

In reply to Dr. David Alan Gilbert (git)
Quoted 27 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

The pbr403 subsection is part of the tlb6xx state, so I believe it's name needs to be:

    .name = "cpu/tlb6xx/pbr403",
---
 target/ppc/machine.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
target/ppc/machine.c +1 -2
index 756d8de5d8..e535edb7c4 100644
@@ -718,7 +718,7 @@ static bool pbr403_needed(void *opaque)
 }
 
 static const VMStateDescription vmstate_pbr403 = {
-    .name = "cpu/pbr403",
+    .name = "cpu/tlb6xx/pbr403",
     .version_id = 1,
     .minimum_version_id = 1,
     .needed = pbr403_needed,
--
Quoted 1 line

2.34.1

thanks
-- PMM

Reviewed-by on v2 3/3
#10 lore raw

Re: [PATCH v2 3/3] migration: Perform vmsd structure check during tests

In reply to Dr. David Alan Gilbert (git)
Quoted 10 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Perform a check on vmsd structures during test runs in the hope of catching any missing terminators and other simple screwups.

---
 migration/savevm.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

thanks
-- PMM

Reviewed-by Philippe Mathieu-Daudé on v2 2/3
#7 lore raw

Re: [PATCH v2 2/3] migration: Add canary to VMSTATE_END_OF_LIST

In reply to Dr. David Alan Gilbert (git)
Quoted 10 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

We fairly regularly forget VMSTATE_END_OF_LIST markers off descriptions; given that the current check is only for ->name being NULL, sometimes we get unlucky and the code apparently works and no one spots the error.

Explicitly add a flag, VMS_END that should be set, and assert it is set during the traversal.

Note: This can't go in until we update the copy of vmstate.h in slirp.

Do we need a libslirp buildsys version check to get this patch merged?

Quoted 7 lines
---
  include/migration/vmstate.h | 7 ++++++-
  migration/savevm.c          | 1 +
  migration/vmstate.c         | 2 ++
  3 files changed, 9 insertions(+), 1 deletion(-)
#8 lore raw

Re: [PATCH v2 2/3] migration: Add canary to VMSTATE_END_OF_LIST

Quoted 7 lines · Philippe Mathieu-Daudé
Quoted 6 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

We fairly regularly forget VMSTATE_END_OF_LIST markers off descriptions; given that the current check is only for ->name being NULL, sometimes we get unlucky and the code apparently works and no one spots the error.

Quoted 6 lines
Quoted 4 lines

Explicitly add a flag, VMS_END that should be set, and assert it is set during the traversal.

Note: This can't go in until we update the copy of vmstate.h in slirp.

Do we need a libslirp buildsys version check to get this patch merged?

In that case we should use an intermediate function which would eventually call assert() after checking SLIRP_MAJOR_VERSION/... values.

Quoted 9 lines
Quoted 7 lines

---
  include/migration/vmstate.h | 7 ++++++-
  migration/savevm.c          | 1 +
  migration/vmstate.c         | 2 ++

  3 files changed, 9 insertions(+), 1 deletion(-)
#3 lore raw

Re: [PATCH v2 1/3] ppc: Fix vmstate_pbr403 subsection name

In reply to Peter Maydell
Quoted 22 lines · * Peter Maydell ([email protected]) wrote:
Quoted 20 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

The pbr403 subsection is part of the tlb6xx state, so I believe it's name needs to be:

    .name = "cpu/tlb6xx/pbr403",
---
 target/ppc/machine.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
target/ppc/machine.c +0 -0
index 756d8de5d8..e535edb7c4 100644
@@ -718,7 +718,7 @@ static bool pbr403_needed(void *opaque)
 }
 
Quoted 10 lines
Quoted 8 lines
target/ppc/machine.c +1 -2
 static const VMStateDescription vmstate_pbr403 = {
-    .name = "cpu/pbr403",
+    .name = "cpu/tlb6xx/pbr403",
     .version_id = 1,
     .minimum_version_id = 1,
     .needed = pbr403_needed,
--

2.34.1

Actually, we'll drop this one; Cédric Le Goater has just published 'target/ppc: Finish removal of 401/403 CPUs' that kills this code off.

Dave

Quoted 3 lines

thanks
-- PMM

Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#4 lore raw

Re: [PATCH v2 1/3] ppc: Fix vmstate_pbr403 subsection name

In reply to Dr. David Alan Gilbert (git)
Quoted 8 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

The pbr403 subsection is part of the tlb6xx state, so I believe it's name needs to be:

    .name = "cpu/tlb6xx/pbr403",

Dropped as code don't exist anymore.

Quoted 17 lines
---
 target/ppc/machine.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
target/ppc/machine.c +1 -1
index 756d8de5d8..e535edb7c4 100644
@@ -718,7 +718,7 @@ static bool pbr403_needed(void *opaque)
 }
 
 static const VMStateDescription vmstate_pbr403 = {
-    .name = "cpu/pbr403",
+    .name = "cpu/tlb6xx/pbr403",
     .version_id = 1,
     .minimum_version_id = 1,
     .needed = pbr403_needed,
Reviewed-by on v2 3/3
#11 lore raw

Re: [PATCH v2 3/3] migration: Perform vmsd structure check during tests

In reply to Dr. David Alan Gilbert (git)
Quoted 6 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Perform a check on vmsd structures during test runs in the hope of catching any missing terminators and other simple screwups.

queued.

#12 lore raw

Re: [PATCH v2 3/3] migration: Perform vmsd structure check during tests

Quoted 11 lines · * Juan Quintela ([email protected]) wrote:
Quoted 6 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Perform a check on vmsd structures during test runs in the hope of catching any missing terminators and other simple screwups.

queued.

Careful; I think that'll break with slirp until libslirp gets updated first.

Dave

Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#13 lore raw

Re: [PATCH v2 3/3] migration: Perform vmsd structure check during tests

Quoted 7 lines · Dr. David Alan Gilbert
Quoted 6 lines · * Juan Quintela ([email protected]) wrote:
Quoted 5 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Perform a check on vmsd structures during test runs in the hope of catching any missing terminators and other simple screwups.

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

queued.

Careful; I think that'll break with slirp until libslirp gets updated first.

As expected, it broke it.

I resend the PULL request wihtout that two patches.

Once that we are here, how it is that make check didn't catch this?

Later, Juan.

#14 lore raw

Re: [PATCH v2 3/3] migration: Perform vmsd structure check during tests

Quoted 14 lines · * Juan Quintela ([email protected]) wrote:
Quoted 13 lines · Dr. David Alan Gilbert
Quoted 11 lines · * Juan Quintela ([email protected]) wrote:
Quoted 6 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Perform a check on vmsd structures during test runs in the hope of catching any missing terminators and other simple screwups.

queued.

Quoted 8 lines
Quoted 2 lines

Careful; I think that'll break with slirp until libslirp gets updated first.

As expected, it broke it.

I resend the PULL request wihtout that two patches.

Once that we are here, how it is that make check didn't catch this?

Because in my local world I did the changes to libslirp; I wanted to make sure qemu people were happy with the changes before proposing them to libslirp.

Which I've just done:

https://gitlab.freedesktop.org/slirp/libslirp/-/merge_requests/112

Dave

Quoted 2 lines

Later, Juan.

Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#15 lore raw

Re: [PATCH v2 3/3] migration: Perform vmsd structure check during tests

Quoted 24 lines · Dr. David Alan Gilbert
Quoted 14 lines · * Juan Quintela ([email protected]) wrote:
Quoted 13 lines · Dr. David Alan Gilbert
Quoted 11 lines · * Juan Quintela ([email protected]) wrote:
Quoted 6 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Perform a check on vmsd structures during test runs in the hope of catching any missing terminators and other simple screwups.

queued.

Quoted 8 lines
Quoted 2 lines

Careful; I think that'll break with slirp until libslirp gets updated first.

As expected, it broke it.

I resend the PULL request wihtout that two patches.

Once that we are here, how it is that make check didn't catch this?

Quoted 7 lines

Because in my local world I did the changes to libslirp; I wanted to make sure qemu people were happy with the changes before proposing them to libslirp.

Which I've just done:

https://gitlab.freedesktop.org/slirp/libslirp/-/merge_requests/112

I mean make check.

It worked for me on my PULL request. I would have assumed that it checks slirp.

Later, Juan.

#16 lore raw

Re: [PATCH v2 3/3] migration: Perform vmsd structure check during tests

In reply to Dr. David Alan Gilbert
Quoted 7 lines · Dr. David Alan Gilbert

Because in my local world I did the changes to libslirp; I wanted to make sure qemu people were happy with the changes before proposing them to libslirp.

Which I've just done:

https://gitlab.freedesktop.org/slirp/libslirp/-/merge_requests/112

Does QEMU's own vmstate handling code see the libslirp vmstate structures ? Looking at the code it seems to me like QEMU's migration code only interacts with slirp via the slirp_state_save() and slirp_state_load() functions. Internally those work with some use of a vmstate structure, but the code that iterates over field arrays in those is all inside slirp itself (in src/slirp/vmstate.c if you're looking at the in-tree copy).

So maybe I'm missing something but I'm not sure there really is a dependency on the libslirp change here...

-- PMM