Received: from eggs.gnu.org ([2001:4830:134:3::10]:53254) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cg9yO-0001AC-3Y for qemu-devel@nongnu.org; Tue, 21 Feb 2017 07:55:18 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cg9yK-00054J-Uy for qemu-devel@nongnu.org; Tue, 21 Feb 2017 07:55:16 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:40314 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cg9yK-00054A-ON for qemu-devel@nongnu.org; Tue, 21 Feb 2017 07:55:12 -0500 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v1LCrR5H133193 for ; Tue, 21 Feb 2017 07:55:11 -0500 Received: from e06smtp13.uk.ibm.com (e06smtp13.uk.ibm.com [195.75.94.109]) by mx0b-001b2d01.pphosted.com with ESMTP id 28rku6dr1f-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 21 Feb 2017 07:55:11 -0500 Received: from localhost by e06smtp13.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 21 Feb 2017 12:55:09 -0000 References: <20170216121140.9061-1-pasic@linux.vnet.ibm.com> <20170216121140.9061-4-pasic@linux.vnet.ibm.com> <20170217194237.GA20911@work-vm> <56bb4f43-9dc7-83d5-7cb9-d3e8036675e6@linux.vnet.ibm.com> <20170221122207.GF2377@work-vm> From: Halil Pasic Date: Tue, 21 Feb 2017 13:55:06 +0100 MIME-Version: 1.0 In-Reply-To: <20170221122207.GF2377@work-vm> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Message-Id: <6b39cefc-16b2-4a77-9efc-400a73fff88d@linux.vnet.ibm.com> Subject: Re: [Qemu-devel] [PATCH 3/5] migration/vmstate: fix array of ptr with nullptrs List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Dr. David Alan Gilbert" Cc: Amit Shah , qemu-devel@nongnu.org, Juan Quintela On 02/21/2017 01:22 PM, Dr. David Alan Gilbert wrote: > * Halil Pasic (pasic@linux.vnet.ibm.com) wrote: >> >> >> On 02/17/2017 08:42 PM, Dr. David Alan Gilbert wrote: >>> * Halil Pasic (pasic@linux.vnet.ibm.com) wrote: >>>> Make VMS_ARRAY_OF_POINTER cope with null pointers. Previously the >>>> reward for trying to migrate an array with some null pointers in it was >>>> an illegal memory access, that is a swift and painless death of the >>>> process. Let's make vmstate cope with this scenario. >>>> >>>> The general approach is, when we encounter a null pointer (element), >>>> instead of following the pointer to save/load the data behind it, we >>>> save/load a placeholder. This way we can detect if we expected a null >>>> pointer at the load side but not null data was saved instead. >>>> >>>> Signed-off-by: Halil Pasic >>>> Reviewed-by: Guenther Hutzl >>>> >>>> --- >>>> We will need this to load/save some on demand created state in the >>>> (s390x) channel subsystem (see ChannelSubSys.css in hw/s390x/css.c for >>>> an example). >>>> --- >>>> include/migration/vmstate.h | 4 ++++ >>>> migration/vmstate.c | 33 +++++++++++++++++++++++++++++++-- >>>> 2 files changed, 35 insertions(+), 2 deletions(-) >>>> >>>> diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h >>>> index 63e7b02..f2dbf84 100644 >>>> --- a/include/migration/vmstate.h >>>> +++ b/include/migration/vmstate.h >>>> @@ -253,6 +253,10 @@ extern const VMStateInfo vmstate_info_uint16; >>>> extern const VMStateInfo vmstate_info_uint32; >>>> extern const VMStateInfo vmstate_info_uint64; >>>> >>>> +/** Put this in the stream when migrating a null pointer.*/ >>>> +#define VMS_NULLPTR_MARKER (0x30U) /* '0' */ >>>> +extern const VMStateInfo vmstate_info_nullptr; >>>> + >>>> extern const VMStateInfo vmstate_info_float64; >>>> extern const VMStateInfo vmstate_info_cpudouble; >>>> >>>> diff --git a/migration/vmstate.c b/migration/vmstate.c >>>> index 836a7a4..cb81cef 100644 >>>> --- a/migration/vmstate.c >>>> +++ b/migration/vmstate.c >>>> @@ -117,7 +117,11 @@ int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd, >>>> if (field->flags & VMS_ARRAY_OF_POINTER) { >>>> curr_elem = *(void **)curr_elem; >>>> } >>>> - if (field->flags & VMS_STRUCT) { >>>> + if (!curr_elem) { >>>> + /* if null pointer check placeholder and do not follow */ >>>> + assert(field->flags & VMS_ARRAY_OF_POINTER); >>> >>> That can return an error instead of asserting. >>> >>>> + vmstate_info_nullptr.get(f, curr_elem, size, NULL); >>> >>> You've ignored the return value of the get; that should be ret = >>> >>>> + } else if (field->flags & VMS_STRUCT) { >>>> ret = vmstate_load_state(f, field->vmsd, curr_elem, >>>> field->vmsd->version_id); >>>> } else { >>>> @@ -332,7 +336,11 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd, >>>> assert(curr_elem); >>>> curr_elem = *(void **)curr_elem; >>>> } >>>> - if (field->flags & VMS_STRUCT) { >>>> + if (!curr_elem) { >>>> + /* if null pointer write placeholder and do not follow */ >>>> + assert(field->flags & VMS_ARRAY_OF_POINTER); >>>> + vmstate_info_nullptr.put(f, curr_elem, size, NULL, NULL); >>>> + } else if (field->flags & VMS_STRUCT) { >>>> vmstate_save_state(f, field->vmsd, curr_elem, vmdesc_loop); >>>> } else { >>>> field->info->put(f, curr_elem, size, field, vmdesc_loop); >>>> @@ -747,6 +755,27 @@ const VMStateInfo vmstate_info_uint64 = { >>>> .put = put_uint64, >>>> }; >>>> >>>> +static int get_nullptr(QEMUFile *f, void *pv, size_t size, VMStateField *field) >>>> + >>>> +{ >>>> + return qemu_get_byte(f) == VMS_NULLPTR_MARKER ? 0 : -EINVAL; >>>> +} >>>> + >>>> +static int put_nullptr(QEMUFile *f, void *pv, size_t size, >>>> + VMStateField *field, QJSON *vmdesc) >>>> + >>>> +{ >>>> + assert(pv == NULL); >>>> + qemu_put_byte(f, VMS_NULLPTR_MARKER); >>>> + return 0; >>>> +} >>> >>> Again that assert could just turn into a return -EINVAL >>> >>> Dave >>> >> >> @Dave: I have accidentally answered with 'reply' instead of 'reply all' >> yesterday. Sorry for the noise! >> >> Thanks for the review! You are right, my code is messy. > > It's not that bad - just a few minor issues. > >> Yet I'm not sure what you propose is the best way to clean it up. My >> concern is, that we are loosing some info about what exactly went wrong >> (and where), if returning -EINVAL is not combined with additional >> logging/error reporting. >> >> After re-checking the asserts they all indicate programming errors and >> there is not much (IMHO) client code can do to recover and the user >> should never observe. Unfortunately, I lack knowledge about how is the >> client code handling errors and if there is something we need to clean >> up. I assumed asserting is OK because of the per-existing asserts. > >> You are perfectly right about ignoring the return value of >> vmstate_info_nullptr.get and at least I will have to fix that. >> IMHO we have three options to fix this code: >> a) Make vmstate_info_nullptr.get assert too as we expect null pointer >> and we got not null is pretty much an indicator that we can't proceed >> sanely. Possibly add ret = ... for aesthetic reasons. Keep the asserts. >> b) Follow your suggestions without anything on top of it. >> c) Follow your suggestions and add error_report stating what exactly >> went wrong. >> >> My priority is to get this in, so I will do what you say, or send >> option b) late on Wednesday if no guidance until then. > > It's OK to add an error_report as needed; in particular I try and avoid > assert's on the save path where possible, since the user apparently > has a happy running VM, so even if the migration code has an error in it > I'd rather the user didn't lose their VM. The load path it's OK to > add the asserts since they've not got a working VM yet. > > Dave > You are right, distinguishing between save and load paths makes a lot of sense. So on the load path I will use asserts, and on the save path, that is static int put_nullptr(QEMUFile *f, void *pv, size_t size, VMStateField *field, QJSON *vmdesc) { here, I could do: - assert(pv == NULL); + if (pv != NULL) { + error_report("put_nullptr must be called with pv == NULL"); + return -EINVAL; + } qemu_put_byte(f, VMS_NULLPTR_MARKER); return 0; } And of course, I will fix ignoring the return value too. Would that be satisfactory? If not, please complain. By the way, I could also omit the check on pv, and ignore pv altogether. A call to put_nullptr means we encountered a nullptr and the check is just for catching buggy client code -- what might be an overkill in this case. Thank you very much. FYI I intend to send out the next version tomorrow. Regards, Halil