Received: from eggs.gnu.org ([2001:4830:134:3::10]:52730) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xluge-0000wh-Fa for qemu-devel@nongnu.org; Wed, 05 Nov 2014 02:07:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XlugX-00071U-2S for qemu-devel@nongnu.org; Wed, 05 Nov 2014 02:07:24 -0500 Received: from ozlabs.org ([103.22.144.67]:33943) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XlugW-000706-9C for qemu-devel@nongnu.org; Wed, 05 Nov 2014 02:07:16 -0500 Date: Wed, 5 Nov 2014 17:47:29 +1100 From: David Gibson Message-ID: <20141105064729.GB26255@voom.redhat.com> References: <1412358473-31398-1-git-send-email-dgilbert@redhat.com> <1412358473-31398-32-git-send-email-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LyciRD1jyfeSSjG0" Content-Disposition: inline In-Reply-To: <1412358473-31398-32-git-send-email-dgilbert@redhat.com> Subject: Re: [Qemu-devel] [PATCH v4 31/47] postcopy: Incoming initialisation List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Dr. David Alan Gilbert (git)" Cc: aarcange@redhat.com, yamahata@private.email.ne.jp, lilei@linux.vnet.ibm.com, quintela@redhat.com, cristian.klein@cs.umu.se, qemu-devel@nongnu.org, amit.shah@redhat.com, yanghy@cn.fujitsu.com --LyciRD1jyfeSSjG0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Oct 03, 2014 at 06:47:37PM +0100, Dr. David Alan Gilbert (git) wrot= e: > From: "Dr. David Alan Gilbert" >=20 > Signed-off-by: Dr. David Alan Gilbert > --- > arch_init.c | 11 ++++ > include/migration/migration.h | 1 + > include/migration/postcopy-ram.h | 12 +++++ > migration.c | 1 + > postcopy-ram.c | 110 +++++++++++++++++++++++++++++++++= +++++- > savevm.c | 4 ++ > 6 files changed, 138 insertions(+), 1 deletion(-) >=20 > diff --git a/arch_init.c b/arch_init.c > index 030d189..4a03171 100644 > --- a/arch_init.c > +++ b/arch_init.c > @@ -1345,6 +1345,17 @@ void ram_handle_compressed(void *host, uint8_t ch,= uint64_t size) > } > } > =20 > +/* > + * Allocate data structures etc needed by incoming migration with postco= py-ram > + * postcopy-ram's similarly names postcopy_ram_incoming_init does the wo= rk > + */ > +int ram_postcopy_incoming_init(MigrationIncomingState *mis) > +{ > + size_t ram_pages =3D last_ram_offset() >> TARGET_PAGE_BITS; > + > + return postcopy_ram_incoming_init(mis, ram_pages); > +} Um.. yeah. I'm sure ram_postcopy_incoming_init versus postcopy_ram_incoming_init won't get confusing o_O. [snip] > +/* > + * Setup an area of RAM so that it *can* be used for postcopy later; this > + * must be done right at the start prior to pre-copy. > + * opaque should be the MIS. > + */ > +static int init_area(const char *block_name, void *host_addr, > + ram_addr_t offset, ram_addr_t length, void *opaque) > +{ > + MigrationIncomingState *mis =3D opaque; > + > + DPRINTF("init_area: %s: %p offset=3D%zx length=3D%zd(%zx)", > + block_name, host_addr, offset, length, length); > + /* > + * We need the whole of RAM to be truly empty for postcopy, so things > + * like ROMs and any data tables built during init must be zero'd > + * - we're going to get the copy from the source anyway. > + */ > + if (postcopy_ram_discard_range(mis, host_addr, (host_addr + length -= 1))) { > + return -1; > + } > + > + /* > + * We also need the area to be normal 4k pages, not huge pages > + * (otherwise we can't be sure we can use remap_anon_pages to put > + * a 4k page in later). THP might come along and map a 2MB page > + * and when it's partially accessed in precopy it might not break > + * it down, but leave a 2MB zero'd page. > + */ > + if (madvise(host_addr, length, MADV_NOHUGEPAGE)) { > + perror("init_area: NOHUGEPAGE"); > + return -1; > + } I'm assuming this is because remap_anon_pages() can't automatically split a THP itself. It's not immediately obvious to me why it can't though. Also.. what effect will this have on an actual hugetlbfs memory region? If there's code to handle that case I haven't spotted it yet. > + > + return 0; > +} > + > +/* > + * At the end of migration, undo the effects of init_area > + * opaque should be the MIS. > + */ > +static int cleanup_area(const char *block_name, void *host_addr, > + ram_addr_t offset, ram_addr_t length, void *opaq= ue) > +{ > + /* Turn off userfault here as well? */ This comment appears to be obsoleted by the code below. > + > + DPRINTF("cleanup_area: %s: %p offset=3D%zx length=3D%zd(%zx)", > + block_name, host_addr, offset, length, length); > + /* > + * We turned off hugepage for the precopy stage with postcopy enabled > + * we can turn it back on now. > + */ > + if (madvise(host_addr, length, MADV_HUGEPAGE)) { > + perror("init_area: HUGEPAGE"); > + return -1; > + } > + > + /* > + * We can also turn off userfault now since we should have all the > + * pages. It can be useful to leave it on to debug postcopy > + * if you're not sure it's always getting every page. > + */ > + if (madvise(host_addr, length, MADV_NOUSERFAULT)) { > + perror("init_area: NOUSERFAULT"); > + return -1; > + } > + > + return 0; > +} > + > +/* > + * Initialise postcopy-ram, setting the RAM to a state where we can go i= nto > + * postcopy later; must be called prior to any precopy. > + * called from arch_init's similarly named ram_postcopy_incoming_init > + */ > +int postcopy_ram_incoming_init(MigrationIncomingState *mis, size_t ram_p= ages) > +{ > + postcopy_pmi_init(mis, ram_pages); > + > + if (qemu_ram_foreach_block(init_area, mis)) { > + return -1; > + } > + > + return 0; > +} > + > +/* > + * At the end of a migration where postcopy_ram_incoming_init was called. > + */ > +int postcopy_ram_incoming_cleanup(MigrationIncomingState *mis) > +{ > + /* TODO: Join the fault thread once we're sure it will exit */ > + if (qemu_ram_foreach_block(cleanup_area, mis)) { > + return -1; > + } > + > + return 0; > +} > + > #else > /* No target OS support, stubs just fail */ > =20 > @@ -404,6 +501,17 @@ void postcopy_hook_early_receive(MigrationIncomingSt= ate *mis, > /* We don't support postcopy so don't care */ > } > =20 > +int postcopy_ram_incoming_init(MigrationIncomingState *mis, size_t ram_p= ages) > +{ > + error_report("postcopy_ram_incoming_init: No OS support"); > + return -1; > +} > + > +int postcopy_ram_incoming_cleanup(MigrationIncomingState *mis) > +{ > + assert(0); > +} > + > void postcopy_pmi_destroy(MigrationIncomingState *mis) > { > /* Called in normal cleanup path - so it's OK */ > diff --git a/savevm.c b/savevm.c > index 7f9e0b2..54bdb26 100644 > --- a/savevm.c > +++ b/savevm.c > @@ -1166,6 +1166,10 @@ static int loadvm_postcopy_ram_handle_advise(Migra= tionIncomingState *mis, > return -1; > } > =20 > + if (ram_postcopy_incoming_init(mis)) { > + return -1; > + } > + > mis->postcopy_ram_state =3D POSTCOPY_RAM_INCOMING_ADVISE; > =20 > /* --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --LyciRD1jyfeSSjG0 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJUWcgBAAoJEGw4ysog2bOSOXwQAI+8iy5zSIaXp1fC3xOfpblF FDknEpLEKBBZx44WW320j1H3AQr6CYkPJRNQeH11JoQFr9zAqmnl0cBrM67/cLbb 8hno7DC2mAdoeT0VYr7WNbIfeAZlzyEj47TOLYnt5pz34iodTaCXsk0JKyfN4gLB nj3wIkBFhyhO2VSqgRSIUAGLaLAMtSvjW9RS209w/R6Pmnu60gkGDpMhqkLTO/6k oYxY4dyR48UlJJL8D1u7dETvfzP+zJ0pmYBCv6CKwo56YKD0Jo1DtsmUzvCeZF6u w3I1XUisnPSye6ySrZJ04gY+mU+i6j5fhc1++c3C/YLmSMBzdykPkMCD0Tf4EfLJ WM06m50a7K51858mhSfXlJL0aKXxMCXp5dq1wFYF40wqFEo49WE7YLwiROxZ6NKO wGfWmLSy1du4TZstPWd/N08U5kVU4AL9zVvh1MWMZ6064vjWCLFRIbwykphW5ya7 tHWeBpkug6hIsxTrW/U/5eykhM675SqgmzlFGmvU8zeO9vXxhjfq7X04kQBd+nOf p0iLPjUh+yt/ZZnHTwpT1aAkfEoU/eJ8k2oWS2DIqWsSTx41LPDsF59yqT8gneiF jwh2QdS1D2q+4DqFVcFO7iMpoTMHY58+sxC9suf/UGFC87vKF2fAE2VXCfCavAr6 WYx2YC02alqJ3h44RYpU =t0FV -----END PGP SIGNATURE----- --LyciRD1jyfeSSjG0--