xenpaging changes for xen-unstable
cover xenpaging changes for xen-unstable
Here are some changes for xenpaging in xen-unstable.
Patches 1 to 14 are likely non-controversial and could be applied. All later patches need more review (15/16), and more work from my side (17+).
With a similar series of changes for 4.0.1 my testcase, which is a continuous reboot of a Linux guest, passes now. It ran for more than 1000 iterations and exercised the early bootcode.
There are still some issues like guests not starting up when running the reboot loop with 4x1G guests parallel on a 4GB box with all guest memory paged. This seems to happen when the guest initially needs the full 1G until xenpaging frees that memory again.
Other than that, xenpaging is now more mature.
I havent seen data corruption anymore.
Olaf
PATCH 1/21 xenpaging: break endless loop during inital page-out with large pagefiles
To allow the starting for xenpaging right after 'xm start XYZ', I specified a pagefile size equal to the guest memory size in the hope to catch more errors where the paged-out state of a p2mt is not checked.
While doing that, xenpaging got into an endless loop because some pages cant be paged out right away. Now the policy reports an error if the gfn number wraps.
--- tools/xenpaging/policy_default.c | 35 ++++++++++++++++++++++++++++------- tools/xenpaging/xenpaging.c | 7 +++++-- 2 files changed, 33 insertions(+), 9 deletions(-)
tools/xenpaging/policy_default.c +28 -7
@@ -30,8 +30,12 @@static unsigned long mru[MRU_SIZE ]; -static unsigned int i_mru =0 ; +static unsigned int i_mru;static unsigned long *bitmap; +static unsigned long *unconsumed; +static unsigned long current_gfn; +static unsigned long bitmap_size; +static unsigned long max_pages;int policy_init (xenpaging_t *paging)
@@ -43,6 +47,14 @@ int policy_init(xenpaging_t *paging) rc =alloc_bitmap (&bitmap, paging->bitmap_size);if ( rc !=0 ) goto out; +/* Allocate bitmap to track unusable pages */ + rc =alloc_bitmap (&unconsumed, paging->bitmap_size); +if ( rc !=0 ) + goto out; + +/* record bitmap_size */ + bitmap_size = paging->bitmap_size; + max_pages = paging->domain_info->max_pages;/* Initialise MRU list of paged in pages */ for ( i =0 ; i <MRU_SIZE ; i++ )
@@ -51,8 +63,6 @@ int policy_init(xenpaging_t *paging)/* Don't page out page 0 */ set_bit (0 , bitmap); - rc =0 ; - out:return rc; }
@@ -61,17 +71,27 @@ int policy_choose_victim(xc_interface *x xenpaging_t *paging, domid_t domain_id, xenpaging_victim_t *victim) { + unsigned long wrap = current_gfn;ASSERT (victim !=NULL );/* Domain to pick on */ victim->domain_id = domain_id; - +do { -/* Randomly choose a gfn to evict */ - victim->gfn =rand () % paging->domain_info->max_pages; + current_gfn++; +if ( current_gfn >= max_pages ) + current_gfn =0 ; +if ( wrap == current_gfn ) + { + victim->gfn =INVALID_MFN ; +return -ENOSPC ; + } } -while (test_bit (victim->gfn, bitmap) ); +while (test_bit (current_gfn, bitmap) ||test_bit (current_gfn, unconsumed) ); + +set_bit (current_gfn, unconsumed); + victim->gfn = current_gfn;return 0 ; }
@@ -79,6 +99,7 @@ int policy_choose_victim(xc_interface *xvoid policy_notify_paged_out (domid_t domain_id,unsigned long gfn) {set_bit (gfn, bitmap); +clear_bit (gfn, unconsumed); }void policy_notify_paged_in (domid_t domain_id,unsigned long gfn)
tools/xenpaging/xenpaging.c +5 -2
@@ -446,7 +446,8 @@ static int evict_victim(xc_interface *xc ret =policy_choose_victim (xch, paging, domain_id, victim);if ( ret !=0 ) { -ERROR ("Error choosing victim" ); +if ( ret != -ENOSPC ) +ERROR ("Error choosing victim" ); goto out; }
@@ -525,7 +526,9 @@ int main(int argc, char *argv[])memset (victims,0 ,sizeof (xenpaging_victim_t) * num_pages);for ( i =0 ; i < num_pages; i++ ) { -evict_victim (xch, paging, domain_id, &victims[i], fd, i); + rc =evict_victim (xch, paging, domain_id, &victims[i], fd, i); +if ( rc == -ENOSPC ) +break ;if ( i %100 ==0 )DPRINTF ("%d pages evicted\n" , i); }
PATCH 2/21 xenpaging: Open paging file only if xenpaging_init() succeeds
In reply to Olaf HeringOpen paging file only if xenpaging_init() succeeds. It can fail if the host does not support the required virtualization features such as EPT or if xenpaging was already started for this domain_id.
--- tools/xenpaging/xenpaging.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
tools/xenpaging/xenpaging.c +9 -9
@@ -502,15 +502,6 @@ int main(int argc, char *argv[]) victims =calloc (num_pages,sizeof (xenpaging_victim_t)); -/* Open file */ -sprintf (filename,"page_cache_%d" , domain_id); - fd =open (filename, open_flags, open_mode); -if ( fd <0 ) - { -perror ("failed to open file" ); -return -1 ; - } -/* Seed random-number generator */ srand (time (NULL ));
@@ -522,6 +513,15 @@ int main(int argc, char *argv[]) goto out; } +/* Open file */ +sprintf (filename,"page_cache_%d" , domain_id); + fd =open (filename, open_flags, open_mode); +if ( fd <0 ) + { +perror ("failed to open file" ); +return -1 ; + } +/* Evict pages */ memset (victims,0 ,sizeof (xenpaging_victim_t) * num_pages);for ( i =0 ; i < num_pages; i++ )
PATCH 3/21 xenpaging: allow only one xenpaging binary per guest
In reply to Olaf HeringMake sure only one xenpaging binary is active per domain. Print info when the host lacks the required features for xenpaging.
---
v2: use perror for default case
tools/xenpaging/xenpaging.c | 12 +++++++++++- xen/arch/x86/mm/mem_event.c | 7 +++++++ 2 files changed, 18 insertions(+), 1 deletion(-)
tools/xenpaging/xenpaging.c +11 -1
@@ -123,7 +123,17 @@ xenpaging_t *xenpaging_init(xc_interface paging->mem_event.ring_page);if ( rc !=0 ) { -ERROR ("Error initialising shared page" ); +switch ( errno ) { +case EBUSY : +ERROR ("xenpaging is (or was) active on this domain" ); +break ; +case ENODEV : +ERROR ("EPT not supported for this guest" ); +break ; +default : +perror ("Error initialising shared page" ); +break ; + } goto err; }
xen/arch/x86/mm/mem_event.c +7 -0
@@ -214,6 +214,13 @@ int mem_event_domctl(struct domain *d, xmfn_t ring_mfn;mfn_t shared_mfn; +/* Only one xenpaging at a time. If xenpaging crashed, +* the cache is in an undefined state and so is the guest +*/ + rc = -EBUSY ; +if ( d->mem_event.ring_page ) +break ; +/* Currently only EPT is supported */ rc = -ENODEV ; if ( !(hap_enabled(d) &&
PATCH 4/21 xenpaging: populate paged-out pages unconditionally
In reply to Olaf HeringPopulate a page unconditionally to avoid missing a page-in request. If the page is already in the process of being paged-in, the this vcpu will be stopped and later resumed once the page content is usable again.
This matches other p2m_mem_paging_populate usage in the source tree.
--- xen/common/grant_table.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
xen/common/grant_table.c +3 -2
@@ -156,10 +156,11 @@ static int __get_paged_frame(unsigned loif (p2m_is_valid (p2mt) ) { *frame =mfn_x (mfn); -if (p2m_is_paged (p2mt) ) -p2m_mem_paging_populate (p2m, gfn);if (p2m_is_paging (p2mt) ) + { +p2m_mem_paging_populate (p2m, gfn); rc = GNTST_eagain; + } }else { *frame =INVALID_MFN ; rc = GNTST_bad_page;
PATCH 5/21 xenpaging: add signal handling
In reply to Olaf HeringLeave paging loop if xenpaging gets a signal.
Remove paging file on exit.
---
v2:
unlink pagefile in signal handler to avoid stale pagefiles if xenpaging is stuck in some loop
tools/xenpaging/xenpaging.c | 42 +++++++++++++++++++++++++++++++++--------- 1 file changed, 33 insertions(+), 9 deletions(-)
tools/xenpaging/xenpaging.c +33 -9
@@ -25,6 +25,7 @@#include <stdlib.h> #include <stdarg.h> #include <time.h> +#include <signal.h> #include <xc_private.h> #include <xen/mem_event.h>
@@ -43,6 +44,14 @@#define DPRINTF (...) ((void)0)#endif +static char filename[80 ]; +static int interrupted; +static void close_handler (int sig) +{ + interrupted = sig; +if ( filename[0 ] ) +unlink (filename); +}static void *init_page (void ) {
@@ -248,7 +257,6 @@ int xenpaging_teardown(xc_interface *xchif ( rc !=0 ) {ERROR ("Error tearing down domain paging in xen" ); - goto err; }/* Unbind VIRQ */
@@ -256,7 +264,6 @@ int xenpaging_teardown(xc_interface *xchif ( rc !=0 ) {ERROR ("Error unbinding event port" ); - goto err; } paging->mem_event.port =-1 ;
@@ -265,7 +272,6 @@ int xenpaging_teardown(xc_interface *xchif ( rc !=0 ) {ERROR ("Error closing event channel" ); - goto err; } paging->mem_event.xce_handle =-1 ;
@@ -274,7 +280,6 @@ int xenpaging_teardown(xc_interface *xchif ( rc !=0 ) {ERROR ("Error closing connection to xen" ); - goto err; } paging->xc_handle =NULL ;
@@ -380,7 +385,7 @@ int xenpaging_evict_page(xc_interface *xreturn ret; } -int xenpaging_resume_page (xenpaging_t *paging,mem_event_response_t *rsp) +static int xenpaging_resume_page (xenpaging_t *paging,mem_event_response_t *rsp) {int ret;
@@ -461,6 +466,11 @@ static int evict_victim(xc_interface *xc goto out; } +if ( interrupted ) + { + ret = -EINTR ; + goto out; + } ret =xc_mem_paging_nominate (paging->xc_handle, paging->mem_event.domain_id, victim->gfn);if ( ret ==0 )
@@ -485,6 +495,7 @@ static int evict_victim(xc_interface *xcint main (int argc,char *argv[]) { +struct sigaction act;domid_t domain_id;int num_pages;xenpaging_t *paging;
@@ -498,7 +509,6 @@ int main(int argc, char *argv[])int open_flags =O_CREAT |O_TRUNC |O_RDWR ;mode_t open_mode =S_IRUSR |S_IRGRP |S_IROTH |S_IWUSR |S_IWGRP |S_IWOTH ; -char filename[80 ];int fd;if ( argc !=3 )
@@ -520,7 +530,7 @@ int main(int argc, char *argv[])if ( paging ==NULL ) {ERROR ("Error initialising paging" ); - goto out; +return 1 ; }/* Open file */
@@ -529,9 +539,18 @@ int main(int argc, char *argv[])if ( fd <0 ) {perror ("failed to open file" ); -return -1 ; +return 2 ; } +/* ensure that if we get a signal, we'll do cleanup, then exit */ + act.sa_handler = close_handler; + act.sa_flags =0 ; +sigemptyset (&act.sa_mask); +sigaction (SIGHUP , &act,NULL ); +sigaction (SIGTERM , &act,NULL ); +sigaction (SIGINT , &act,NULL ); +sigaction (SIGALRM , &act,NULL ); +/* Evict pages */ memset (victims,0 ,sizeof (xenpaging_victim_t) * num_pages);for ( i =0 ; i < num_pages; i++ )
@@ -539,6 +558,8 @@ int main(int argc, char *argv[]) rc =evict_victim (xch, paging, domain_id, &victims[i], fd, i);if ( rc == -ENOSPC )break ; +if ( rc == -EINTR ) +break ;if ( i %100 ==0 )DPRINTF ("%d pages evicted\n" , i); }
@@ -546,7 +567,7 @@ int main(int argc, char *argv[])DPRINTF ("pages evicted\n" );/* Swap pages in and out */ -while (1 ) +while ( !interrupted ) {/* Wait for Xen to signal that a page needs paged in */ rc =xc_wait_for_event_or_timeout (xch, paging->mem_event.xce_handle,100 );
@@ -637,8 +658,10 @@ int main(int argc, char *argv[]) } } } +DPRINTF ("xenpaging got signal %d\n" , interrupted); out: +close (fd);free (victims);/* Tear down domain paging */
@@ -651,6 +674,7 @@ int main(int argc, char *argv[])xc_interface_close (xch); +DPRINTF ("xenpaging exit code %d\n" , rc);return rc; }
PATCH 6/21 xenpaging: print info when free request slots drop below 2
In reply to Olaf HeringAdd debugging aid to free request slots in the ring buffer.
It should not happen that the ring gets full, print info anyway if it happens.
--- xen/arch/x86/mm/mem_event.c | 5 +++++ 1 file changed, 5 insertions(+)
xen/arch/x86/mm/mem_event.c +5 -0
@@ -152,6 +152,11 @@ int mem_event_check_ring(struct domain *mem_event_ring_lock (d); free_requests =RING_FREE_REQUESTS (&d->mem_event.front_ring); +if (unlikely (free_requests <2 ) ) + { +gdprintk (XENLOG_INFO ,"free request slots: %d\n" , free_requests); +WARN_ON (free_requests ==0 ); + } ring_full = free_requests <MEM_EVENT_RING_THRESHOLD ;if ( (curr->domain->domain_id == d->domain_id) && ring_full )
PATCH 7/21 xenpaging: print p2mt for already paged-in pages
In reply to Olaf HeringAdd more debug output, print p2mt for pages which were requested more than once.
--- tools/xenpaging/xenpaging.c | 2 ++ 1 file changed, 2 insertions(+)
tools/xenpaging/xenpaging.c +2 -0
@@ -634,8 +634,10 @@ int main(int argc, char *argv[])else {DPRINTF ("page already populated (domain = %d; vcpu = %d;" +" p2mt = %x;" " gfn = %" PRIx64"; paused = %" PRId64")\n" , paging->mem_event.domain_id, req.vcpu_id, + req.p2mt, req.gfn, req.flags &MEM_EVENT_FLAG_VCPU_PAUSED );/* Tell Xen to resume the vcpu */
PATCH 8/21 xenpaging: notify policy only on resume
In reply to Olaf HeringIf a page is requested more than once, the policy is also notified more than once about the page-in. However, a page-in happens only once. Any further resume will only unpause the other vcpu. The multiple notify will put the page into the mru list multiple times and it will unlock other already resumed pages too early. In the worst case, a page that was just resumed can be evicted right away, causing a deadlock in the guest.
--- tools/xenpaging/xenpaging.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
tools/xenpaging/xenpaging.c +5 -4
@@ -385,7 +385,7 @@ int xenpaging_evict_page(xc_interface *xreturn ret; } -static int xenpaging_resume_page (xenpaging_t *paging,mem_event_response_t *rsp) +static int xenpaging_resume_page (xenpaging_t *paging,mem_event_response_t *rsp,int notify_policy) {int ret;
@@ -395,7 +395,8 @@ static int xenpaging_resume_page(xenpagi goto out;/* Notify policy of page being paged in */ -policy_notify_paged_in (paging->mem_event.domain_id, rsp->gfn); +if ( notify_policy ) +policy_notify_paged_in (paging->mem_event.domain_id, rsp->gfn);/* Tell Xen page is ready */ ret = xc_mem_paging_resume(paging->xc_handle, paging->mem_event.domain_id,
@@ -621,7 +622,7 @@ int main(int argc, char *argv[]) rsp.vcpu_id = req.vcpu_id; rsp.flags = req.flags; - rc =xenpaging_resume_page (paging, &rsp); + rc =xenpaging_resume_page (paging, &rsp,1 );if ( rc !=0 ) {ERROR ("Error resuming page" );
@@ -650,7 +651,7 @@ int main(int argc, char *argv[]) rsp.vcpu_id = req.vcpu_id; rsp.flags = req.flags; - rc =xenpaging_resume_page (paging, &rsp); + rc =xenpaging_resume_page (paging, &rsp,0 );if ( rc !=0 ) {ERROR ("Error resuming" );
PATCH 9/21 xenpaging: allow negative num_pages and limit num_pages
In reply to Olaf HeringSimplify paging size argument. If a negative number is specified, it means the entire guest memory should be paged out. This is useful for debugging. Also limit num_pages to the guests max_pages.
--- tools/xenpaging/xenpaging.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
tools/xenpaging/xenpaging.c +7 -2
@@ -521,8 +521,6 @@ int main(int argc, char *argv[]) domain_id =atoi (argv[1 ]); num_pages =atoi (argv[2 ]); - victims =calloc (num_pages,sizeof (xenpaging_victim_t)); -/* Seed random-number generator */ srand (time (NULL ));
@@ -543,6 +541,13 @@ int main(int argc, char *argv[])return 2 ; } +if ( num_pages <0 || num_pages > paging->domain_info->max_pages ) + { + num_pages = paging->domain_info->max_pages; +DPRINTF ("setting num_pages to %d\n" , num_pages); + } + victims =calloc (num_pages,sizeof (xenpaging_victim_t)); +/* ensure that if we get a signal, we'll do cleanup, then exit */ act.sa_handler = close_handler; act.sa_flags =0 ;
PATCH 10/21 xenpaging: when populating a page, check if populating is already in progress
In reply to Olaf Heringp2m_mem_paging_populate can be called serveral times from different vcpus. If the page is already in state p2m_ram_paging_in and has a new valid mfn, invalidating this new mfn will cause trouble later if p2m_mem_paging_resume will set the new gfn/mfn pair back to state p2m_ram_rw.
Detect this situation and keep p2m state if the page is in the process of being still paged-out or already paged-in.
In fact, p2m state p2m_ram_paged is the only state where the mfn type can be invalidated.
--- xen/arch/x86/mm/p2m.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
xen/arch/x86/mm/p2m.c +1 -1
@@ -2767,7 +2767,7 @@ void p2m_mem_paging_populate(struct p2m_/* XXX: It seems inefficient to have this here, as it's only needed * in one case (ept guest accessing paging out page) */ gfn_to_mfn (p2m, gfn, &p2mt); -if ( p2mt != p2m_ram_paging_out ) +if ( p2mt == p2m_ram_paged ) {p2m_lock (p2m);set_p2m_entry (p2m, gfn,_mfn (PAGING_MFN ),0 , p2m_ram_paging_in_start);
PATCH 11/21 xenpaging: optimize p2m_mem_paging_populate
In reply to Olaf Heringp2m_mem_paging_populate will always put another request in the ring. To reduce pressure on the ring, place only required requests in the ring. If the gfn was already processed by another thread, and the current vcpu does not need to be paused, p2m_mem_paging_resume will do nothing with the request. And also xenpaging will drop the request if the vcpu does not need a wakeup.
--- xen/arch/x86/mm/p2m.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
xen/arch/x86/mm/p2m.c +7 -2
@@ -2757,12 +2757,12 @@ void p2m_mem_paging_populate(struct p2m_p2m_type_t p2mt;struct domain *d = p2m->domain; -memset (&req,0 ,sizeof (req)); -/* Check that there's space on the ring for this request */ if (mem_event_check_ring (d) )return ; +memset (&req,0 ,sizeof (req)); +/* Fix p2m mapping */ /* XXX: It seems inefficient to have this here, as it's only needed * in one case (ept guest accessing paging out page) */
@@ -2781,6 +2781,11 @@ void p2m_mem_paging_populate(struct p2m_vcpu_pause_nosync (v); req.flags |=MEM_EVENT_FLAG_VCPU_PAUSED ; } +else if ( p2mt != p2m_ram_paging_out && p2mt != p2m_ram_paged ) + { +/* gfn is already on its way back and vcpu is not paused */ +return ; + }/* Send request to pager */ req.gfn = gfn;
PATCH 12/21 xenpaging: print xenpaging cmdline options
In reply to Olaf HeringPrint xenpaging arguments to simplify domain_id mapping from xenpaging logfile to other logfiles and Xen console output.
--- tools/xenpaging/xenpaging.c | 2 ++ 1 file changed, 2 insertions(+)
tools/xenpaging/xenpaging.c +2 -0
@@ -532,6 +532,8 @@ int main(int argc, char *argv[])return 1 ; } +DPRINTF ("starting %s %u %d\n" , argv[0 ], domain_id, num_pages); +/* Open file */ sprintf (filename,"page_cache_%d" , domain_id); fd =open (filename, open_flags, open_mode);
PATCH 13/21 xenpaging: handle temporary out-of-memory conditions during page-in
In reply to Olaf Heringp2m_mem_paging_prep() should return -ENOMEM if a new page could not be allocated. This can be handled in xenpaging to retry the page-in. Right now such condition would stall the guest because the requested page will not come back, xenpaging simply exits. So xenpaging could very well retry the allocation forever to rescue the guest.
--- tools/xenpaging/xenpaging.c | 27 ++++++++++++++++++++------- xen/arch/x86/mm/p2m.c | 2 +- 2 files changed, 21 insertions(+), 8 deletions(-)
tools/xenpaging/xenpaging.c +20 -7
@@ -26,6 +26,7 @@#include <stdarg.h> #include <time.h> #include <signal.h> +#include <unistd.h> #include <xc_private.h> #include <xen/mem_event.h>
@@ -415,19 +416,31 @@ static int xenpaging_populate_page(unsigned long _gfn;void *page;int ret; +unsigned char oom =0 ; -/* Tell Xen to allocate a page for the domain */ - ret =xc_mem_paging_prep (paging->xc_handle, paging->mem_event.domain_id, - *gfn); -if ( ret !=0 ) + _gfn = *gfn; +do { -ERROR ("Error preparing for page in" ); - goto out_map; +/* Tell Xen to allocate a page for the domain */ + ret =xc_mem_paging_prep (paging->xc_handle, paging->mem_event.domain_id, + _gfn); +if ( ret !=0 ) + { +if ( errno ==ENOMEM ) + { +if ( oom++ ==0 ) +DPRINTF ("ENOMEM while preparing gfn %lx\n" , _gfn); +sleep (1 ); +continue ; + } +ERROR ("Error preparing for page in" ); + goto out_map; + } } +while ( ret && !interrupted );/* Map page */ ret = -EFAULT ; - _gfn = *gfn; page =xc_map_foreign_pages (paging->xc_handle, paging->mem_event.domain_id,PROT_READ |PROT_WRITE , &_gfn,1 ); *gfn = _gfn;
xen/arch/x86/mm/p2m.c +1 -1
@@ -2802,7 +2802,7 @@ int p2m_mem_paging_prep(struct p2m_domai/* Get a free page */ page =alloc_domheap_page (p2m->domain,0 );if (unlikely (page ==NULL ) ) -return -EINVAL ; +return -ENOMEM ;/* Fix p2m mapping */ p2m_lock (p2m);
PATCH 14/21 xenpaging: increase recently used pages from 4MB to 64MB
In reply to Olaf HeringIncrease recently used pages from 4MB to 64MB.
Keeping more pages in memory allows the guest to make more progress if the paging file spans the entire guest memory.
--- tools/xenpaging/policy_default.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/xenpaging/policy_default.c +1 -1
@@ -26,7 +26,7 @@#include "policy.h" -#define MRU_SIZE 1024 +#define MRU_SIZE (1024 * 16)static unsigned long mru[MRU_SIZE ];
PATCH 15/21 xenpaging: update machine_to_phys_mapping during page-in and page deallocation
In reply to Olaf HeringThe machine_to_phys_mapping array needs updating during page-in, and during page deallocation. If a page is gone, a call to get_gpfn_from_mfn will still return the old gfn for an already paged-out page. This happens when the entire guest ram is paged-out before xen_vga_populate_vram() runs. Then XENMEM_populate_physmap is called with gfn 0xff000. A new page is allocated with alloc_domheap_pages. This new page does not have a gfn yet. However, in guest_physmap_add_entry() the passed mfn maps still to an old gfn (perhaps from another old guest). This old gfn is in paged-out state in this guests context and has no mfn anymore. As a result, the ASSERT() triggers because p2m_is_ram() is true for p2m_ram_paging* types.
If the machine_to_phys_mapping array is updated properly, both loops in guest_physmap_add_entry() turn into no-ops for the new page and the mfn/gfn mapping will be done at the end of the function.
The same thing needs to happen dring a page-in, the current gfn must be written for the page.
If XENMEM_add_to_physmap is used with XENMAPSPACE_gmfn, get_gpfn_from_mfn() will return an appearently valid gfn. As a result, guest_physmap_remove_page() is called. The ASSERT in p2m_remove_page triggers because the passed mfn does not match the old mfn for the passed gfn.
---
v3:
invalidate machine_to_phys_mapping[] during page deallocation
v2:
call set_gpfn_from_mfn only if mfn is valid
--- xen/arch/x86/mm/p2m.c | 15 +++++++++++---- xen/common/page_alloc.c | 9 +++++++++ 2 files changed, 20 insertions(+), 4 deletions(-)
xen/arch/x86/mm/p2m.c +11 -4
@@ -2825,10 +2825,17 @@ void p2m_mem_paging_resume(struct p2m_do/* Fix p2m entry */ mfn =gfn_to_mfn (p2m, rsp.gfn, &p2mt); -p2m_lock (p2m); -set_p2m_entry (p2m, rsp.gfn, mfn,0 , p2m_ram_rw); -audit_p2m (p2m,1 ); -p2m_unlock (p2m); +if (mfn_valid (mfn) ) + { +p2m_lock (p2m); +set_p2m_entry (p2m, rsp.gfn, mfn,0 , p2m_ram_rw); +set_gpfn_from_mfn (mfn_x (mfn), rsp.gfn); +audit_p2m (p2m,1 ); +p2m_unlock (p2m); + }else { +gdprintk (XENLOG_ERR ,"invalid mfn %lx for gfn %lx p2mt %x flags %lx\n" , +mfn_x (mfn), rsp.gfn, p2mt, (unsigned long )rsp.flags); + }/* Unpause domain */ if ( rsp.flags &MEM_EVENT_FLAG_VCPU_PAUSED )
xen/common/page_alloc.c +9 -0
@@ -1199,9 +1199,18 @@ void free_domheap_pages(struct page_info {int i, drop_dom_ref;struct domain *d =page_get_owner (pg); +unsigned long mfn;ASSERT (!in_irq ()); +/* this page is not a gfn anymore */ + mfn =page_to_mfn (pg); +if (mfn_valid (mfn) ) + { +for ( i =0 ; i < (1 << order); i++ ) +set_gpfn_from_mfn (mfn + i,INVALID_M2P_ENTRY ); + } +if (unlikely (is_xen_heap_page (pg)) ) {/* NB. May recursively lock from relinquish_memory(). */
PATCH 16/21 xenpaging: drop paged pages in guest_remove_page
In reply to Olaf HeringSimply drop paged-pages in guest_remove_page(), and notify xenpaging to drop its reference to the gfn.
---
v2:
resume dropped page to unpause vcpus
tools/xenpaging/xenpaging.c | 17 +++++++--- xen/arch/x86/mm/p2m.c | 68 +++++++++++++++++++++++++++++++---------- xen/common/memory.c | 6 +++ xen/include/asm-x86/p2m.h | 4 ++ xen/include/public/mem_event.h | 1 5 files changed, 75 insertions(+), 21 deletions(-)
tools/xenpaging/xenpaging.c +12 -5
@@ -628,12 +628,19 @@ int main(int argc, char *argv[]) goto out; } -/* Populate the page */ - rc =xenpaging_populate_page (xch, paging, &req.gfn, fd, i); -if ( rc !=0 ) +if ( req.flags &MEM_EVENT_FLAG_DROP_PAGE ) { -ERROR ("Error populating page" ); - goto out; +DPRINTF ("Dropping page %" PRIx64" p2mt %x\n" , req.gfn, req.p2mt); + } +else + { +/* Populate the page */ + rc =xenpaging_populate_page (xch, paging, &req.gfn, fd, i); +if ( rc !=0 ) + { +ERROR ("Error populating page" ); + goto out; + } }/* Prepare the response */
xen/arch/x86/mm/p2m.c +52 -16
88 lines not on this page · raw message
xen/common/memory.c +6 -0
@@ -162,6 +162,12 @@ int guest_remove_page(struct domain *d,#ifdef CONFIG_X86 mfn =mfn_x (gfn_to_mfn (p2m_get_hostp2m (d), gmfn, &p2mt)); +if (unlikely (p2m_is_paging (p2mt)) ) + { +guest_physmap_remove_page (d, gmfn, mfn,0 ); +p2m_mem_paging_drop_page (p2m_get_hostp2m (d), gmfn); +return 1 ; + }#else mfn =gmfn_to_mfn (d, gmfn);#endif
xen/include/asm-x86/p2m.h +4 -0
@@ -471,6 +471,8 @@ int set_shared_p2m_entry(struct p2m_domaint p2m_mem_paging_nominate (struct p2m_domain *p2m,unsigned long gfn);/* Evict a frame */ int p2m_mem_paging_evict (struct p2m_domain *p2m,unsigned long gfn); +/* Tell xenpaging to drop a paged out frame */ +void p2m_mem_paging_drop_page (struct p2m_domain *p2m,unsigned long gfn);/* Start populating a paged out frame */ void p2m_mem_paging_populate (struct p2m_domain *p2m,unsigned long gfn);/* Prepare the p2m for paging a frame in */
@@ -478,6 +480,8 @@ int p2m_mem_paging_prep(struct p2m_domai/* Resume normal operation (in case a domain was paused) */ void p2m_mem_paging_resume (struct p2m_domain *p2m);#else +static inline void p2m_mem_paging_drop_page (struct p2m_domain *p2m,unsigned long gfn) +{ }static inline void p2m_mem_paging_populate (struct p2m_domain *p2m,unsigned long gfn) { }#endif
xen/include/public/mem_event.h +1 -0
@@ -37,6 +37,7 @@#define MEM_EVENT_FLAG_VCPU_PAUSED (1 << 0)#define MEM_EVENT_FLAG_DOM_PAUSED (1 << 1)#define MEM_EVENT_FLAG_OUT_OF_MEM (1 << 2) +#define MEM_EVENT_FLAG_DROP_PAGE (1 << 3) typedef struct mem_event_shared_page {
PATCH 17/21 xenpaging: handle HVMCOPY_gfn_paged_out in copy_from/to_user
In reply to Olaf HeringThis is a quick fix to keep my guests running. The proper fix will use the recently added waitqueue API.
copy_from_user_hvm can fail when __hvm_copy returns HVMCOPY_gfn_paged_out for a referenced gfn, for example during guests pagetable walk. This has to be handled in some way. One hypercall that failed was do_memory_op/XENMEM_decrease_reservation which lead to a BUG_ON balloon.c. Since do_memory_op already has restart support for the hypercall, copy_from_guest uses this existing retry code. In addition, cleanup on error was added to increase_reservation and populate_physmap.
--- xen/arch/x86/hvm/hvm.c | 4 ++++ xen/common/memory.c | 44 +++++++++++++++++++++++++++++++++++++++----- 2 files changed, 43 insertions(+), 5 deletions(-)
xen/arch/x86/hvm/hvm.c +4 -0
@@ -2066,6 +2066,8 @@ unsigned long copy_to_user_hvm(void *to, rc =hvm_copy_to_guest_virt_nofault ((unsigned long )to, (void *)from, len,0 ); +if ( rc == HVMCOPY_gfn_paged_out ) +return -EAGAIN ;return rc ? len :0 ;/* fake a copy_to_user() return code */ }
@@ -2083,6 +2085,8 @@ unsigned long copy_from_user_hvm(void *t#endif rc =hvm_copy_from_guest_virt_nofault (to, (unsigned long )from, len,0 ); +if ( rc == HVMCOPY_gfn_paged_out ) +return -EAGAIN ;return rc ? len :0 ;/* fake a copy_from_user() return code */ }
xen/common/memory.c +39 -5
103 lines not on this page · raw message
PATCH 18/21 xenpaging: prevent page-out of first 16MB
In reply to Olaf HeringThis is more a workaround than a bugfix:
Don't page out first 16MB of memory.
When the BIOS does its initialization process and xenpaging removes pages, crashes will occour due to lack of support of xenpaging.
--- tools/xenpaging/policy_default.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
tools/xenpaging/policy_default.c +3 -2
@@ -60,8 +60,9 @@ int policy_init(xenpaging_t *paging)for ( i =0 ; i <MRU_SIZE ; i++ ) mru[i] =INVALID_MFN ; -/* Don't page out page 0 */ -set_bit (0 , bitmap); +/* Don't page out first 16MB */ +for ( i =0 ; i < ((16 *1024 *1024 )/4096 ); i++ ) +set_bit (i, bitmap); out:return rc;
PATCH 19/21 xenpaging: start xenpaging via config option
In reply to Olaf HeringStart xenpaging via config option.
TODO: add config option for different pagefile directory
TODO: add libxl support
TODO: parse config values like 42K, 42M, 42G, 42%
---
v2:
unlink logfile instead of truncating it.
allows hardlinking for further inspection
tools/examples/xmexample.hvm | 3 + tools/python/README.XendConfig | 1 tools/python/README.sxpcfg | 1 tools/python/xen/xend/XendConfig.py | 3 + tools/python/xen/xend/XendDomainInfo.py | 6 ++ tools/python/xen/xend/image.py | 91 ++++++++++++++++++++++++++++++++ tools/python/xen/xm/create.py | 5 + tools/python/xen/xm/xenapi_create.py | 1 8 files changed, 111 insertions(+)
tools/examples/xmexample.hvm +3 -0
@@ -127,6 +127,9 @@ disk = [ 'file:/var/images/min-el3-i386. # Device Model to be used device_model = 'qemu-dm' +# xenpaging, number of pages +xenpaging = 42 + #----------------------------------------------------------------------------- # boot on floppy (a), hard disk (c), Network (n) or CD-ROM (d) # default: hard disk, cd-rom, floppy
tools/python/README.XendConfig +1 -0
@@ -120,6 +120,7 @@ otherConfig image.vncdisplay image.vncunused image.hvm.device_model + image.hvm.xenpaging image.hvm.display image.hvm.xauthority image.hvm.vncconsole
tools/python/README.sxpcfg +1 -0
@@ -51,6 +51,7 @@ image - vncunused (HVM) - device_model + - xenpaging - display - xauthority - vncconsole
tools/python/xen/xend/XendConfig.py +3 -0
@@ -147,6 +147,7 @@ XENAPI_PLATFORM_CFG_TYPES = { 'apic': int, 'boot': str, 'device_model': str, + 'xenpaging': int, 'loader': str, 'display' : str, 'fda': str,
@@ -508,6 +509,8 @@ class XendConfig(dict): self['platform']['nomigrate'] = 0 if self.is_hvm(): + if 'xenpaging' not in self['platform']: + self['platform']['xenpaging'] = None if 'timer_mode' not in self['platform']: self['platform']['timer_mode'] = 1 if 'viridian' not in self['platform']:
tools/python/xen/xend/XendDomainInfo.py +6 -0
@@ -2390,6 +2390,7 @@ class XendDomainInfo: if self.image: self.image.createDeviceModel() + self.image.createXenPaging() #if have pass-through devs, need the virtual pci slots info from qemu self.pci_device_configure_boot()
@@ -2402,6 +2403,11 @@ class XendDomainInfo: self.image.destroyDeviceModel() except Exception, e: log.exception("Device model destroy failed %s" % str(e)) + try: + log.debug("stopping xenpaging") + self.image.destroyXenPaging() + except Exception, e: + log.exception("stopping xenpaging failed %s" % str(e)) else: log.debug("No device model")
tools/python/xen/xend/image.py +91 -0
109 lines not on this page · raw message
tools/python/xen/xm/create.py +5 -0
@@ -491,6 +491,10 @@ gopts.var('nfs_root', val="PATH", fn=set_value, default=None, use="Set the path of the root NFS directory.") +gopts.var('xenpaging', val='NUM', + fn=set_int, default=None, + use="Number of pages to swap.") + gopts.var('device_model', val='FILE', fn=set_value, default=None, use="Path to device model program.")
@@ -1076,6 +1080,7 @@ def configure_hvm(config_image, vals): args = [ 'acpi', 'apic', 'boot', 'cpuid', 'cpuid_check', + 'xenpaging', 'device_model', 'display', 'fda', 'fdb', 'gfx_passthru', 'guest_os_type',
tools/python/xen/xm/xenapi_create.py +1 -0
@@ -1085,6 +1085,7 @@ class sxp2xml: 'acpi', 'apic', 'boot', + 'xenpaging', 'device_model', 'loader', 'fda',
PATCH 20/21 xenpaging: add dynamic startup delay for xenpaging
In reply to Olaf HeringThis is a debug helper. Since the xenpaging support is still fragile, run xenpaging at different stages in the bootprocess. Different delays will trigger more bugs. This implementation starts without delay for 5 reboots, then increments the delay by 0.1 seconds It uses xenstore for presistant storage of delay values
TODO: find the correct place to remove the xenstore directory when the guest is shutdown or crashed
--- tools/python/xen/xend/image.py | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)
tools/python/xen/xend/image.py +32 -0
@@ -123,6 +123,18 @@ class ImageHandler: self.device_model = vmConfig['platform'].get('device_model') self.xenpaging = vmConfig['platform'].get('xenpaging') + self.xenpaging_delay = xstransact.Read("/local/domain/0/xenpaging/%s/xenpaging_delay" % self.vm.info['name_label']) + if self.xenpaging_delay == None: + log.warn("XXX creating /local/domain/0/xenpaging/%s" % self.vm.info['name_label']) + xstransact.Mkdir("/local/domain/0/xenpaging/%s" % self.vm.info['name_label']) + xstransact.Store("/local/domain/0/xenpaging/%s" % self.vm.info['name_label'], ('xenpaging_delay', '0.0')) + xstransact.Store("/local/domain/0/xenpaging/%s" % self.vm.info['name_label'], ('xenpaging_delay_inc', '0.1')) + xstransact.Store("/local/domain/0/xenpaging/%s" % self.vm.info['name_label'], ('xenpaging_delay_use', '5')) + xstransact.Store("/local/domain/0/xenpaging/%s" % self.vm.info['name_label'], ('xenpaging_delay_used', '0')) + self.xenpaging_delay = float(xstransact.Read("/local/domain/0/xenpaging/%s/xenpaging_delay" % self.vm.info['name_label'])) + self.xenpaging_delay_inc = float(xstransact.Read("/local/domain/0/xenpaging/%s/xenpaging_delay_inc" % self.vm.info['name_label'])) + self.xenpaging_delay_use = int(xstransact.Read("/local/domain/0/xenpaging/%s/xenpaging_delay_use" % self.vm.info['name_label'])) + self.xenpaging_delay_used = int(xstransact.Read("/local/domain/0/xenpaging/%s/xenpaging_delay_used" % self.vm.info['name_label'])) self.display = vmConfig['platform'].get('display') self.xauthority = vmConfig['platform'].get('xauthority')
@@ -401,6 +413,17 @@ class ImageHandler: return if self.xenpaging_pid: return + if self.xenpaging_delay_used < self.xenpaging_delay_use: + self.xenpaging_delay_used += 1 + else: + self.xenpaging_delay_used = 0 + self.xenpaging_delay += self.xenpaging_delay_inc + log.info("delay_used %s" % self.xenpaging_delay_used) + log.info("delay_use %s" % self.xenpaging_delay_use) + log.info("delay %s" % self.xenpaging_delay) + log.info("delay_inc %s" % self.xenpaging_delay_inc) + xstransact.Store("/local/domain/0/xenpaging/%s" % self.vm.info['name_label'], ('xenpaging_delay', self.xenpaging_delay)) + xstransact.Store("/local/domain/0/xenpaging/%s" % self.vm.info['name_label'], ('xenpaging_delay_used', self.xenpaging_delay_used)) xenpaging_bin = auxbin.pathTo("xenpaging") args = [xenpaging_bin] args = args + ([ "%d" % self.vm.getDomid()])
@@ -434,6 +457,9 @@ class ImageHandler: except: log.warn("chdir %s failed" % xenpaging_dir) try: + if self.xenpaging_delay != 0.0: + log.info("delaying xenpaging startup %s seconds ..." % self.xenpaging_delay) + time.sleep(self.xenpaging_delay) log.info("starting %s" % args) os.execve(xenpaging_bin, args, env) except Exception, e:
@@ -449,10 +475,16 @@ class ImageHandler: self.xenpaging_pid = xenpaging_pid os.close(null) os.close(logfd) + if self.xenpaging_delay == 0.0: + log.warn("waiting for xenpaging ...") + time.sleep(22) + log.warn("waiting for xenpaging done.") def destroyXenPaging(self): if self.xenpaging is None: return + # FIXME find correct place for guest shutdown or crash + #xstransact.Remove("/local/domain/0/xenpaging/%s" % self.vm.info['name_label']) if self.xenpaging_pid: try: os.kill(self.xenpaging_pid, signal.SIGHUP)
PATCH 21/21 xenpaging: (sparse) documenation
In reply to Olaf HeringWrite up some sparse documentation about xenpaging usage.
--- docs/misc/xenpaging.txt | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+)
docs/misc/xenpaging.txt new file +48 -0
@@ -0,0 +1,48 @@ +Warning: + +The xenpaging code is new and not fully debugged. +Usage of xenpaging can crash Xen or cause severe data corruption in the +guest memory and its filesystems! + +Description: + +xenpaging writes memory pages of a given guest to a file and moves the +pages back to the pool of available memory. Once the guests wants to +access the paged-out memory, the page is read from disk and placed into +memory. This allows the sum of all running guests to use more memory +than physically available on the host. + +Usage: + +Once the guest is running, run xenpaging with the guest_id and the +number of pages to page-out: + + chdir /var/lib/xen/xenpaging + xenpaging <guest_id> <number_of_pages> + +To obtain the guest_id, run 'xm list'. +xenpaging will write the pagefile to the current directory. +Example with 128MB pagefile on guest 1: + + xenpaging 1 32768 + +Caution: stopping xenpaging manually will cause the guest to stall or +crash because the paged-out memory is not written back into the guest! + +After a reboot of a guest, its guest_id changes, the current xenpaging +binary has no target anymore. To automate restarting of xenpaging after +guest reboot, specify the number if pages in the guest configuration +file /etc/xen/vm/<guest_name>: + +xenpaging=32768 + +Redo the guest with 'xm create /etc/xen/vm/<guest_name>' to activate the +changes. + + +Todo: +- implement stopping of xenpaging +- implement/test live migration + + +# vim: tw=72
Re: [PATCH 05/21] xenpaging: add signal handling
In reply to Olaf HeringOlaf, I haven't been looking at these patches as we've been going along, but there seem to be two things happening in this patch not mentioned in the description:
* Making xenpaging_teardown() not skip when a tear-down item fails, but continue to try to tear down the rest
* Making return values for the program as a whole (1 for initializing the paging, 2 for a failed file open)
These kinds of things should at least be mentioned in the description; and I would personally probably pull them out and put them in a separate patch.
-George
Quoted 180 lines · Olaf Hering
Leave paging loop if xenpaging gets a signal.
Remove paging file on exit.Signed-off-by Olaf Hering <[email protected]>---
v2:
unlink pagefile in signal handler to avoid stale pagefiles if xenpaging is stuck in some looptools/xenpaging/xenpaging.c | 42 +++++++++++++++++++++++++++++++++---------
1 file changed, 33 insertions(+), 9 deletions(-)tools/xenpaging/xenpaging.c +31
-9@@ -25,6 +25,7 @@ #include <stdlib.h> #include <stdarg.h> #include <time.h> +#include <signal.h> #include <xc_private.h> #include <xen/mem_event.h>@@ -43,6 +44,14 @@ #define DPRINTF(...) ((void)0) #endif +static char filename[80 ]; +static int interrupted; +static void close_handler (int sig) +{ + interrupted = sig; + if ( filename[0] ) + unlink(filename); +} static void *init_page(void) {@@ -248,7 +257,6 @@ int xenpaging_teardown(xc_interface *xch if ( rc != 0 ) { ERROR("Error tearing down domain paging in xen"); - goto err; } /* Unbind VIRQ */@@ -256,7 +264,6 @@ int xenpaging_teardown(xc_interface *xch if ( rc != 0 ) { ERROR("Error unbinding event port"); - goto err; } paging->mem_event.port = -1;@@ -265,7 +272,6 @@ int xenpaging_teardown(xc_interface *xch if ( rc != 0 ) { ERROR("Error closing event channel"); - goto err; } paging->mem_event.xce_handle = -1;@@ -274,7 +280,6 @@ int xenpaging_teardown(xc_interface *xch if ( rc != 0 ) { ERROR("Error closing connection to xen"); - goto err; } paging->xc_handle = NULL;@@ -380,7 +385,7 @@ int xenpaging_evict_page(xc_interface *x return ret; } -int xenpaging_resume_page (xenpaging_t *paging,mem_event_response_t *rsp) +static int xenpaging_resume_page (xenpaging_t *paging,mem_event_response_t *rsp) { int ret;@@ -461,6 +466,11 @@ static int evict_victim(xc_interface *xc goto out; } + if ( interrupted ) + { + ret = -EINTR; + goto out; + } ret = xc_mem_paging_nominate(paging->xc_handle, paging->mem_event.domain_id, victim->gfn); if ( ret == 0 )@@ -485,6 +495,7 @@ static int evict_victim(xc_interface *xc int main(int argc, char *argv[]) { + struct sigaction act; domid_t domain_id; int num_pages; xenpaging_t *paging;@@ -498,7 +509,6 @@ int main(int argc, char *argv[]) int open_flags = O_CREAT | O_TRUNC | O_RDWR; mode_t open_mode = S_IRUSR | S_IRGRP | S_IROTH | S_IWUSR | S_IWGRP | S_IWOTH; - char filename[80]; int fd; if ( argc != 3 )@@ -520,7 +530,7 @@ int main(int argc, char *argv[]) if ( paging == NULL ) { ERROR("Error initialising paging"); - goto out; + return 1; } /* Open file */@@ -529,9 +539,18 @@ int main(int argc, char *argv[]) if ( fd < 0 ) { perror("failed to open file"); - return -1; + return 2; } + /* ensure that if we get a signal, we'll do cleanup, then exit */ + act.sa_handler = close_handler; + act.sa_flags = 0; + sigemptyset(&act.sa_mask); + sigaction(SIGHUP, &act, NULL); + sigaction(SIGTERM, &act, NULL); + sigaction(SIGINT, &act, NULL); + sigaction(SIGALRM, &act, NULL); + /* Evict pages */ memset(victims, 0, sizeof(xenpaging_victim_t) * num_pages); for ( i = 0; i < num_pages; i++ )@@ -539,6 +558,8 @@ int main(int argc, char *argv[]) rc = evict_victim(xch, paging, domain_id, &victims[i], fd, i); if ( rc == -ENOSPC ) break; + if ( rc == -EINTR ) + break; if ( i % 100 == 0 ) DPRINTF("%d pages evicted\n", i); }@@ -546,7 +567,7 @@ int main(int argc, char *argv[]) DPRINTF("pages evicted\n"); /* Swap pages in and out */ - while ( 1 ) + while ( !interrupted ) { /* Wait for Xen to signal that a page needs paged in */ rc = xc_wait_for_event_or_timeout(xch, paging->mem_event.xce_handle, 100);@@ -637,8 +658,10 @@ int main(int argc, char *argv[]) } } } + DPRINTF("xenpaging got signal %d\n", interrupted);out:
+ close(fd);
free(victims);/* Tear down domain paging */
tools/xenpaging/xenpaging.c +1
-0@@ -651,6 +674,7 @@ int main(int argc, char *argv[]) xc_interface_close(xch); + DPRINTF("xenpaging exit code %d\n", rc); return rc; }_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Re: [PATCH 05/21] xenpaging: add signal handling
Quoted 3 lines · George Dunlap
Olaf, I haven't been looking at these patches as we've been going along, but there seem to be two things happening in this patch not mentioned in the description:
This was a "grown" patch.
Quoted 2 lines
* Making
xenpaging_teardown()not skip when a tear-down item fails, but continue to try to tear down the rest
If a domain is shutting down, xc_mem_event_disable will always fail because d->is_dying is checked. Thats why I removed the bail_out part.
Quoted 2 lines
* Making return values for the program as a whole (1 for initializing the paging, 2 for a failed file open)
return codes are currently not perfect, sometimes -1 is leaked. I think xenpaging should either return 0 or 1.
Quoted 3 lines
These kinds of things should at least be mentioned in the description; and I would personally probably pull them out and put them in a separate patch.
Will do better next time.
Thanks for the comment.
Olaf
Re: [PATCH 05/21] xenpaging: add signal handling
Yeah, they certainly seem like reasonable changes. They just need to be pointed out, so that people have an idea the actual impact of the patch (both now when accepting it, and later when going back trying to figure out where something broke).
FYI, I normally do a bunch of work (involving a large number of changes) first, then do "hg diff > working.diff ; hg update -C" and then go through working.diff to see where the individual changes are best suited, whether in a new patch, or in a modification to an existing patch. (emacs diff-mode is really helpful here.) That helps me review my own code, and promotes good patch hygiene. :-)
Peace,
-George
Quoted 36 lines · Olaf Hering
Quoted 3 lines · George Dunlap
Olaf, I haven't been looking at these patches as we've been going along, but there seem to be two things happening in this patch not mentioned in the description:
This was a "grown" patch.
Quoted 2 lines
* Making
xenpaging_teardown()not skip when a tear-down item fails, but continue to try to tear down the restIf a domain is shutting down,
xc_mem_event_disablewill always fail becaused->is_dyingis checked. Thats why I removed thebail_outpart.Quoted 2 lines
* Making return values for the program as a whole (1 for initializing the paging, 2 for a failed file open)
return codes are currently not perfect, sometimes -1 is leaked. I think xenpaging should either return 0 or 1.
Quoted 3 lines
These kinds of things should at least be mentioned in the description; and I would personally probably pull them out and put them in a separate patch.
Will do better next time.
Thanks for the comment.Olaf
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Olaf Hering writes ("[Xen-devel] [PATCH 00/21] xenpaging changes for xen-unstable"):
Quoted 3 lines
Here are some changes for xenpaging in xen-unstable.
Patches 1 to 14 are likely non-controversial and could be applied.
Having eyeballed them (but not in depth) I'd like to give notice that I intend to apply the tools patches in 1-14 unless someone objects.
Thanks,
Ian.
Re: [PATCH 21/21] xenpaging: (sparse) documenation
In reply to Olaf HeringOlaf Hering writes ("[Xen-devel] [PATCH 21/21] xenpaging: (sparse) documenation"):
Quoted 3 lines
Write up some sparse documentation about xenpaging usage.
Signed-off-by Olaf Hering <[email protected]>
Marvellous, thanks. I think this should be applied right away.
Ian.
Quoted 8 lines · Ian Jackson
Olaf Hering writes ("[Xen-devel] [PATCH 00/21] xenpaging changes for xen-unstable"):
Quoted 3 lines
Here are some changes for xenpaging in xen-unstable.
Patches 1 to 14 are likely non-controversial and could be applied.
Having eyeballed them (but not in depth) I'd like to give notice that I intend to apply the tools patches in 1-14 unless someone objects.
I think you are looking at a stale patch series. Olaf's latest runs 00-17, with patches 1-7 only considered by him suitable for immediate application.
K.
Quoted 7 lines
Thanks,
Ian._______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Quoted 6 lines · Keir Fraser
Quoted 4 lines · Ian Jackson
Olaf Hering writes ("[Xen-devel] [PATCH 00/21] xenpaging changes for xen-unstable"):
Quoted 2 lines
Here are some changes for xenpaging in xen-unstable.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Patches 1 to 14 are likely non-controversial and could be applied.
Having eyeballed them (but not in depth) I'd like to give notice that I intend to apply the tools patches in 1-14 unless someone objects.
I think you are looking at a stale patch series. Olaf's latest runs 00-17, with patches 1-7 only considered by him suitable for immediate application.
All of those patches are to tools/xenpaging, so I'll flush them from my queue and leave them to you.
-- Keir
Quoted 11 lines
K.
Quoted 7 lines
Thanks,
Ian._______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Keir Fraser writes ("Re: [Xen-devel] [PATCH 00/21] xenpaging changes for xen-unstable"):
Quoted 2 lines
I think you are looking at a stale patch series. Olaf's latest runs 00-17, with patches 1-7 only considered by him suitable for immediate application.
Oh, sorry. I hadn't seen the second series and I'm rather behind on the list.
Ian.
Re: [PATCH 18/21] xenpaging: prevent page-out of first 16MB
In reply to Olaf HeringQuoted 4 lines · Olaf Hering
This is more a workaround than a bugfix:
Don't page out first 16MB of memory.
When the BIOS does its initialization process and xenpaging removes pages, crashes will occour due to lack of support of xenpaging.
While looking at this again, I came up with this change. Any idea whats at 512K during BIOS startup?
Subject: xenpaging: prevent page-out of gfn 0x80
Add a workaround for missing handling of paged-out pages during BIOS startup. For some reason, only gfn 0x80 is affected.
(XEN) HVM3: HVM Loader
(XEN) traps.c:649:d3 Bad GMFN 80 (MFN ffffffffff) to MSR 40000000 (XEN) HVM3: Detected Xen v4.1
(XEN) HVM3: HVMLoader bug at util.c:604
(XEN) hvm.c:1085:d3 Triple fault on VCPU0 - invoking HVM system reset.
(XEN) HVM4: HVM Loader
(XEN) traps.c:649:d4 Bad GMFN 80 (MFN ffffffffff) to MSR 40000000 (XEN) io.c:194:d4 MMIO emulation failed @ 0018:9ffff: 00 6a 10 80 c4 82 (XEN) hvm.c:1085:d4 Triple fault on VCPU0 - invoking HVM system reset.
--- tools/xenpaging/policy_default.c | 3 +++ 1 file changed, 3 insertions(+)
tools/xenpaging/policy_default.c +3 -0
@@ -76,6 +76,9 @@ int policy_init(xenpaging_t *paging)/* Don't page out page 0 */ set_bit (0 , bitmap); +/* this is the trap page, disabled for the time being */ +set_bit (0x80 , bitmap); + out:return rc; }
Re: [PATCH 18/21] xenpaging: prevent page-out of first 16MB
Quoted 12 lines
Quoted 1 line
Quoted 1 line · Olaf Hering
Quoted 4 lines · Olaf Hering
This is more a workaround than a bugfix:
Don't page out first 16MB of memory.
When the BIOS does its initialization process and xenpaging removes pages, crashes will occour due to lack of support of xenpaging.While looking at this again, I came up with this change. Any idea whats at 512K during BIOS startup?
Quoted 8 lines
Subject: xenpaging: prevent page-out of gfn
0x80Add a workaround for missing handling of paged-out pages during BIOS startup.
For some reason, only gfn0x80is affected.(XEN) HVM3: HVM Loader
(XEN) traps.c:649:d3 Bad GMFN 80 (MFN ffffffffff) to MSR 40000000
This clearly is another place where page-in needs to be triggered. Isn't it the case that *any* gmfn_to_mfn() needs this special treatment? Couldn't therefore the translation function itself deal with requesting the page-in (and waiting for its completion, now that this can be done)?
Jan
Re: [PATCH 18/21] xenpaging: prevent page-out of first 16MB
Quoted 14 lines · Jan Beulich
Quoted 14 lines
Quoted 1 line
Quoted 1 line · Olaf Hering
Quoted 4 lines · Olaf Hering
This is more a workaround than a bugfix:
Don't page out first 16MB of memory.
When the BIOS does its initialization process and xenpaging removes pages, crashes will occour due to lack of support of xenpaging.While looking at this again, I came up with this change. Any idea whats at 512K during BIOS startup?
Subject: xenpaging: prevent page-out of gfn
0x80
Quoted 8 lines
Quoted 6 lines
Add a workaround for missing handling of paged-out pages during BIOS startup.
For some reason, only gfn0x80is affected.(XEN) HVM3: HVM Loader
(XEN) traps.c:649:d3 Bad GMFN 80 (MFN ffffffffff) to MSR 40000000This clearly is another place where page-in needs to be triggered.
Yes, thats true.
I had a printk in gfn_to_mfn for some months, and it did not trigger during my testing. But then, I also had the patch which keeps the first 16MB in memory.
So as it stands, gfn_to_mfn() should call p2m_mem_paging_populate() at least, even if that does not fix this crash.
I think the few gfn_to_mfn* variants should not return the paging types anymore, and just sleep until the page is back. Thats something for next year, as time runs out for me.
And: I think the various redefines of those functions/macros can go as well.
Olaf