Skip to contentSkip to conversation

Pagedir specific routines.

PATCH New Nigel Cunningham 1 participant 10 messages Jun 26, 2006

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Pagedir specific routines.

Routines used in the allocation of extra memory for atomic copies, calculating which pages belong in each part of the image and finding usable pages when resuming.

#1 lore raw

PATCH 1/9 Pagedir.c header.

Add the header for pagedir functions - that is, functions specifically related to the image metadata.

 kernel/power/pagedir.c |   33 +++++++++++++++++++++++++++++++++
 1 files changed, 33 insertions(+), 0 deletions(-)
kernel/power/pagedir.c new file +33 -0
new file mode 100644
index 0000000..1e0995e
@@ -0,0 +1,33 @@
+/*
+ * kernel/power/pagedir.c
+ *
+ * Copyright (C) 1998-2001 Gabor Kuti <[email protected]>
+ * Copyright (C) 1998,2001,2002 Pavel Machek <[email protected]>
+ * Copyright (C) 2002-2003 Florent Chabaud <[email protected]>
+ * Copyright (C) 2002-2006 Nigel Cunningham <[email protected]>
+ *
+ * This file is released under the GPLv2.
+ *
+ * Routines for handling pagesets.
+ * Note that pbes aren't actually stored as such. They're stored as
+ * bitmaps and extents.
+ */
+
+#include <linux/suspend.h>
+#include <linux/highmem.h>
+#include <linux/bootmem.h>
+#include <linux/hardirq.h>
+#include <linux/sched.h>
+
+#include "pageflags.h"
+#include "ui.h"
+#include "pagedir.h"
+
+int extra_pagedir_pages_allocated;
+
+struct extras {
+	struct page *page;
+	int order;
+	struct extras *next;
+} *extras_list;
+

--
Nigel Cunningham nigel at suspend2 dot net

#2 lore raw

PATCH 2/9 Free extra memory allocated for the atomic copy.

In reply to Nigel Cunningham

If the page cache is not large enough to store the atomic copy, we allocate extra pages. This routine is used at cleanup to free that memory.

 kernel/power/pagedir.c |   23 +++++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)
kernel/power/pagedir.c +23 -0
index 1e0995e..9c26990 100644
@@ -31,3 +31,26 @@ struct extras {
 	struct extras *next;
 } *extras_list;
 
+/* suspend_free_extra_pagedir_memory
+ *
+ * Description:	Free previously allocated extra pagedir memory.
+ */
+void suspend_free_extra_pagedir_memory(void)
+{
+	/* Free allocated pages */
+	while (extras_list) {
+		struct extras *this = extras_list;
+		int i;
+
+		extras_list = this->next;
+
+		for (i = 0; i < (1 << this->order); i++)
+			ClearPageNosave(this->page + i);
+
+		__free_pages(this->page, this->order);
+		kfree(this);
+	}
+
+	extra_pagedir_pages_allocated = 0;
+}
+

--
Nigel Cunningham nigel at suspend2 dot net

#3 lore raw

PATCH 3/9 Allocate extra memory for the atomic copy.

In reply to Nigel Cunningham

Allocate extra memory needed for an atomic copy when the page cache is too small to store it.

 kernel/power/pagedir.c |   75 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 75 insertions(+), 0 deletions(-)
kernel/power/pagedir.c +75 -0
index 9c26990..76da2e5 100644
@@ -54,3 +54,78 @@ void suspend_free_extra_pagedir_memory(v
 	extra_pagedir_pages_allocated = 0;
 }
 
+/* suspend_allocate_extra_pagedir_memory
+ *
+ * Description:	Allocate memory for making the atomic copy of pagedir1 in the
+ * 		case where it is bigger than pagedir2.
+ * Arguments:	struct pagedir *: 	The pagedir for which we should 
+ * 					allocate memory.
+ * 		int:			Size of pageset 1.
+ * 		int:			Size of pageset 2.
+ * Result:	int. Zero on success. One if unable to allocate enough memory.
+ */
+int suspend_allocate_extra_pagedir_memory(struct pagedir *p, int pageset_size,
+		int alloc_from)
+{
+	int num_to_alloc = pageset_size - alloc_from - extra_pagedir_pages_allocated;
+	int j, order, num_added = 0;
+
+	if (num_to_alloc < 1)
+		num_to_alloc = 0;
+
+	if (num_to_alloc) {
+		order = fls(num_to_alloc);
+		if (order >= MAX_ORDER)
+			order = MAX_ORDER - 1;
+
+		while (num_added < num_to_alloc) {
+			struct page *newpage;
+			unsigned long virt;
+			struct extras *extras_entry;
+			
+			while ((1 << order) > (num_to_alloc - num_added))
+				order--;
+
+			virt = __get_free_pages(GFP_ATOMIC | __GFP_NOWARN, order);
+			while ((!virt) && (order > 0)) {
+				order--;
+				virt = __get_free_pages(GFP_ATOMIC | __GFP_NOWARN, order);
+			}
+
+			if (!virt) {
+				p->pageset_size += num_added;
+				extra_pagedir_pages_allocated += num_added;
+				return 1;
+			}
+
+			newpage = virt_to_page(virt);
+
+			extras_entry = (struct extras *) kmalloc(sizeof(struct extras), GFP_ATOMIC);
+
+			if (!extras_entry) {
+				__free_pages(newpage, order);
+				extra_pagedir_pages_allocated += num_added;
+				return 1;
+			}
+
+			extras_entry->page = newpage;
+			extras_entry->order = order;
+			extras_entry->next = NULL;
+
+			if (extras_list)
+				extras_entry->next = extras_list;
+
+			extras_list = extras_entry;
+
+			for (j = 0; j < (1 << order); j++) {
+				SetPageNosave(newpage + j);
+				SetPagePageset1Copy(newpage + j);
+			}
+			num_added+= (1 << order);
+		}
+	}
+
+	extra_pagedir_pages_allocated += num_added;
+	return 0;
+}
+

--
Nigel Cunningham nigel at suspend2 dot net

#4 lore raw

PATCH 4/9 Mark a task as pageset 1.

In reply to Nigel Cunningham

Mark a task's pages as for the atomic copy, even when they would normally be saved as LRU pages. This is used for the userspace helpers, which need to keep running during suspend, and not have their data overwritten by the atomic copy.

 kernel/power/pagedir.c |   49 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 49 insertions(+), 0 deletions(-)
kernel/power/pagedir.c +49 -0
index 76da2e5..b7fc3bc 100644
@@ -129,3 +129,52 @@ int suspend_allocate_extra_pagedir_memor
 	return 0;
 }
 
+/*
+ * suspend_mark_task_as_pageset1
+ * Functionality   : Marks all the pages belonging to a given process as
+ *                   pageset 1 pages.
+ * Called From     : pagedir.c - mark_pages_for_pageset2
+ *
+ */
+extern struct page *suspend2_follow_page(struct mm_struct *mm, unsigned long address);
+
+void suspend_mark_task_as_pageset1(struct task_struct *t)
+{
+	struct vm_area_struct *vma;
+	struct mm_struct *mm;
+
+	mm = t->active_mm;
+
+	if (!mm || !mm->mmap) return;
+
+	/* Don't try to take the sem when processes are frozen, 
+	 * drivers are suspended and irqs are disabled. We're
+	 * not racing with anything anyway.  */
+	BUG_ON(in_atomic() && !irqs_disabled());
+
+	if (!irqs_disabled())
+		down_read(&mm->mmap_sem);
+	
+	for (vma = mm->mmap; vma; vma = vma->vm_next) {
+		if (vma->vm_flags & VM_PFNMAP)
+			continue;
+		if (vma->vm_start) {
+			unsigned long posn;
+			for (posn = vma->vm_start; posn < vma->vm_end;
+					posn += PAGE_SIZE) {
+				struct page *page = 
+					suspend2_follow_page(mm, posn);
+				if (page) {
+					ClearPagePageset2(page);
+					SetPagePageset1(page);
+				}
+			}
+		}
+	}
+
+	BUG_ON(in_atomic() && !irqs_disabled());
+
+	if (!irqs_disabled())
+		up_read(&mm->mmap_sem);
+}
+

--
Nigel Cunningham nigel at suspend2 dot net

#5 lore raw

PATCH 5/9 Mark pages for pageset 2.

In reply to Nigel Cunningham

Reset the contents of pageset 2: clear the map, mark LRU pages as being in, then remove userspace helpers' pages.

 kernel/power/pagedir.c |   92 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 92 insertions(+), 0 deletions(-)
kernel/power/pagedir.c +92 -0
index b7fc3bc..c23ca58 100644
@@ -178,3 +178,95 @@ void suspend_mark_task_as_pageset1(struc
 		up_read(&mm->mmap_sem);
 }
 
+/* mark_pages_for_pageset2
+ *
+ * Description:	Mark unshared pages in processes not needed for suspend as
+ * 		being able to be written out in a separate pagedir.
+ * 		HighMem pages are simply marked as pageset2. They won't be
+ * 		needed during suspend.
+ */
+
+struct attention_list {
+	struct task_struct *task;
+	struct attention_list *next;
+};
+
+void suspend_mark_pages_for_pageset2(void)
+{
+	struct zone *zone;
+	struct task_struct *p;
+	struct attention_list *attention_list = NULL, *last = NULL;
+	unsigned long flags;
+
+	BUG_ON(in_atomic() && !irqs_disabled());
+
+	clear_dyn_pageflags(pageset2_map);
+
+	if (test_action_state(SUSPEND_NO_PAGESET2))
+		return;
+
+	/* 
+	 * Note that we don't clear the map to begin with!
+	 * This is because if we eat memory, we loose track
+	 * of LRU pages that are still in use but taken off
+	 * the LRU. If I can figure out how the VM keeps
+	 * track of them, I might be able to tweak this a
+	 * little further and decrease pageset one's size
+	 * further.
+	 *
+	 * (Memory grabbing clears the pageset2 flag on
+	 * pages that are really freed!).
+	 */
+	
+	for_each_zone(zone) {
+		spin_lock_irqsave(&zone->lru_lock, flags);
+		if (zone->nr_inactive) {
+			struct page *page;
+			list_for_each_entry(page, &zone->inactive_list, lru)
+				SetPagePageset2(page);
+		}
+		if (zone->nr_active) {
+			struct page *page;
+			list_for_each_entry(page, &zone->active_list, lru)
+				SetPagePageset2(page);
+		}
+		spin_unlock_irqrestore(&zone->lru_lock, flags);
+	}
+
+	BUG_ON(in_atomic() && !irqs_disabled());
+
+	/* Now we find all userspace process (with task->mm) marked PF_NOFREEZE
+	 * and move them into pageset1.
+	 */
+	read_lock(&tasklist_lock);
+	for_each_process(p)
+		if ((p->flags & PF_NOFREEZE) || p == current) {
+			struct attention_list *this = kmalloc(sizeof(struct attention_list), GFP_ATOMIC);
+			BUG_ON(!this);
+			this->task = p;
+			this->next = NULL;
+			if (attention_list) {
+				last->next = this;
+				last = this;
+			} else
+				attention_list = last = this;
+		}
+	read_unlock(&tasklist_lock);
+
+	BUG_ON(in_atomic() && !irqs_disabled());
+
+	/* Because the tasks in attention_list are ones related to suspending,
+	 * we know that they won't go away under us.
+	 */
+
+	while (attention_list) {
+		suspend_mark_task_as_pageset1(attention_list->task);
+		last = attention_list;
+		attention_list = attention_list->next;
+		kfree(last);
+	}
+
+	BUG_ON(in_atomic() && !irqs_disabled());
+
+}
+

--
Nigel Cunningham nigel at suspend2 dot net

#6 lore raw

PATCH 6/9 Get nonconflicting page.

In reply to Nigel Cunningham

In preparation for the atomic restore, get a page that can safely be used during the restore - ie, that isn't a location of a page that will be restored.

 kernel/power/pagedir.c |   18 ++++++++++++++++++
 1 files changed, 18 insertions(+), 0 deletions(-)
kernel/power/pagedir.c +18 -0
index c23ca58..4cdc0ef 100644
@@ -270,3 +270,21 @@ void suspend_mark_pages_for_pageset2(voi
 
 }
 
+/* suspend_get_nonconflicting_page
+ *
+ * Description: Gets order zero pages that won't be overwritten
+ *		while copying the original pages.
+ */
+
+unsigned long suspend_get_nonconflicting_page(void)
+{
+	struct page *page;
+
+	do {
+		page = alloc_pages(GFP_ATOMIC | __GFP_NOWARN | __GFP_ZERO, 0);
+		BUG_ON(!page);
+	} while(PagePageset1(page));
+
+	return (unsigned long) page_address(page);
+}
+

--
Nigel Cunningham nigel at suspend2 dot net

#7 lore raw

PATCH 7/9 Relocate a piece of memory if required.

In reply to Nigel Cunningham

Given an address and size (never more than a page and never overlapping page boundaries), relocate the data to an address that can be safely used during the atomic restore if that relocation is necessary. The memory to be freed might be slab or a normal page.

 kernel/power/pagedir.c |   20 ++++++++++++++++++++
 1 files changed, 20 insertions(+), 0 deletions(-)
kernel/power/pagedir.c +20 -0
index 4cdc0ef..ec75e79 100644
@@ -288,3 +288,23 @@ unsigned long suspend_get_nonconflicting
 	return (unsigned long) page_address(page);
 }
 
+/* relocate_page_if_required
+ *
+ * Description: Given the address of a pointer to a page, we check if the page
+ * 		needs relocating and do so if needs be, adjusting the pointer
+ * 		too.
+ */
+
+void suspend_relocate_if_required(unsigned long *current_value, unsigned int size)
+{
+	if (PagePageset1(virt_to_page(*current_value))) {
+		unsigned long new_page = suspend_get_nonconflicting_page();
+		memcpy((char *) new_page, (char *) *current_value, size);
+		if (PageSlab(virt_to_page(*current_value)))
+			kfree((void *) *current_value);
+		else
+			free_page((unsigned long) *current_value);
+		*current_value = new_page;
+	}
+}
+

--
Nigel Cunningham nigel at suspend2 dot net

#8 lore raw

PATCH 8/9 Get pageset1 load addresses.

In reply to Nigel Cunningham

Get the addresses into which the atomically copied memory should loaded prior to restoring it. They need to be locations that won't be overwritten in the process.

 kernel/power/pagedir.c |   26 ++++++++++++++++++++++++++
 1 files changed, 26 insertions(+), 0 deletions(-)
kernel/power/pagedir.c +26 -0
index ec75e79..daf498a 100644
@@ -308,3 +308,29 @@ void suspend_relocate_if_required(unsign
 	}
 }
 
+/* get_pageset1_load_addresses
+ * 
+ * Description: We check here that pagedir & pages it points to won't collide
+ * 		with pages where we're going to restore from the loaded pages
+ * 		later.
+ * Returns:	Zero on success, one if couldn't find enough pages (shouldn't
+ * 		happen).
+ */
+
+int suspend_get_pageset1_load_addresses(void)
+{
+	int i, result = 0;
+	void *this;
+
+	for(i=0; i < pagedir1.pageset_size; i++) {
+		this = (void *) suspend_get_nonconflicting_page();
+		if (!this) {
+			abort_suspend("Error: Ran out of memory seeking locations for reloading data.");
+			result = 1;
+			break;
+		}
+		SetPagePageset1Copy(virt_to_page(this));
+	}
+
+	return result;
+}

--
Nigel Cunningham nigel at suspend2 dot net

#9 lore raw

PATCH 9/9 Header file for pagedir functions.

In reply to Nigel Cunningham

kernel/power/pagedir.h declares structures and functions related to pagedirs. A pagedir is a set of pages that are being stored in part of the image. It used to be a far more substantial structure, containing a list of pages. Since the advent of bitmaps, it just has the size. Pagedir.c contains the routines for allocating and freeing memory when pagedir1 is larger than pageset2 (generally only init S), marking which pages belong to which pageset, and for obtaining pages that can safely be used during the atomic copy.

 kernel/power/pagedir.h |   37 +++++++++++++++++++++++++++++++++++++
 1 files changed, 37 insertions(+), 0 deletions(-)
kernel/power/pagedir.h new file +37 -0
new file mode 100644
index 0000000..608b29a
@@ -0,0 +1,37 @@
+/*
+ * kernel/power/pagedir.h
+ *
+ * Copyright (C) 2004-2006 Nigel Cunningham <[email protected]>
+ *
+ * This file is released under the GPLv2.
+ *
+ * Declarations for routines for handling pagesets.
+ */
+
+/* Pagedir
+ *
+ * Contains the metadata for a set of pages saved in the image.
+ */
+
+struct pagedir {
+	long pageset_size;
+	long lastpageset_size;
+};
+
+extern struct pagedir pagedir1, pagedir2;
+
+extern void suspend_copy_pageset1(void);
+
+extern void suspend_free_extra_pagedir_memory(void);
+
+extern int suspend_allocate_extra_pagedir_memory(struct pagedir *p, int pageset_size, int alloc_from);
+
+extern void suspend_mark_task_as_pageset1 (struct task_struct *t);
+extern void suspend_mark_pages_for_pageset2(void);
+
+extern void suspend_relocate_if_required(unsigned long *current_value, unsigned int size);
+extern int suspend_get_pageset1_load_addresses(void);
+
+extern int extra_pagedir_pages_allocated;
+
+extern unsigned long suspend_get_nonconflicting_page(void);

--
Nigel Cunningham nigel at suspend2 dot net