cpuset: fix the problem that cpuset_mem_spread_node() returns an offline node

PATCH In review Miao Xie 2 participants 3 messages Mar 8, 2010 – Mar 9, 2010 Memory management

lore

Share & tools

lore

Miao Xie

author

2010-03-08 10:11 UTC #0 lore raw

PATCH v2 1/4 cpuset: fix the problem that cpuset_mem_spread_node() returns an offline node

Changes from V1 to V2:
- cleanup two unnecessary smp_wmb() at cpuset_migrate_mm()

cpuset_mem_spread_node() returns an offline node, and causes an oops.

This patch fixes it by initializing task->mems_allowed to node_states[N_HIGH_MEMORY], and updating task->mems_allowed when doing memory hotplug.

Signed-off-by Miao Xie <[email protected]>

---
 init/main.c      |    2 +-
 kernel/cpuset.c  |   16 ++++++++--------
 kernel/kthread.c |    2 +-
 3 files changed, 10 insertions(+), 10 deletions(-)

init/main.c +1 -1

index a1ab78c..cbead27 100644
@@ -858,7 +858,7 @@ static int __init kernel_init(void * unused)
 	/*
 	 * init can allocate pages on any node
 	 */
-	set_mems_allowed(node_possible_map);
+	set_mems_allowed(node_states[N_HIGH_MEMORY]);
 	/*
 	 * init can run on any cpu.
 	 */

kernel/cpuset.c +8 -8

index ba401fa..9c5bb4f 100644
@@ -920,9 +920,6 @@ static int update_cpumask(struct cpuset *cs, struct cpuset *trialcs,
  *    call to guarantee_online_mems(), as we know no one is changing
  *    our task's cpuset.
  *
- *    Hold callback_mutex around the two modifications of our tasks
- *    mems_allowed to synchronize with cpuset_mems_allowed().
- *
  *    While the mm_struct we are migrating is typically from some
  *    other task, the task_struct mems_allowed that we are hacking
  *    is for our current task, which must allocate new pages for that

@@ -1391,11 +1388,10 @@ static void cpuset_attach(struct cgroup_subsys *ss, struct cgroup *cont,
 
 	if (cs == &top_cpuset) {
 		cpumask_copy(cpus_attach, cpu_possible_mask);
-		to = node_possible_map;
 	} else {
 		guarantee_online_cpus(cs, cpus_attach);
-		guarantee_online_mems(cs, &to);
 	}
+	guarantee_online_mems(cs, &to);
 
 	/* do per-task migration stuff possibly for each in the threadgroup */
 	cpuset_attach_task(tsk, &to, cs);

@@ -2090,15 +2086,19 @@ static int cpuset_track_online_cpus(struct notifier_block *unused_nb,
 static int cpuset_track_online_nodes(struct notifier_block *self,
 				unsigned long action, void *arg)
 {
+	nodemask_t oldmems;
+
 	cgroup_lock();
 	switch (action) {
 	case MEM_ONLINE:
-	case MEM_OFFLINE:
+		oldmems = top_cpuset.mems_allowed;
 		mutex_lock(&callback_mutex);
 		top_cpuset.mems_allowed = node_states[N_HIGH_MEMORY];
 		mutex_unlock(&callback_mutex);
-		if (action == MEM_OFFLINE)
-			scan_for_empty_cpusets(&top_cpuset);
+		update_tasks_nodemask(&top_cpuset, &oldmems, NULL);
+		break;
+	case MEM_OFFLINE:
+		scan_for_empty_cpusets(&top_cpuset);
 		break;
 	default:
 		break;

kernel/kthread.c +1 -1

index 82ed0ea..83911c7 100644
@@ -219,7 +219,7 @@ int kthreadd(void *unused)
 	set_task_comm(tsk, "kthreadd");
 	ignore_signals(tsk);
 	set_cpus_allowed_ptr(tsk, cpu_all_mask);
-	set_mems_allowed(node_possible_map);
+	set_mems_allowed(node_states[N_HIGH_MEMORY]);
 
 	current->flags |= PF_NOFREEZE | PF_FREEZER_NOSIG;

Signature · 8 lines

1.6.5.2


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

David Rientjes

2010-03-08 21:22 UTC #1 lore raw

Quoted 3 lines · Miao Xie

Changes from V1 to V2:
- cleanup two unnecessary smp_wmb() at cpuset_migrate_mm()

This patch is already in -mm without this update, so it's probably better to make this an incremental series basedo n mmotm-2010-03-04-18-05 or later.

Quoted 13 lines

kernel/cpuset.c +3 -1

@@ -2090,15 +2086,19 @@ static int cpuset_track_online_cpus(struct notifier_block *unused_nb,
 static int cpuset_track_online_nodes(struct notifier_block *self,
 				unsigned long action, void *arg)
 {
+	nodemask_t oldmems;
+
 	cgroup_lock();
 	switch (action) {
 	case MEM_ONLINE:
-	case MEM_OFFLINE:
+		oldmems = top_cpuset.mems_allowed;
 		mutex_lock(&callback_mutex);
 		top_cpuset.mems_allowed = node_states[N_HIGH_MEMORY];

kernel/cpuset.c static int cpuset_track_online_cpus(struct notifier_block *unused_nb, · line 2104

Quoted 10 lines

kernel/cpuset.c +4 -2

 		mutex_unlock(&callback_mutex);
-		if (action == MEM_OFFLINE)
-			scan_for_empty_cpusets(&top_cpuset);
+		update_tasks_nodemask(&top_cpuset, &oldmems, NULL);
+		break;
+	case MEM_OFFLINE:
+		scan_for_empty_cpusets(&top_cpuset);
 		break;
 	default:
 		break;

This looks wrong, why isn't top_cpuset.mems_allowed updated for MEM_OFFLINE? If you're going to update it when a new node comes online for (struct memory_notify *)arg->status_change_nid is >= 0, then it should be removed from the nodemask when offlined as well. You'd be calling scan_for_empty_cpusets() needlessly in this code since it'll never change under your hotplug code.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Miao Xie

author

2010-03-09 01:25 UTC #2 lore raw

Quoted 9 lines · David Rientjes

Quoted 3 lines · Miao Xie
Changes from V1 to V2:
- cleanup two unnecessary smp_wmb() at cpuset_migrate_mm()
This patch is already in -mm without this update, so it's probably better to make this an incremental series basedo n mmotm-2010-03-04-18-05 or later.

ok, I'll do it.

Quoted 23 lines

kernel/cpuset.c +7 -3

@@ -2090,15 +2086,19 @@ static int cpuset_track_online_cpus(struct notifier_block *unused_nb,
 static int cpuset_track_online_nodes(struct notifier_block *self,
 				unsigned long action, void *arg)
 {
+	nodemask_t oldmems;
+
 	cgroup_lock();
 	switch (action) {
 	case MEM_ONLINE:
-	case MEM_OFFLINE:
+		oldmems = top_cpuset.mems_allowed;
 		mutex_lock(&callback_mutex);
 		top_cpuset.mems_allowed = node_states[N_HIGH_MEMORY];
 		mutex_unlock(&callback_mutex);
-		if (action == MEM_OFFLINE)
-			scan_for_empty_cpusets(&top_cpuset);
+		update_tasks_nodemask(&top_cpuset, &oldmems, NULL);
+		break;
+	case MEM_OFFLINE:
+		scan_for_empty_cpusets(&top_cpuset);
 		break;
 	default:

Quoted 8 lines

Review static int cpuset_track_online_cpus(struct notifier_block *unused_nb, · line 2104
Quoted 1 line
(quoted diff) +0 -0
 		break;
This looks wrong, why isn't top_cpuset.mems_allowed updated for MEM_OFFLINE? If you're going to update it when a new node comes online for (struct memory_notify *)arg->status_change_nid is >= 0, then it should be removed from the nodemask when offlined as well. You'd be calling scan_for_empty_cpusets() needlessly in this code since it'll never change under your hotplug code.

scan_for_empty_cpusets() will update top_cpuset.mems_allowed when doing MEM_OFFLINE.

The comment of this source is necessary. I'll add it.

Thanks!
Miao

Quoted 2 lines