Skip to contentSkip to conversation

cpu_down: move migrate_enable() back

PATCH Applied Tiejun Chen 3 participants 4 messages Nov 7, 2013 – Nov 16, 2013 CPU hotplug

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 cpu_down: move migrate_enable() back

Commit 08c1ab68, "hotplug-use-migrate-disable.patch", intends to use migrate_enable()/migrate_disable() to replace that combination of preempt_enable() and preempt_disable(), but actually in !CONFIG_PREEMPT_RT_FULL case, migrate_enable()/migrate_disable() are still equal to preempt_enable()/preempt_disable(). So that followed cpu_hotplug_begin()/cpu_unplug_begin(cpu) would go schedule() to trigger schedule_debug() like this:

_cpu_down()

	|
	+ migrate_disable() = preempt_disable()
	|
	+ cpu_hotplug_begin() or cpu_unplug_begin()
		|
		+ schedule()
			|
			+ __schedule()
				|
				+ preempt_disable();
				|
				+ __schedule_bug() is true!

So we should move migrate_enable() as the original scheme.

---
 kernel/cpu.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/cpu.c +1 -1
index 98f2ea3..da6e128 100644
@@ -594,6 +594,7 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 		err = -EBUSY;
 		goto restore_cpus;
 	}
+	migrate_enable();
 
 	cpu_hotplug_begin();
 	err = cpu_unplug_begin(cpu);
@@ -647,7 +648,6 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 out_release:
 	cpu_unplug_done(cpu);
 out_cancel:
-	migrate_enable();
 	cpu_hotplug_done();
 	if (!err)
 		cpu_notify_nofail(CPU_POST_DEAD | mod, hcpu);
Signature · 1 line
1.7.9.5
#1 lore raw
Quoted 2 lines · Tiejun Chen

Commit 08c1ab68, "hotplug-use-migrate-disable.patch", intends to use migrate_enable()/migrate_disable() to replace that combination

Since you quote a commit ID, it would be nice if you mentioned that it is in v3.8-rt from the stable-rt repo, instead of making people look it up.

Also, since you have cc'd stable-rt@vger, there should be some mention of what versions it might be appropriate for (3.4? 3.10?).

Quoted 20 lines

of preempt_enable() and preempt_disable(), but actually in !CONFIG_PREEMPT_RT_FULL case, migrate_enable()/migrate_disable() are still equal to preempt_enable()/preempt_disable(). So that followed cpu_hotplug_begin()/cpu_unplug_begin(cpu) would go schedule() to trigger schedule_debug() like this:

_cpu_down()

	|
	+ migrate_disable() = preempt_disable()
	|
	+ cpu_hotplug_begin() or cpu_unplug_begin()
		|
		+ schedule()
			|
			+ __schedule()
				|
				+ preempt_disable();
				|
				+ __schedule_bug() is true!
Quoted 1 line

So we should move migrate_enable() as the original scheme.

It is unclear to me what context you are thinking of/referencing as the "original scheme"...

Quoted 11 lines
---
 kernel/cpu.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/cpu.c +0 -0
index 98f2ea3..da6e128 100644

kernel/cpu.c static int __ref _cpu_down(unsigned int cpu, int tasks_frozen) · line 598

Quoted 5 lines
kernel/cpu.c +1 -0
@@ -594,6 +594,7 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 		err = -EBUSY;
 		goto restore_cpus;
 	}
+	migrate_enable();
 

So, what happens if we now get migrated right here, before the hotplug_begin call below?

P.
--

Quoted 11 lines
kernel/cpu.c +0 -1
 	cpu_hotplug_begin();
 	err = cpu_unplug_begin(cpu);
@@ -647,7 +648,6 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 out_release:
 	cpu_unplug_done(cpu);
 out_cancel:
-	migrate_enable();
 	cpu_hotplug_done();
 	if (!err)
 		cpu_notify_nofail(CPU_POST_DEAD | mod, hcpu);
#2 lore raw
Quoted 20 lines · * Paul Gortmaker | 2013-11-11 10:32:55 [-0500]:
Quoted 20 lines

of preempt_enable() and preempt_disable(), but actually in !CONFIG_PREEMPT_RT_FULL case, migrate_enable()/migrate_disable() are still equal to preempt_enable()/preempt_disable(). So that followed cpu_hotplug_begin()/cpu_unplug_begin(cpu) would go schedule() to trigger schedule_debug() like this:

_cpu_down()

	|
	+ migrate_disable() = preempt_disable()
	|
	+ cpu_hotplug_begin() or cpu_unplug_begin()
		|
		+ schedule()
			|
			+ __schedule()
				|
				+ preempt_disable();
				|
				+ __schedule_bug() is true!
Quoted 4 lines
Quoted 1 line

So we should move migrate_enable() as the original scheme.

It is unclear to me what context you are thinking of/referencing as the "original scheme"...

I think to what happening before the patch he quoted was applied.

Quoted 11 lines

kernel/cpu.c static int __ref _cpu_down(unsigned int cpu, int tasks_frozen) · line 598

Quoted 8 lines
kernel/cpu.c +1 -0
@@ -594,6 +594,7 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 		err = -EBUSY;
 		goto restore_cpus;
 	}
+	migrate_enable();
 

So, what happens if we now get migrated right here, before the hotplug_begin call below?

It should not matter. Before this we call set_cpus_allowed_ptr() to ensure that current() may run on any CPU except that one we try to bring down. We call later smp_processor_id() to check if this is correct. The only reason for the preempt_disable() seems to be to keep smp_processor_id() quiet and not spill a warning.
Later on we could switch CPUs as long as we don't get on that CPU that is going down since we don't hold / access any per CPU data. The only reason where set_cpus_allowed_ptr() might not work as expected is when user land changes the mask because we don't have PF_NO_SETAFFINITY set.

Quoted 1 line

P.

Sebastian

#3 lore raw
In reply to Tiejun Chen
Quoted 22 lines · * Tiejun Chen | 2013-11-07 10:06:07 [+0800]:

Commit 08c1ab68, "hotplug-use-migrate-disable.patch", intends to use migrate_enable()/migrate_disable() to replace that combination of preempt_enable() and preempt_disable(), but actually in !CONFIG_PREEMPT_RT_FULL case, migrate_enable()/migrate_disable() are still equal to preempt_enable()/preempt_disable(). So that followed cpu_hotplug_begin()/cpu_unplug_begin(cpu) would go schedule() to trigger schedule_debug() like this:

_cpu_down()

	|
	+ migrate_disable() = preempt_disable()
	|
	+ cpu_hotplug_begin() or cpu_unplug_begin()
		|
		+ schedule()
			|
			+ __schedule()
				|
				+ preempt_disable();
				|
				+ __schedule_bug() is true!
Quoted 3 lines

So we should move migrate_enable() as the original scheme.

applied.

Sebastian