Skip to contentSkip to conversation

CPU hotplug boot change for IA64

Linux-ia64 David Mosberger 5 participants 10 messages Aug 2, 2002 – Aug 8, 2002

lore
Share & tools
lore
#0 lore raw

What's the reason for putting cpu_initialize in setup.c instead of smp.c? Also, I much prefer to have all global variables declared in header files; especially for special things that are marked __init or __devinit, etc. Nothing worse than someone accidentally trying to use the variable after its gone. (Yes, there are other offenders, but that's no reason to add to the problem.)

	--david
#1 lore raw

Hi David,

Sorry, I missed to remove that. :-(
cpu_initialized (or something like that) is needed when CPU Hot Plug is implemented.
__cpu_up() must detect that the CPU has already been initialized and but offlined or the CPU hasn't been initialized. But for 2.5.29, such the variable isn't necessary.
And as you pointed out, it should not be in setup.c. I'll update the patch and send it later.

Regards,
Kimi

Quoted 6 lines · On Fri, 2 Aug 2002 14:44:16 -0700 David Mosberger <[email protected]>…

What's the reason for putting cpu_initialize in setup.c instead of smp.c? Also, I much prefer to have all global variables declared in header files; especially for special things that are marked __init or __devinit, etc. Nothing worse than someone accidentally trying to use the variable after its gone. (Yes, there are other offenders, but that's no reason to add to the problem.)

Signature · 1 line
Kimio Suganuma <[email protected]>
#2 lore raw
In reply to David Mosberger

Hi David,

I removed cpu_initialized and fixed several bugs. I tested SMP and UP kernels on 2-way Bigsur. Please let me know if there is any problem on the patch.

Thanks,
Kimi

Quoted 8 lines · On Fri, 02 Aug 2002 15:18:01 -0700 Kimio Suganuma…

Sorry, I missed to remove that. :-(
cpu_initialized (or something like that) is needed when CPU Hot Plug is implemented.
__cpu_up() must detect that the CPU has already been initialized and but offlined or the CPU hasn't been initialized. But for 2.5.29, such the variable isn't necessary.
And as you pointed out, it should not be in setup.c. I'll update the patch and send it later.

Signature · 1 line
Kimio Suganuma <[email protected]>
#3 lore raw
In reply to David Mosberger

In message <[email protected]> you write:

Quoted 11 lines

Bit for CPU#0 on cpu_online_map was set at smp_boot_cpus() before. But on the 2.5.29, place of calling smp_boot_cpus() is changed to later timing and this seems to make a chance that smp_call_function() is called before setting cpu_online_map, (I guess.)
In that case, smp_call_function won't return because num_online_cpus() returns 0 and the system will hang.

I have an experience of this problem only on a large system like Tiger, but it may occur on another system.

Rusty, I guess this modification should be needed for i386, too.

You should *not* be calling smp_call_function before smp_prepare_cpus.

I know kgdb can do this, but it's a special case.

Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

#4 lore raw
In reply to David Mosberger

Hi Rusty,

Quoted 5 lines · On Sat, 03 Aug 2002 13:29:02 +1000 Rusty Russell <[email protected]> wrote:
Quoted 1 line

Rusty, I guess this modification should be needed for i386, too.

You should *not* be calling smp_call_function before smp_prepare_cpus.

I know kgdb can do this, but it's a special case.

OK.
I cannot remember who was calling smp_call_function, but I think it might be kdb. I'm using kdb for debugging in usual.

Thanks,
Kimi

Signature · 1 line
Kimio Suganuma <[email protected]>
#5 lore raw
In reply to David Mosberger

In message <[email protected]> you write:

Quoted 7 lines

Hi Rusty,

Quoted 2 lines · On Sat, 03 Aug 2002 13:29:02 +1000 Rusty Russell <[email protected]> wrote:
Quoted 1 line

Rusty, I guess this modification should be needed for i386, too.

Quoted 7 lines
Quoted 3 lines

You should *not* be calling smp_call_function before smp_prepare_cpus.

I know kgdb can do this, but it's a special case.

OK.
I cannot remember who was calling smp_call_function, but I think it might be kdb. I'm using kdb for debugging in usual.

Yes, in which case the fix I asked Andrew Morton to use was:

	/* Before smp_prepare_cpus, cpu_online not usable */
	if (num_online_cpus() > 1)
		smp_call_function(...)

Cheers,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

#6 lore raw
In reply to David Mosberger
Quoted 17 lines · Rusty Russell

In message <[email protected]> you write:

Quoted 7 lines

Hi Rusty,

Quoted 2 lines · On Sat, 03 Aug 2002 13:29:02 +1000 Rusty Russell <[email protected]> wrote:
Quoted 1 line

Rusty, I guess this modification should be needed for i386, too.

Quoted 7 lines
Quoted 3 lines

You should *not* be calling smp_call_function before smp_prepare_cpus.

I know kgdb can do this, but it's a special case.

OK.
I cannot remember who was calling smp_call_function, but I think it might be kdb. I'm using kdb for debugging in usual.

Quoted 5 lines

Yes, in which case the fix I asked Andrew Morton to use was:

        /* Before smp_prepare_cpus, cpu_online not usable */
        if (num_online_cpus() > 1)
                smp_call_function(...)

It works, too. http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.30/kgdb.patch

#7 lore raw
In reply to David Mosberger

Hi Kimi,

Quoted 5 lines · Kimio Suganuma

Hi David,

I removed cpu_initialized and fixed several bugs. I tested SMP and UP kernels on 2-way Bigsur. Please let me know if there is any problem on the patch.

The patch looks fine. Before applying it to 2.5.29 my BigSur ended up with no tasks beeing scheduled to cpu#1 and no migration_task and ksoftirqd for the same cpu. But the system came up. After applying it the system hangs in the boot sequence right after "Freeing unused kernel memory". But CPU#1 was woken up correctly. Sorry, have no kdb to be able to tell you more...

Does anybody else have these problems?

Thanks,
Erich

#8 lore raw
In reply to David Mosberger

Hi Erich,

Quoted 4 lines · On Tue, 6 Aug 2002 20:33:09 +0200 Erich Focht <[email protected]> wrote:
Quoted 3 lines

I removed cpu_initialized and fixed several bugs. I tested SMP and UP kernels on 2-way Bigsur. Please let me know if there is any problem on the patch.

Quoted 8 lines

The patch looks fine. Before applying it to 2.5.29 my BigSur ended up with no tasks beeing scheduled to cpu#1 and no migration_task and ksoftirqd for the same cpu. But the system came up. After applying it the system hangs in the boot sequence right after "Freeing unused kernel memory". But CPU#1 was woken up correctly. Sorry, have no kdb to be able to tell you more...

Does anybody else have these problems?

I had a problem that the system hang at booting on 2.5.29 without my patch. After that, I turned off CONFIGs as much as possible and then I succeeded to boot the kernel.
I suspect IDE/ATAPI driver or something was causing the problem (I'm not sure, though.)
Anyway, I'd like to reproduce the problem. Could you send me your .config?

Regards,
Kimi

Signature · 1 line
Kimio Suganuma <[email protected]>
#9 lore raw
In reply to David Mosberger

Hi,

Quoted 10 lines · On Tue, 6 Aug 2002 20:33:09 +0200 Erich Focht <[email protected]> wrote:

The patch looks fine. Before applying it to 2.5.29 my BigSur ended up with no tasks beeing scheduled to cpu#1 and no migration_task and ksoftirqd for the same cpu.

But the system came up. After applying it
the system hangs in the boot sequence right after "Freeing unused kernel memory". But CPU#1 was woken up correctly. Sorry, have no kdb to be able to tell you more...

Does anybody else have these problems?

ksoftirqd and migration_thread are spawned in cpu_up() in kernel/cpu.c. But, cpu_up() wasn't called because the cpu had already been running at that time (see arch/ia64/kernel/smpboot.c and init/main.c.) I tried to change init/main.c to call cpu_up() for APs, and I made sure that the kernel caused the system hangs as you reported. This means that 2.5.29 kernel is not stable on SMP regardless of CPU Hot Plug patch applying.
I found that kudzu(1) and shutdown of ethernet may cause system hangs but I'm not sure why.

Regards,
Kimi

Signature · 1 line
Kimio Suganuma <[email protected]>