i.MX6 Solo: maxcpus=1 vs. nosmp: Warning from smp_twd.c?

Hi,

working on a i.MX6 Solo SoC (one Cortex A9 core) we are usually booting with maxcpus=1 a kernel built with CONFIG_NR_CPUS=4. This is done to be able to boot the same kernel on the Solo and Quad version of that SoC.

Now, we tried to switch from 'maxcpus=1' to 'nosmp'. With 'nosmp' we get the warning [1] in the boot messages. This isn't there with 'maxcpus=1'.

Is it a good idea to use 'nosmp' instead of 'maxcpus=1' on a single Cortex A9 SoC?

Looking into the warning, it's the result of setup_max_cpus being 0 in case of 'nosmp' (and being 1 in case of 'maxcpus=1'). So with 'nosmp' the function local_timer_register()

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=blob;f=arch/arm/kernel/smp.c;h=84f4cbf652e58b3acfea846ad1cad9ddbee72d95;hb=refs/heads/master#l514

returns -ENXIO (== -6).

Is this the intended behaviour for 'nosmp' resulting in below boot warning?

The call stack is

    imx6q_timer_init()

-> twd_local_timer_of_register()()
-> twd_local_timer_common_register()
-> local_timer_register() (returns -ENXIO due to setup_max_cpus == 0)

Best regards

Dirk

[1]
...
sched_clock: 32 bits at 49MHz, resolution 20ns, wraps every 86767ms

------------[ cut here ]------------
WARNING: at arch/arm/kernel/smp_twd.c:345 
twd_local_timer_of_register+0x7c/0x90()
twd_local_timer_of_register failed (-6)
Modules linked in:
Backtrace:
[<80011f14>] (dump_backtrace+0x0/0x10c) from [<8044dd30>] 
(dump_stack+0x18/0x1c)
  r7:805e9f58 r6:805ba84c r5:80539331 r4:00000159
[<8044dd18>] (dump_stack+0x0/0x1c) from [<80020fbc>] 
(warn_slowpath_common+0x54/0x6c)
[<80020f68>] (warn_slowpath_common+0x0/0x6c) from [<80021078>] 
(warn_slowpath_fmt+0x38/0x40)
  r9:412fc09a r8:8fffffff r7:ffffffff r6:00000001 r5:80633b8c

r4:80b32da8
[<80021040>] (warn_slowpath_fmt+0x0/0x40) from [<805ba84c>] (twd_local_timer_of_register+0x7c/0x90)
r3:fffffffa r2:8053934b
[<805ba7d0>] (twd_local_timer_of_register+0x0/0x90) from [<805c0bec>] (imx6q_timer_init+0x18/0x4c)
r5:80633800 r4:8053b701
[<805c0bd4>] (imx6q_timer_init+0x0/0x4c) from [<805ba4e8>] (time_init+0x28/0x38)
r5:80633800 r4:805dc0f4
[<805ba4c0>] (time_init+0x0/0x38) from [<805b6854>] (start_kernel+0x1a0/0x310)
[<805b66b4>] (start_kernel+0x0/0x310) from [<10008044>] (0x10008044)
r8:1000406a r7:805f3f8c r6:805dc0c4 r5:805f0518 r4:10c5387d

---[ end trace 1b75b31a2719ed1c ]---

CPU identified as i.MX6S/DL, silicon rev 1.0
...

And what happens if you compile a kernel without SMP support and boot it on this SoC?

- Case 1:

Kernel command line: ... nosmp ...

$ cat .config | grep SMP
CONFIG_USE_GENERIC_SMP_HELPERS=y
CONFIG_GENERIC_SMP_IDLE_THREAD=y
CONFIG_HAVE_SMP=y
CONFIG_SMP=y
CONFIG_SMP_ON_UP=y

=> Gives warning [1] as reported

- Case 2:

Kernel command line: ... nosmp ...

$ cat .config | grep SMP
CONFIG_BROKEN_ON_SMP=y
CONFIG_GENERIC_SMP_IDLE_THREAD=y
CONFIG_HAVE_SMP=y
# CONFIG_SMP is not set

=> Boots without warning

Best regards

Dirk

[1]

...
sched_clock: 32 bits at 49MHz, resolution 20ns, wraps every 86767ms

------------[ cut here ]------------
WARNING: at arch/arm/kernel/smp_twd.c:345 
twd_local_timer_of_register+0x7c/0x90()
twd_local_timer_of_register failed (-6)
Modules linked in:
Backtrace:
[<80011f14>] (dump_backtrace+0x0/0x10c) from [<8044dd30>] 
(dump_stack+0x18/0x1c)
  r7:805e9f58 r6:805ba84c r5:80539331 r4:00000159
[<8044dd18>] (dump_stack+0x0/0x1c) from [<80020fbc>] 
(warn_slowpath_common+0x54/0x6c)
[<80020f68>] (warn_slowpath_common+0x0/0x6c) from [<80021078>] 
(warn_slowpath_fmt+0x38/0x40)
  r9:412fc09a r8:8fffffff r7:ffffffff r6:00000001 r5:80633b8c

r4:80b32da8
[<80021040>] (warn_slowpath_fmt+0x0/0x40) from [<805ba84c>] (twd_local_timer_of_register+0x7c/0x90)
r3:fffffffa r2:8053934b
[<805ba7d0>] (twd_local_timer_of_register+0x0/0x90) from [<805c0bec>] (imx6q_timer_init+0x18/0x4c)
r5:80633800 r4:8053b701
[<805c0bd4>] (imx6q_timer_init+0x0/0x4c) from [<805ba4e8>] (time_init+0x28/0x38)
r5:80633800 r4:805dc0f4
[<805ba4c0>] (time_init+0x0/0x38) from [<805b6854>] (start_kernel+0x1a0/0x310)
[<805b66b4>] (start_kernel+0x0/0x310) from [<10008044>] (0x10008044)
r8:1000406a r7:805f3f8c r6:805dc0c4 r5:805f0518 r4:10c5387d

---[ end trace 1b75b31a2719ed1c ]---

CPU identified as i.MX6S/DL, silicon rev 1.0
...

Thanks Dirk for bringing this up on the list. We have hit this fat warning on FSL internal 3.5.7 kernel tree when booting SMP kernel with nosmp on kernel cmdline.

As what Dirk confirmed, we do not see the warning with kernel built with !SMP.

Russell,

The following is an immediate and less invasive change to lower the warning level. If you agree, I will post a patch and then put it into patch system.

Shawn

index c092115..f796b83 100644
@@ -392,6 +392,6 @@ void __init twd_local_timer_of_register(void)
        err = twd_local_timer_common_register(np);
 
 out:
-       WARN(err, "twd_local_timer_of_register failed (%d)\n", err);
+       pr_warn("twd_local_timer_of_register failed (%d)\n", err);
 }
 #endif

Thanks for reminding. I will send it out today.

Shawn

Hi Dirk,

maxcpus=1 and nosmp are not equivalent as per the comment from kernel/smp.c:

/*
* Setup routine for controlling SMP activation
*
* Command-line option of "nosmp" or "maxcpus=0" will disable SMP
* activation entirely (the MPS table probe still happens, though). *
* Command-line option of "maxcpus=<NUM>", where <NUM> is an integer
* greater than 0, limits the maximum number of CPUs activated in
* SMP mode to <NUM>.
*/

void __weak arch_disable_smp_support(void) { }

Yes, understood that both options are not equal.

But should both options work on a single ARM Cortex A9 SoC without a warning? Or should one of the both options be preferred on a single ARM Cortex A9 SoC?

Thanks

Dirk