4.5-rc4 kernel is failed to bootup on CN6880

DISCUSSION Yang Shi 4 participants 6 messages Feb 20, 2016 – Feb 22, 2016 MIPS

lore

Share & tools

lore

Yang Shi

author

2016-02-20 01:12 UTC #0 lore raw

Hi David,

I tried to boot 4.5-rc4 kernel on my CN6880 board, but it is failed at booting up secondary cores. The error is:

CPU31 revision is: 000d9101 (Cavium Octeon II)
SMP: Booting CPU32 (CoreId 32)...
Secondary boot timeout

I passed "numcores=32" in kernel commandline since there are 32 cores ion CN6880. And, the bootloader information is as below:

U-Boot 2013.07 ( (U-BOOT build: 104, SDK version: 3.1.1-544), svnversion: u-boot:107133M, exec:)-svn107117 (Build time: Oct 31 2014 - 19:39:37)

Skipping PCIe port 0 BIST, reset not done. (port not configured)
Skipping PCIe port 1 BIST, reset not done. (port not configured)
BIST check passed.
EBB6800 board revision major:2, minor:0, serial #: 2011-2.0-00120
OCTEON CN6880-AAP pass 1.1, Core clock: 1200 MHz, IO clock: 800 MHz, DDR clock: 667 MHz (1334 Mhz DDR) Base DRAM address used by u-boot: 0x20f000000, size: 0x1000000
DRAM: 8 GiB
Clearing DRAM...... done
NAND: 4096 MiB
Registered IDE device 0 from IDE bus:dev 0:0
Flash: 8 MiB
0:PCIe: Port 0 is unknown, skipping.
0:PCIe: Port 1 is unknown, skipping.
PCI console init succeeded, 1 consoles, 1024 bytes each

Net:   octmgmt0, octeth0, octeth1, octeth2, octeth3
Bus 0: OK

Device 0: Model: CF 1GB Firm: 20071116 Ser#: TSS20037110113081057
Type: Hard Disk
Capacity: 967.6 MB = 0.9 GB (1981728 x 512)
USB0: USB EHCI 1.00
scanning bus 0 for devices... 1 USB Device(s) found
Type the command 'usb start' to scan for USB storage devices.

Any hint is appreciated.

Thanks,
Yang

David Daney

2016-02-20 01:15 UTC #1 lore raw

Quoted 4 lines · Yang Shi

Hi David,
I tried to boot 4.5-rc4 kernel on my CN6880 board, but it is failed at booting up secondary cores. The error is:

Have you had luck with any other kernel.org versions?

FWIW, I recently posed some patches that may help. I haven't recently tested on cn68xx though, so I can't really say what it might be.

Quoted 42 lines

CPU31 revision is: 000d9101 (Cavium Octeon II)
SMP: Booting CPU32 (CoreId 32)...
Secondary boot timeout
I passed "numcores=32" in kernel commandline since there are 32 cores ion CN6880. And, the bootloader information is as below:
U-Boot 2013.07 ( (U-BOOT build: 104, SDK version: 3.1.1-544),
svnversion: u-boot:107133M, exec:)-svn107117 (Build time: Oct 31 2014 - 19:39:37)
Skipping PCIe port 0 BIST, reset not done. (port not configured) Skipping PCIe port 1 BIST, reset not done. (port not configured) BIST check passed.
EBB6800 board revision major:2, minor:0, serial #: 2011-2.0-00120 OCTEON CN6880-AAP pass 1.1, Core clock: 1200 MHz, IO clock: 800 MHz, DDR
clock: 667 MHz (1334 Mhz DDR)
Base DRAM address used by u-boot: 0x20f000000, size: 0x1000000
DRAM: 8 GiB
Clearing DRAM...... done
NAND: 4096 MiB
Registered IDE device 0 from IDE bus:dev 0:0
Flash: 8 MiB
0:PCIe: Port 0 is unknown, skipping.
0:PCIe: Port 1 is unknown, skipping.
PCI console init succeeded, 1 consoles, 1024 bytes each
Net:   octmgmt0, octeth0, octeth1, octeth2, octeth3
Bus 0: OK
Device 0: Model: CF 1GB Firm: 20071116 Ser#: TSS20037110113081057
Type: Hard Disk
Capacity: 967.6 MB = 0.9 GB (1981728 x 512)
USB0: USB EHCI 1.00
scanning bus 0 for devices... 1 USB Device(s) found
Type the command 'usb start' to scan for USB storage devices.
Any hint is appreciated.
Thanks,
Yang

Yang Shi

author

2016-02-20 01:19 UTC #2 lore raw

Quoted 7 lines · David Daney

Quoted 4 lines · Yang Shi
Hi David,
I tried to boot 4.5-rc4 kernel on my CN6880 board, but it is failed at booting up secondary cores. The error is:
Have you had luck with any other kernel.org versions?

Not tried yet. I will try to bisect it.

Quoted 2 lines

FWIW, I recently posed some patches that may help. I haven't recently tested on cn68xx though, so I can't really say what it might be.

Do you mean the change in CN78xx support patchset?

Thanks,
Yang

Quoted 46 lines

Quoted 44 lines
CPU31 revision is: 000d9101 (Cavium Octeon II)
SMP: Booting CPU32 (CoreId 32)...
Secondary boot timeout
I passed "numcores=32" in kernel commandline since there are 32 cores ion CN6880. And, the bootloader information is as below:
U-Boot 2013.07 ( (U-BOOT build: 104, SDK version: 3.1.1-544),
svnversion: u-boot:107133M, exec:)-svn107117 (Build time: Oct 31 2014 - 19:39:37)
Skipping PCIe port 0 BIST, reset not done. (port not configured) Skipping PCIe port 1 BIST, reset not done. (port not configured) BIST check passed.
EBB6800 board revision major:2, minor:0, serial #: 2011-2.0-00120 OCTEON CN6880-AAP pass 1.1, Core clock: 1200 MHz, IO clock: 800 MHz, DDR
clock: 667 MHz (1334 Mhz DDR)
Base DRAM address used by u-boot: 0x20f000000, size: 0x1000000
DRAM: 8 GiB
Clearing DRAM...... done
NAND: 4096 MiB
Registered IDE device 0 from IDE bus:dev 0:0
Flash: 8 MiB
0:PCIe: Port 0 is unknown, skipping.
0:PCIe: Port 1 is unknown, skipping.
PCI console init succeeded, 1 consoles, 1024 bytes each
Net:   octmgmt0, octeth0, octeth1, octeth2, octeth3
Bus 0: OK
Device 0: Model: CF 1GB Firm: 20071116 Ser#: TSS20037110113081057
Type: Hard Disk
Capacity: 967.6 MB = 0.9 GB (1981728 x 512)
USB0: USB EHCI 1.00
scanning bus 0 for devices... 1 USB Device(s) found
Type the command 'usb start' to scan for USB storage devices.
Any hint is appreciated.
Thanks,
Yang

Aaro Koskinen

2016-02-22 12:43 UTC #3 lore raw

In reply to Yang Shi

Hi,

Quoted 2 lines · Yang Shi

I tried to boot 4.5-rc4 kernel on my CN6880 board, but it is failed at booting up secondary cores. The error is:

With v4.5-rc5, EBB6800 is booting fine:

[    0.000000] CPU0 revision is: 000d9108 (Cavium Octeon II)
[...]
[ 2286.273935] SMP: Booting CPU01 (CoreId  1)...
[ 2286.278201] CPU1 revision is: 000d9108 (Cavium Octeon II)
[...]
[ 2287.214953] SMP: Booting CPU31 (CoreId 31)...
[ 2287.224668] CPU31 revision is: 000d9108 (Cavium Octeon II)
[ 2287.224865] Brought up 32 CPUs

Quoted 6 lines

CPU31 revision is: 000d9101 (Cavium Octeon II)
SMP: Booting CPU32 (CoreId 32)...
Secondary boot timeout
I passed "numcores=32" in kernel commandline since there are 32 cores ion CN6880.

You shouldn't have CPU32 in that case, the numbering starts from zero. Also the coremask is 32-bit.

I can reproduce your issue with CONFIG_NR_CPUS=64. Possibly this code is incorrect for NR_CPUS bigger than 32:

        /* The present CPUs get the lowest CPU numbers. */
        cpus = 1;
        for (id = 0; id < NR_CPUS; id++) {
                if ((id != coreid) && (core_mask & (1 << id))) {
                        set_cpu_possible(cpus, true);
                        set_cpu_present(cpus, true);

What CONFIG_NR_CPUS did you use?

Yang Shi

author

2016-02-22 19:15 UTC #4 lore raw

Quoted 28 lines · Aaro Koskinen

Hi,
Quoted 2 lines · Yang Shi
I tried to boot 4.5-rc4 kernel on my CN6880 board, but it is failed at booting up secondary cores. The error is:
With v4.5-rc5, EBB6800 is booting fine:
[    0.000000] CPU0 revision is: 000d9108 (Cavium Octeon II)
[...]
[ 2286.273935] SMP: Booting CPU01 (CoreId  1)...
[ 2286.278201] CPU1 revision is: 000d9108 (Cavium Octeon II)
[...]
[ 2287.214953] SMP: Booting CPU31 (CoreId 31)...
[ 2287.224668] CPU31 revision is: 000d9108 (Cavium Octeon II)
[ 2287.224865] Brought up 32 CPUs
Quoted 6 lines
CPU31 revision is: 000d9101 (Cavium Octeon II)
SMP: Booting CPU32 (CoreId 32)...
Secondary boot timeout
I passed "numcores=32" in kernel commandline since there are 32 cores ion CN6880.
You shouldn't have CPU32 in that case, the numbering starts from zero. Also the coremask is 32-bit.
I can reproduce your issue with CONFIG_NR_CPUS=64. Possibly this code is incorrect for NR_CPUS bigger than 32:

Quoted 8 lines

         /* The present CPUs get the lowest CPU numbers. */
         cpus = 1;
         for (id = 0; id < NR_CPUS; id++) {
                 if ((id != coreid) && (core_mask & (1 << id))) {
                         set_cpu_possible(cpus, true);
                         set_cpu_present(cpus, true);

What CONFIG_NR_CPUS did you use?

Thanks. I did have 48 NR_CPUS set. It works when I changed it to 32.

I think the problem is core_mask is 32 bit. But when NR_CPUS > 32, in "core_mask & (1 << id)" core_mask will be sign extended, then the statement will return non-zero all the time.

Yang

Quoted 3 lines

A.

David Daney

2016-02-22 20:56 UTC #5 lore raw

Quoted 37 lines · Yang Shi

Quoted 36 lines · Aaro Koskinen
Hi,
Quoted 2 lines · Yang Shi
I tried to boot 4.5-rc4 kernel on my CN6880 board, but it is failed at booting up secondary cores. The error is:
With v4.5-rc5, EBB6800 is booting fine:
[    0.000000] CPU0 revision is: 000d9108 (Cavium Octeon II)
[...]
[ 2286.273935] SMP: Booting CPU01 (CoreId  1)...
[ 2286.278201] CPU1 revision is: 000d9108 (Cavium Octeon II)
[...]
[ 2287.214953] SMP: Booting CPU31 (CoreId 31)...
[ 2287.224668] CPU31 revision is: 000d9108 (Cavium Octeon II)
[ 2287.224865] Brought up 32 CPUs
Quoted 7 lines
CPU31 revision is: 000d9101 (Cavium Octeon II)
SMP: Booting CPU32 (CoreId 32)...
Secondary boot timeout
I passed "numcores=32" in kernel commandline since there are 32 cores ion
CN6880.
You shouldn't have CPU32 in that case, the numbering starts from zero. Also the coremask is 32-bit.
I can reproduce your issue with CONFIG_NR_CPUS=64. Possibly this code is incorrect for NR_CPUS bigger than 32:
         /* The present CPUs get the lowest CPU numbers. */
         cpus = 1;
         for (id = 0; id < NR_CPUS; id++) {
                 if ((id != coreid) && (core_mask & (1 << id))) {
                         set_cpu_possible(cpus, true);
                         set_cpu_present(cpus, true);

Quoted 8 lines

Quoted 1 line
What CONFIG_NR_CPUS did you use?
Thanks. I did have 48 NR_CPUS set. It works when I changed it to 32.
I think the problem is core_mask is 32 bit. But when NR_CPUS > 32, in "core_mask & (1 << id)" core_mask will be sign extended, then the statement will return non-zero all the time.

That is correct, and perhaps not coincidentally, the reason I have already submitted patches to fix this.

David.

Quoted 7 lines

Yang
Quoted 3 lines
A.