Skip to contentSkip to conversation

mainline u-boot was stuck before booting kernel

U-Boot Ziyuan Xu 2 participants 8 messages Jul 25, 2016 – Jul 26, 2016 Arm (32-bit)

lore
Share & tools
lore

Ziyuan Xu

author
#0 lore raw

Hi All,

I'm sorry to tell you that I failed to boot linux kernel with the mainline u-boot on rk3288 board(both evb-rk3288 & fennec-rk3288). It was stuck in cleanup_before_linux() before jumping to linux, and the boot_stage_flag is BOOTSTAGE_ID_BOOTM_HANDOFF.

## Current stack ends at 0x7df638b0 * kernel: cmdline image address = 0x02000000
## No init Ramdisk

    ramdisk start = 0x00000000, ramdisk end = 0x00000000

## No Flattened Device Tree
Continuing to boot without FDT
Initial value for argc=3
Final value for argc=3
using: ATAGS
## Transferring control to Linux (at address 02000000)...

Starting kernel ...

With the further investigation, it never returnned back from invalidate_dcache_all(). I mean that I can't reach stage 4 as below.

cleanup_before_linux

     ==>cleanup_before_linux_select
         ==>1.disable_interrupts
         ==>2.dcache_disable
         ==>3.invalidate_dcache_all
         ==>4.icache_disable

Debug further, invalidate_dcache_all invalidate all cache one-by-one which cache type is DATA_ONLY, INSTRUCTION_DATA or UNIFIED. And invalidate way from one set to another set in order. The problem is that the PC ran away in invalidate way loop [cache level L1!!!].

I add some serial output code in __v7_flush_dcache_all to figure out the bog.
I expect:
Print the value of r9 in sequence, e.g 0x7f, 0x7e, 0x7d..... 0x01, 0x00 (hex)
In fact, print the value of r9 in sequence at first, but print
unexpected value afterwards. e.g 0x7f, 0x7e, 0x7d, ..,0x73, 0x40,
0x85,....

ENTRY(__v7_flush_dcache_all)
[snip]
loop1:
mov r9, r7 @ create working copy of max index
loop2:

+      stmfd    sp!, {r0}
+      ldr     r0, =0xff690000
+      str     r9, [r0]
+      ldmfd    sp!, {r0}
  ARM(    orr    r11, r10, r4, lsl r5    )    @ factor way and cache 
number into r11
  THUMB(    lsl    r6, r4, r5        )
  THUMB(    orr    r11, r10, r6        )    @ factor way and cache 
number into r11
  ARM(    orr    r11, r11, r9, lsl r2    )    @ factor index number into r11
  THUMB(    lsl    r6, r9, r2        )
  THUMB(    orr    r11, r11, r6        )    @ factor index number into r11
     mcr    p15, 0, r11, c7, c14, 2        @ clean & invalidate by set/way

subs r9, r9, #1 @ decrement the index
bge loop2
subs r4, r4, #1 @ decrement the way
bge loop1
skip:
ENDPROC(__v7_flush_dcache_all)

I don't have the jtag, hence I can't address the current pc. I have no doubt that if any glitches in __v7_flush_dcache_all, I reviewed several times, also copy kernel's implement to here. :-( No effect.
A more interesting thing is that Sandy had report it. He and I have similar problem. Everything work fine after I applied his patches, or disable dcache(active CONFIG_SYS_DCACHE_OFF).
@Stany, I'm sorry that I disable dcache during hack.. That was a mistake:-(

@Simon & hieko,
Can you boot linux with the mainline u-boot? have a try?

author
#1 lore raw

hi Stany,

The difference is that you print out assertion log.

Reset not supported on this platform
### ERROR ### Please RESET the board ###

You can add show_boot_progress() function in your bsp file to find out something.
#define CONFIG_SHOW_BOOT_PROGRESS
void show_boot_progress(int progress)

{
     printf("Boot reached stage %d\n", progress);
}
Quoted 90 lines · Ziyuan Xu

Hi All,

I'm sorry to tell you that I failed to boot linux kernel with the mainline u-boot on rk3288 board(both evb-rk3288 & fennec-rk3288). It was stuck in cleanup_before_linux() before jumping to linux, and the boot_stage_flag is BOOTSTAGE_ID_BOOTM_HANDOFF.

## Current stack ends at 0x7df638b0 * kernel: cmdline image address = 0x02000000
## No init Ramdisk
ramdisk start = 0x00000000, ramdisk end = 0x00000000
## No Flattened Device Tree
Continuing to boot without FDT
Initial value for argc=3
Final value for argc=3
using: ATAGS
## Transferring control to Linux (at address 02000000)...

Starting kernel ...

With the further investigation, it never returnned back from invalidate_dcache_all(). I mean that I can't reach stage 4 as below.

cleanup_before_linux

    ==>cleanup_before_linux_select
        ==>1.disable_interrupts
        ==>2.dcache_disable
        ==>3.invalidate_dcache_all
        ==>4.icache_disable

Debug further, invalidate_dcache_all invalidate all cache one-by-one which cache type is DATA_ONLY, INSTRUCTION_DATA or UNIFIED. And invalidate way from one set to another set in order. The problem is that the PC ran away in invalidate way loop [cache level L1!!!].

I add some serial output code in __v7_flush_dcache_all to figure out the bog.
I expect:
Print the value of r9 in sequence, e.g 0x7f, 0x7e, 0x7d..... 0x01, 0x00 (hex)
In fact, print the value of r9 in sequence at first, but print
unexpected value afterwards. e.g 0x7f, 0x7e, 0x7d, ..,0x73, 0x40,
0x85,....

ENTRY(__v7_flush_dcache_all)
[snip]
loop1:
mov r9, r7 @ create working copy of max index
loop2:

(quoted diff) +4 -0
+      stmfd    sp!, {r0}
+      ldr     r0, =0xff690000
+      str     r9, [r0]
+      ldmfd    sp!, {r0}
 ARM(    orr    r11, r10, r4, lsl r5    )    @ factor way and cache 

number into r11

 THUMB(    lsl    r6, r4, r5        )
 THUMB(    orr    r11, r10, r6        )    @ factor way and cache 
number into r11
 ARM(    orr    r11, r11, r9, lsl r2    )    @ factor index number 
into r11
 THUMB(    lsl    r6, r9, r2        )
 THUMB(    orr    r11, r11, r6        )    @ factor index number into r11
    mcr    p15, 0, r11, c7, c14, 2        @ clean & invalidate by set/way

subs r9, r9, #1 @ decrement the index
bge loop2
subs r4, r4, #1 @ decrement the way
bge loop1
skip:
ENDPROC(__v7_flush_dcache_all)

I don't have the jtag, hence I can't address the current pc. I have no doubt that if any glitches in __v7_flush_dcache_all, I reviewed several times, also copy kernel's implement to here. :-( No effect. A more interesting thing is that Sandy had report it. He and I have similar problem. Everything work fine after I applied his patches, or disable dcache(active CONFIG_SYS_DCACHE_OFF).
@Stany, I'm sorry that I disable dcache during hack.. That was a mistake:-(

@Simon & hieko,
Can you boot linux with the mainline u-boot? have a try?

_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

#2 lore raw

Ah, thanks. Your debugging looks the same as what I've seen from printf debugging. I'll try to verify though.

Quoted 112 lines · Ziyuan Xu

hi Stany,

The difference is that you print out assertion log.

Reset not supported on this platform
### ERROR ### Please RESET the board ###

You can add show_boot_progress() function in your bsp file to find out something.
#define CONFIG_SHOW_BOOT_PROGRESS
void show_boot_progress(int progress)

{
    printf("Boot reached stage %d\n", progress);

}

Quoted 90 lines · Ziyuan Xu

Hi All,

I'm sorry to tell you that I failed to boot linux kernel with the mainline u-boot on rk3288 board(both evb-rk3288 & fennec-rk3288). It was stuck in cleanup_before_linux() before jumping to linux, and the boot_stage_flag is BOOTSTAGE_ID_BOOTM_HANDOFF.

## Current stack ends at 0x7df638b0 * kernel: cmdline image address = 0x02000000
## No init Ramdisk
ramdisk start = 0x00000000, ramdisk end = 0x00000000
## No Flattened Device Tree
Continuing to boot without FDT
Initial value for argc=3
Final value for argc=3
using: ATAGS
## Transferring control to Linux (at address 02000000)...

Starting kernel ...

With the further investigation, it never returnned back from invalidate_dcache_all(). I mean that I can't reach stage 4 as below.

cleanup_before_linux

    ==>cleanup_before_linux_select
        ==>1.disable_interrupts
        ==>2.dcache_disable
        ==>3.invalidate_dcache_all
        ==>4.icache_disable

Debug further, invalidate_dcache_all invalidate all cache one-by-one which cache type is DATA_ONLY, INSTRUCTION_DATA or UNIFIED. And invalidate way from one set to another set in order. The problem is that the PC ran away in invalidate way loop [cache level L1!!!].

I add some serial output code in __v7_flush_dcache_all to figure out the bog.
I expect:
Print the value of r9 in sequence, e.g 0x7f, 0x7e, 0x7d..... 0x01, 0x00 (hex)
In fact, print the value of r9 in sequence at first, but print unexpected
value afterwards. e.g 0x7f, 0x7e, 0x7d, ..,0x73, 0x40, 0x85,....

ENTRY(__v7_flush_dcache_all)
[snip]
loop1:
mov r9, r7 @ create working copy of max index
loop2:

(quoted diff) +4 -0
+      stmfd    sp!, {r0}
+      ldr     r0, =0xff690000
+      str     r9, [r0]
+      ldmfd    sp!, {r0}
 ARM(    orr    r11, r10, r4, lsl r5    )    @ factor way and cache

number into r11

 THUMB(    lsl    r6, r4, r5        )
 THUMB(    orr    r11, r10, r6        )    @ factor way and cache number
into r11
 ARM(    orr    r11, r11, r9, lsl r2    )    @ factor index number into
r11
 THUMB(    lsl    r6, r9, r2        )
 THUMB(    orr    r11, r11, r6        )    @ factor index number into r11
    mcr    p15, 0, r11, c7, c14, 2        @ clean & invalidate by set/way

subs r9, r9, #1 @ decrement the index
bge loop2
subs r4, r4, #1 @ decrement the way
bge loop1
skip:
ENDPROC(__v7_flush_dcache_all)

I don't have the jtag, hence I can't address the current pc. I have no doubt that if any glitches in __v7_flush_dcache_all, I reviewed several times, also copy kernel's implement to here. :-( No effect. A more interesting thing is that Sandy had report it. He and I have similar problem. Everything work fine after I applied his patches, or disable dcache(active CONFIG_SYS_DCACHE_OFF).
@Stany, I'm sorry that I disable dcache during hack.. That was a mistake:-(

@Simon & hieko,
Can you boot linux with the mainline u-boot? have a try?

_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

author
#3 lore raw

Dear All,

I add the ISB operation after dcache_disable(), and I can jump to linux kernel entry.:-)

arch/arm/cpu/armv7/cpu.c +1 -0
index 6eac5ef..5cc09ba 100644
@@ -43,6 +43,7 @@ int cleanup_before_linux_select(int flags)
                 */
                 dcache_disable();
                 v7_outer_cache_disable();
+               ISB;
 
                 /*
                 * After D-cache is flushed and before it is disabled 

there may

Sounds crazy. In fact, there is already an 'ISB' operation in set_cr() which to disable dcache and MMU. But it just tell processor that memory had change, not real ISB for armv7.

dcache_disable

         ==>flush_dcache_all()
         ==>set_cr()   disable dcache and MMU.

#define isb() __asm__ __volatile__ ("" : : : "memory") static inline void set_cr(unsigned int val)

{
     if (is_hyp())
         asm volatile("mcr p15, 4, %0, c1, c0, 0    @ set CR" :
                                   : "r" (val)
                                   : "cc");
     else
         asm volatile("mcr p15, 0, %0, c1, c0, 0    @ set CR" :
                                   : "r" (val)
                                   : "cc");
     isb();
}

I also find something in ARM cortex-A17 TRM. ==>"Disable the MMU from the each processor followed by an ISB to ensure the MMU disable operation is complete, then followed by a DSB to drain previous memory transactions."

In my humble opinion, maybe instructions tlb had alter, and cause running away??? I'm not sure about it.
@Sandy, could you have a try with my update?

@Simon, did you hit this glitch? I hope you can help ...:-)

Quoted 126 lines · Sandy Patterson

Ah, thanks. Your debugging looks the same as what I've seen from printf debugging. I'll try to verify though.

On Mon, Jul 25, 2016 at 10:58 AM, Ziyuan Xu <[email protected] <mailto:[email protected]>> wrote:

hi Stany,

The difference is that you print out assertion log.

Reset not supported on this platform
### ERROR ### Please RESET the board ###

You can add show_boot_progress() function in your bsp file to find out something.
#define CONFIG_SHOW_BOOT_PROGRESS
void show_boot_progress(int progress)
{
printf("Boot reached stage %d\n", progress);

}

On 2016?07?25? 22:12, Ziyuan Xu wrote:

Hi All,

I'm sorry to tell you that I failed to boot linux kernel with the mainline u-boot on rk3288 board(both evb-rk3288 & fennec-rk3288). It was stuck in cleanup_before_linux() before jumping to linux, and the boot_stage_flag is BOOTSTAGE_ID_BOOTM_HANDOFF.

## Current stack ends at 0x7df638b0 * kernel: cmdline image address = 0x02000000
## No init Ramdisk
ramdisk start = 0x00000000, ramdisk end = 0x00000000
## No Flattened Device Tree
Continuing to boot without FDT
Initial value for argc=3
Final value for argc=3
using: ATAGS
## Transferring control to Linux (at address 02000000)...

Starting kernel ...

With the further investigation, it never returnned back from invalidate_dcache_all(). I mean that I can't reach stage 4 as below.

cleanup_before_linux
==>cleanup_before_linux_select
==>1.disable_interrupts
==>2.dcache_disable
==>3.invalidate_dcache_all
==>4.icache_disable

Debug further, invalidate_dcache_all invalidate all cache one-by-one which cache type is DATA_ONLY, INSTRUCTION_DATA or UNIFIED. And invalidate way from one set to another set in order. The problem is that the PC ran away in invalidate way loop [cache level L1!!!].

I add some serial output code in __v7_flush_dcache_all to figure out the bog.
I expect:
Print the value of r9 in sequence, e.g 0x7f, 0x7e, 0x7d..... 0x01, 0x00 (hex)
In fact, print the value of r9 in sequence at first, but print
unexpected value afterwards. e.g 0x7f, 0x7e, 0x7d, ..,0x73,
0x40, 0x85,....

ENTRY(__v7_flush_dcache_all)
[snip]
loop1:
mov r9, r7 @ create working copy of max
index
loop2:
+ stmfd sp!, {r0}
+ ldr r0, =0xff690000
+ str r9, [r0]
+ ldmfd sp!, {r0}
ARM( orr r11, r10, r4, lsl r5 ) @ factor way and
cache number into r11
THUMB( lsl r6, r4, r5 )
THUMB( orr r11, r10, r6 ) @ factor way and
cache number into r11
ARM( orr r11, r11, r9, lsl r2 ) @ factor index
number into r11
THUMB( lsl r6, r9, r2 )
THUMB( orr r11, r11, r6 ) @ factor index
number into r11
mcr p15, 0, r11, c7, c14, 2 @ clean & invalidate
by set/way
subs r9, r9, #1 @ decrement the index
bge loop2
subs r4, r4, #1 @ decrement the way
bge loop1
skip:
ENDPROC(__v7_flush_dcache_all)

I don't have the jtag, hence I can't address the current pc. I have no doubt that if any glitches in __v7_flush_dcache_all, I reviewed several times, also copy kernel's implement to here. :-( No effect.
A more interesting thing is that Sandy had report it. He and I have similar problem. Everything work fine after I applied his patches, or disable dcache(active CONFIG_SYS_DCACHE_OFF). @Stany, I'm sorry that I disable dcache during hack.. That was a mistake:-(

@Simon & hieko,
Can you boot linux with the mainline u-boot? have a try?

_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de <mailto:[email protected]>
http://lists.denx.de/mailman/listinfo/u-boot

author
#4 lore raw

+Simon and heiko

Quoted 185 lines · Ziyuan Xu

Dear All,

I add the ISB operation after dcache_disable(), and I can jump to linux kernel entry.:-)

arch/arm/cpu/armv7/cpu.c +1 -0
index 6eac5ef..5cc09ba 100644
@@ -43,6 +43,7 @@ int cleanup_before_linux_select(int flags)
                */
                dcache_disable();
                v7_outer_cache_disable();
+               ISB;
 
                /*
                * After D-cache is flushed and before it is disabled 

there may

Sounds crazy. In fact, there is already an 'ISB' operation in set_cr() which to disable dcache and MMU. But it just tell processor that memory had change, not real ISB for armv7.

dcache_disable

        ==>flush_dcache_all()
        ==>set_cr()   disable dcache and MMU.

#define isb() __asm__ __volatile__ ("" : : : "memory") static inline void set_cr(unsigned int val)

{
    if (is_hyp())
        asm volatile("mcr p15, 4, %0, c1, c0, 0    @ set CR" :
                                  : "r" (val)
                                  : "cc");
    else
        asm volatile("mcr p15, 0, %0, c1, c0, 0    @ set CR" :
                                  : "r" (val)
                                  : "cc");
    isb();
}

I also find something in ARM cortex-A17 TRM. ==>"Disable the MMU from the each processor followed by an ISB to ensure the MMU disable operation is complete, then followed by a DSB to drain previous memory transactions."

In my humble opinion, maybe instructions tlb had alter, and cause running away??? I'm not sure about it.
@Sandy, could you have a try with my update?

@Simon, did you hit this glitch? I hope you can help ...:-)

Quoted 111 lines · Sandy Patterson

Ah, thanks. Your debugging looks the same as what I've seen from printf debugging. I'll try to verify though.

On Mon, Jul 25, 2016 at 10:58 AM, Ziyuan Xu <[email protected] <mailto:[email protected]>> wrote:

hi Stany,

The difference is that you print out assertion log.

Reset not supported on this platform
### ERROR ### Please RESET the board ###

You can add show_boot_progress() function in your bsp file to find out something.
#define CONFIG_SHOW_BOOT_PROGRESS
void show_boot_progress(int progress)
{
printf("Boot reached stage %d\n", progress);

}

On 2016?07?25? 22:12, Ziyuan Xu wrote:

Hi All,

I'm sorry to tell you that I failed to boot linux kernel with the mainline u-boot on rk3288 board(both evb-rk3288 & fennec-rk3288). It was stuck in cleanup_before_linux() before jumping to linux, and the boot_stage_flag is BOOTSTAGE_ID_BOOTM_HANDOFF.

## Current stack ends at 0x7df638b0 * kernel: cmdline image address = 0x02000000
## No init Ramdisk
ramdisk start = 0x00000000, ramdisk end = 0x00000000
## No Flattened Device Tree
Continuing to boot without FDT
Initial value for argc=3
Final value for argc=3
using: ATAGS
## Transferring control to Linux (at address 02000000)...

Starting kernel ...

With the further investigation, it never returnned back from invalidate_dcache_all(). I mean that I can't reach stage 4 as below.

cleanup_before_linux
==>cleanup_before_linux_select
==>1.disable_interrupts
==>2.dcache_disable
==>3.invalidate_dcache_all
==>4.icache_disable

Debug further, invalidate_dcache_all invalidate all cache one-by-one which cache type is DATA_ONLY, INSTRUCTION_DATA or UNIFIED. And invalidate way from one set to another set in order. The problem is that the PC ran away in invalidate way loop [cache level L1!!!].

I add some serial output code in __v7_flush_dcache_all to figure out the bog.
I expect:
Print the value of r9 in sequence, e.g 0x7f, 0x7e, 0x7d..... 0x01, 0x00 (hex)
In fact, print the value of r9 in sequence at first, but print
unexpected value afterwards. e.g 0x7f, 0x7e, 0x7d, ..,0x73,
0x40, 0x85,....

ENTRY(__v7_flush_dcache_all)
[snip]
loop1:
mov r9, r7 @ create working copy of max
index
loop2:
+ stmfd sp!, {r0}
+ ldr r0, =0xff690000
+ str r9, [r0]
+ ldmfd sp!, {r0}
ARM( orr r11, r10, r4, lsl r5 ) @ factor way and
cache number into r11
THUMB( lsl r6, r4, r5 )
THUMB( orr r11, r10, r6 ) @ factor way and
cache number into r11
ARM( orr r11, r11, r9, lsl r2 ) @ factor index
number into r11
THUMB( lsl r6, r9, r2 )
THUMB( orr r11, r11, r6 ) @ factor index
number into r11
mcr p15, 0, r11, c7, c14, 2 @ clean & invalidate
by set/way
subs r9, r9, #1 @ decrement the index
bge loop2
subs r4, r4, #1 @ decrement the way
bge loop1
skip:
ENDPROC(__v7_flush_dcache_all)

I don't have the jtag, hence I can't address the current pc. I have no doubt that if any glitches in __v7_flush_dcache_all, I reviewed several times, also copy kernel's implement to here. :-( No effect.
A more interesting thing is that Sandy had report it. He and I have similar problem. Everything work fine after I applied his patches, or disable dcache(active CONFIG_SYS_DCACHE_OFF). @Stany, I'm sorry that I disable dcache during hack.. That was a mistake:-(

Quoted 15 lines

@Simon & hieko,
Can you boot linux with the mainline u-boot? have a try?

_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de <mailto:[email protected]>
http://lists.denx.de/mailman/listinfo/u-boot

_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

author
#5 lore raw
In reply to Ziyuan Xu

+ Simon and heiko

Quoted 185 lines · Ziyuan Xu

Dear All,

I add the ISB operation after dcache_disable(), and I can jump to linux kernel entry.:-)

arch/arm/cpu/armv7/cpu.c +1 -0
index 6eac5ef..5cc09ba 100644
@@ -43,6 +43,7 @@ int cleanup_before_linux_select(int flags)
                */
                dcache_disable();
                v7_outer_cache_disable();
+               ISB;
 
                /*
                * After D-cache is flushed and before it is disabled 

there may

Sounds crazy. In fact, there is already an 'ISB' operation in set_cr() which to disable dcache and MMU. But it just tell processor that memory had change, not real ISB for armv7.

dcache_disable

        ==>flush_dcache_all()
        ==>set_cr()   disable dcache and MMU.

#define isb() __asm__ __volatile__ ("" : : : "memory") static inline void set_cr(unsigned int val)

{
    if (is_hyp())
        asm volatile("mcr p15, 4, %0, c1, c0, 0    @ set CR" :
                                  : "r" (val)
                                  : "cc");
    else
        asm volatile("mcr p15, 0, %0, c1, c0, 0    @ set CR" :
                                  : "r" (val)
                                  : "cc");
    isb();
}

I also find something in ARM cortex-A17 TRM. ==>"Disable the MMU from the each processor followed by an ISB to ensure the MMU disable operation is complete, then followed by a DSB to drain previous memory transactions."

In my humble opinion, maybe instructions tlb had alter, and cause running away??? I'm not sure about it.
@Sandy, could you have a try with my update?

@Simon, did you hit this glitch? I hope you can help ...:-)

Quoted 111 lines · Sandy Patterson

Ah, thanks. Your debugging looks the same as what I've seen from printf debugging. I'll try to verify though.

On Mon, Jul 25, 2016 at 10:58 AM, Ziyuan Xu <[email protected] <mailto:[email protected]>> wrote:

hi Stany,

The difference is that you print out assertion log.

Reset not supported on this platform
### ERROR ### Please RESET the board ###

You can add show_boot_progress() function in your bsp file to find out something.
#define CONFIG_SHOW_BOOT_PROGRESS
void show_boot_progress(int progress)
{
printf("Boot reached stage %d\n", progress);

}

On 2016?07?25? 22:12, Ziyuan Xu wrote:

Hi All,

I'm sorry to tell you that I failed to boot linux kernel with the mainline u-boot on rk3288 board(both evb-rk3288 & fennec-rk3288). It was stuck in cleanup_before_linux() before jumping to linux, and the boot_stage_flag is BOOTSTAGE_ID_BOOTM_HANDOFF.

## Current stack ends at 0x7df638b0 * kernel: cmdline image address = 0x02000000
## No init Ramdisk
ramdisk start = 0x00000000, ramdisk end = 0x00000000
## No Flattened Device Tree
Continuing to boot without FDT
Initial value for argc=3
Final value for argc=3
using: ATAGS
## Transferring control to Linux (at address 02000000)...

Starting kernel ...

With the further investigation, it never returnned back from invalidate_dcache_all(). I mean that I can't reach stage 4 as below.

cleanup_before_linux
==>cleanup_before_linux_select
==>1.disable_interrupts
==>2.dcache_disable
==>3.invalidate_dcache_all
==>4.icache_disable

Debug further, invalidate_dcache_all invalidate all cache one-by-one which cache type is DATA_ONLY, INSTRUCTION_DATA or UNIFIED. And invalidate way from one set to another set in order. The problem is that the PC ran away in invalidate way loop [cache level L1!!!].

I add some serial output code in __v7_flush_dcache_all to figure out the bog.
I expect:
Print the value of r9 in sequence, e.g 0x7f, 0x7e, 0x7d..... 0x01, 0x00 (hex)
In fact, print the value of r9 in sequence at first, but print
unexpected value afterwards. e.g 0x7f, 0x7e, 0x7d, ..,0x73,
0x40, 0x85,....

ENTRY(__v7_flush_dcache_all)
[snip]
loop1:
mov r9, r7 @ create working copy of max
index
loop2:
+ stmfd sp!, {r0}
+ ldr r0, =0xff690000
+ str r9, [r0]
+ ldmfd sp!, {r0}
ARM( orr r11, r10, r4, lsl r5 ) @ factor way and
cache number into r11
THUMB( lsl r6, r4, r5 )
THUMB( orr r11, r10, r6 ) @ factor way and
cache number into r11
ARM( orr r11, r11, r9, lsl r2 ) @ factor index
number into r11
THUMB( lsl r6, r9, r2 )
THUMB( orr r11, r11, r6 ) @ factor index
number into r11
mcr p15, 0, r11, c7, c14, 2 @ clean & invalidate
by set/way
subs r9, r9, #1 @ decrement the index
bge loop2
subs r4, r4, #1 @ decrement the way
bge loop1
skip:
ENDPROC(__v7_flush_dcache_all)

I don't have the jtag, hence I can't address the current pc. I have no doubt that if any glitches in __v7_flush_dcache_all, I reviewed several times, also copy kernel's implement to here. :-( No effect.
A more interesting thing is that Sandy had report it. He and I have similar problem. Everything work fine after I applied his patches, or disable dcache(active CONFIG_SYS_DCACHE_OFF). @Stany, I'm sorry that I disable dcache during hack.. That was a mistake:-(

Quoted 15 lines

@Simon & hieko,
Can you boot linux with the mainline u-boot? have a try?

_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de <mailto:[email protected]>
http://lists.denx.de/mailman/listinfo/u-boot

_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

#6 lore raw

Hi Ziyuan, good work!

This does in fact fix it for me.

I followed through your notes. What is isr() supposed to do? it looks like it just tells the compiler that the memory cache has been invalidated. But in the TRM it says we need to actually do an ISB.

Would fixing the dcache flush so it does an ISB when it disables the MMU maybe be an alternative solution? (I couldn't figure out which macro I should be using. isb() is defined for arm64 to do the actual op.

Anyway, the following also fixes for me.

arch/arm/lib/cache-cp15.c +2 -0
index 1121dc3..3494a5c 100644
@@ -217,6 +217,8 @@ static void cache_disable(uint32_t cache_bit)
        if (cache_bit == (CR_C | CR_M))
                flush_dcache_all();
        set_cr(reg & ~cache_bit);
+       if(cache_bit & CR_M)
+               asm volatile ("isb" : : : "memory");
 }
 #endif
 

I'm not sure what kinds of side effects for other boards these changes would have.

Quoted 193 lines · Ziyuan Xu

+ Simon and heiko

Quoted 186 lines · Ziyuan Xu

Dear All,

I add the ISB operation after dcache_disable(), and I can jump to linux kernel entry.:-)

arch/arm/cpu/armv7/cpu.c +1 -0
index 6eac5ef..5cc09ba 100644
@@ -43,6 +43,7 @@ int cleanup_before_linux_select(int flags)
                */
                dcache_disable();
                v7_outer_cache_disable();
+               ISB;
 
                /*
                * After D-cache is flushed and before it is disabled

there may

Sounds crazy. In fact, there is already an 'ISB' operation in set_cr() which to disable dcache and MMU. But it just tell processor that memory had change, not real ISB for armv7.

dcache_disable

        ==>flush_dcache_all()
        ==>set_cr()   disable dcache and MMU.

#define isb() __asm__ __volatile__ ("" : : : "memory") static inline void set_cr(unsigned int val)

{
    if (is_hyp())
        asm volatile("mcr p15, 4, %0, c1, c0, 0    @ set CR" :
                                  : "r" (val)
                                  : "cc");
    else
        asm volatile("mcr p15, 0, %0, c1, c0, 0    @ set CR" :
                                  : "r" (val)
                                  : "cc");
    isb();
}

I also find something in ARM cortex-A17 TRM. ==>"Disable the MMU from the each processor followed by an ISB to ensure the MMU disable operation is complete, then followed by a DSB to drain previous memory transactions."

In my humble opinion, maybe instructions tlb had alter, and cause running away??? I'm not sure about it.
@Sandy, could you have a try with my update?

@Simon, did you hit this glitch? I hope you can help ...:-)

Quoted 111 lines · Sandy Patterson

Ah, thanks. Your debugging looks the same as what I've seen from printf debugging. I'll try to verify though.

On Mon, Jul 25, 2016 at 10:58 AM, Ziyuan Xu <[email protected] <mailto:[email protected]>> wrote:

hi Stany,

The difference is that you print out assertion log.

Reset not supported on this platform
### ERROR ### Please RESET the board ###

You can add show_boot_progress() function in your bsp file to find out something.
#define CONFIG_SHOW_BOOT_PROGRESS
void show_boot_progress(int progress)
{
printf("Boot reached stage %d\n", progress);

}

On 2016?07?25? 22:12, Ziyuan Xu wrote:

Hi All,

I'm sorry to tell you that I failed to boot linux kernel with the mainline u-boot on rk3288 board(both evb-rk3288 & fennec-rk3288). It was stuck in cleanup_before_linux() before jumping to linux, and the boot_stage_flag is BOOTSTAGE_ID_BOOTM_HANDOFF.

## Current stack ends at 0x7df638b0 * kernel: cmdline image address = 0x02000000
## No init Ramdisk
ramdisk start = 0x00000000, ramdisk end = 0x00000000
## No Flattened Device Tree
Continuing to boot without FDT
Initial value for argc=3
Final value for argc=3
using: ATAGS
## Transferring control to Linux (at address 02000000)...

Starting kernel ...

With the further investigation, it never returnned back from invalidate_dcache_all(). I mean that I can't reach stage 4 as below.

cleanup_before_linux
==>cleanup_before_linux_select
==>1.disable_interrupts
==>2.dcache_disable
==>3.invalidate_dcache_all
==>4.icache_disable

Debug further, invalidate_dcache_all invalidate all cache one-by-one which cache type is DATA_ONLY, INSTRUCTION_DATA or UNIFIED. And invalidate way from one set to another set in order. The problem is that the PC ran away in invalidate way loop [cache level L1!!!].

I add some serial output code in __v7_flush_dcache_all to figure out the bog.
I expect:
Print the value of r9 in sequence, e.g 0x7f, 0x7e, 0x7d..... 0x01, 0x00 (hex)
In fact, print the value of r9 in sequence at first, but print
unexpected value afterwards. e.g 0x7f, 0x7e, 0x7d, ..,0x73,
0x40, 0x85,....

ENTRY(__v7_flush_dcache_all)
[snip]
loop1:
mov r9, r7 @ create working copy of max
index
loop2:
+ stmfd sp!, {r0}
+ ldr r0, =0xff690000
+ str r9, [r0]
+ ldmfd sp!, {r0}
ARM( orr r11, r10, r4, lsl r5 ) @ factor way and
cache number into r11
THUMB( lsl r6, r4, r5 )
THUMB( orr r11, r10, r6 ) @ factor way and
cache number into r11
ARM( orr r11, r11, r9, lsl r2 ) @ factor index
number into r11
THUMB( lsl r6, r9, r2 )
THUMB( orr r11, r11, r6 ) @ factor index
number into r11
mcr p15, 0, r11, c7, c14, 2 @ clean & invalidate
by set/way
subs r9, r9, #1 @ decrement the index
bge loop2
subs r4, r4, #1 @ decrement the way
bge loop1
skip:
ENDPROC(__v7_flush_dcache_all)

I don't have the jtag, hence I can't address the current pc. I have no doubt that if any glitches in __v7_flush_dcache_all, I reviewed several times, also copy kernel's implement to here. :-( No effect.
A more interesting thing is that Sandy had report it. He and I have similar problem. Everything work fine after I applied his patches, or disable dcache(active CONFIG_SYS_DCACHE_OFF). @Stany, I'm sorry that I disable dcache during hack.. That was a mistake:-(

Quoted 16 lines

@Simon & hieko,
Can you boot linux with the mainline u-boot? have a try?

_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de <mailto:[email protected]>
http://lists.denx.de/mailman/listinfo/u-boot

_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

author
#7 lore raw

Hi Sandy,

I'm pleased to learn that it does fix for you.

Quoted 7 lines · Sandy Patterson

Hi Ziyuan, good work!

This does in fact fix it for me.

I followed through your notes. What is isr() supposed to do? it looks like it just tells the compiler that the memory cache has been invalidated. But in the

isb()?
I'm not very enrich knowledgeable in cache, this issue I'm doing also is a big learning experience for me. I catch something from TRM, and review u-boot again and again in order to figure out the root cause. I suspect the former disable_dcache/mmu operation didn't complete, or branch prediction was abnormal?
I'm not really sure.

Quoted 22 lines

TRM it says we need to actually do an ISB.

Would fixing the dcache flush so it does an ISB when it disables the MMU maybe be an alternative solution? (I couldn't figure out which macro I should be using. isb() is defined for arm64 to do the actual op.

Anyway, the following also fixes for me.

arch/arm/lib/cache-cp15.c +2 -0
index 1121dc3..3494a5c 100644
@@ -217,6 +217,8 @@ static void cache_disable(uint32_t cache_bit)
        if (cache_bit == (CR_C | CR_M))
                flush_dcache_all();
        set_cr(reg & ~cache_bit);
+       if(cache_bit & CR_M)
+               asm volatile ("isb" : : : "memory");
 }
 #endif
 
Quoted 2 lines

I'm not sure what kinds of side effects for other boards these changes would have.

Let's listen to the opinions of others.

Quoted 232 lines

On Tue, Jul 26, 2016 at 8:15 AM, Ziyuan Xu <[email protected] <mailto:[email protected]>> wrote:

+ Simon and heiko

On 2016?07?26? 14:30, Ziyuan Xu wrote:

Dear All,

I add the ISB operation after dcache_disable(), and I can jump to linux kernel entry.:-)

diff --git a/arch/arm/cpu/armv7/cpu.c b/arch/arm/cpu/armv7/cpu.c index 6eac5ef..5cc09ba 100644
--- a/arch/arm/cpu/armv7/cpu.c
+++ b/arch/arm/cpu/armv7/cpu.c
@@ -43,6 +43,7 @@ int cleanup_before_linux_select(int flags)
*/
dcache_disable(); v7_outer_cache_disable();
+ ISB;

/*
* After D-cache is flushed and before it is
disabled there may

Sounds crazy. In fact, there is already an 'ISB' operation in set_cr() which to disable dcache and MMU. But it just tell processor that memory had change, not real ISB for armv7.

dcache_disable
==>flush_dcache_all()
==>set_cr() disable dcache and MMU.

#define isb() __asm__ __volatile__ ("" : : : "memory") static inline void set_cr(unsigned int val) {
if (is_hyp())
asm volatile("mcr p15, 4, %0, c1, c0, 0 @ set CR" :
: "r" (val) : "cc");
else
asm volatile("mcr p15, 0, %0, c1, c0, 0 @ set CR" :
: "r" (val) : "cc");
isb();
}

I also find something in ARM cortex-A17 TRM. ==>"Disable the MMU from the each processor followed by an ISB to ensure the MMU disable operation is complete, then followed by a DSB to drain previous memory transactions."

In my humble opinion, maybe instructions tlb had alter, and cause running away??? I'm not sure about it. @Sandy, could you have a try with my update?

@Simon, did you hit this glitch? I hope you can help ...:-)

On 2016?07?25? 23:00, Sandy Patterson wrote:

Ah, thanks. Your debugging looks the same as what I've seen from printf debugging. I'll try to verify though.

On Mon, Jul 25, 2016 at 10:58 AM, Ziyuan Xu <xzy.xu at rock-chips.com <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>> wrote:

hi Stany,

The difference is that you print out assertion log.

Reset not supported on this platform ### ERROR ### Please RESET the board ###

You can add show_boot_progress() function in your bsp
file to find
out something.
#define CONFIG_SHOW_BOOT_PROGRESS
void show_boot_progress(int progress)
{
printf("Boot reached stage %d\n", progress);

}

On 2016?07?25? 22:12, Ziyuan Xu wrote:

Hi All,

I'm sorry to tell you that I failed to boot linux
kernel with
the mainline u-boot on rk3288 board(both evb-rk3288 & fennec-rk3288). It was stuck in
cleanup_before_linux() before
jumping to linux, and the boot_stage_flag is BOOTSTAGE_ID_BOOTM_HANDOFF.

## Current stack ends at 0x7df638b0 * kernel:
cmdline image
address = 0x02000000
## No init Ramdisk
ramdisk start = 0x00000000, ramdisk end =
0x00000000
## No Flattened Device Tree Continuing to boot without FDT Initial value for argc=3
Final value for argc=3
using: ATAGS
## Transferring control to Linux (at address
02000000)...

Starting kernel ...

With the further investigation, it never returnned
back from
invalidate_dcache_all(). I mean that I can't reach
stage 4 as
below.

cleanup_before_linux
==>cleanup_before_linux_select
==>1.disable_interrupts ==>2.dcache_disable ==>3.invalidate_dcache_all ==>4.icache_disable

Debug further, invalidate_dcache_all invalidate
all cache
one-by-one which cache type is DATA_ONLY,
INSTRUCTION_DATA or
UNIFIED. And invalidate way from one set to
another set in
order. The problem is that the PC ran away in
invalidate way
loop [cache level L1!!!].

I add some serial output code in
__v7_flush_dcache_all to
figure out the bog.
I expect:
Print the value of r9 in sequence, e.g 0x7f, 0x7e,
0x7d.....
0x01, 0x00 (hex)
In fact, print the value of r9 in sequence at
first, but print
unexpected value afterwards. e.g 0x7f, 0x7e,
0x7d, ..,0x73,
0x40, 0x85,....

ENTRY(__v7_flush_dcache_all) [snip]
loop1:
mov r9, r7 @ create working
copy of max
index
loop2:
+ stmfd sp!, {r0}
+ ldr r0, =0xff690000
+ str r9, [r0]
+ ldmfd sp!, {r0}
ARM( orr r11, r10, r4, lsl r5 ) @ factor
way and
cache number into r11
THUMB( lsl r6, r4, r5 )
THUMB( orr r11, r10, r6 ) @
factor way and
cache number into r11
ARM( orr r11, r11, r9, lsl r2 ) @ factor
index
number into r11
THUMB( lsl r6, r9, r2 )
THUMB( orr r11, r11, r6 ) @
factor index
number into r11
mcr p15, 0, r11, c7, c14, 2 @ clean
& invalidate
by set/way
subs r9, r9, #1 @ decrement the
index
bge loop2
subs r4, r4, #1 @ decrement the way
bge loop1
skip:
ENDPROC(__v7_flush_dcache_all)

I don't have the jtag, hence I can't address the
current pc. I
have no doubt that if any glitches in
__v7_flush_dcache_all, I
reviewed several times, also copy kernel's
implement to here.
:-( No effect.
A more interesting thing is that Sandy had report
it. He and I
have similar problem. Everything work fine after I
applied his
patches, or disable dcache(active
CONFIG_SYS_DCACHE_OFF).
@Stany, I'm sorry that I disable dcache during
hack.. That was
a mistake:-(

@Simon & hieko,
Can you boot linux with the mainline u-boot? have
a try?

_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de <mailto:[email protected]> <mailto:U-Boot at lists.denx.de <mailto:[email protected]>>
http://lists.denx.de/mailman/listinfo/u-boot

_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de <mailto:[email protected]>
http://lists.denx.de/mailman/listinfo/u-boot