Workaround the Out-of-order Issue of Loongson-2F
cover Workaround the Out-of-order Issue of Loongson-2F
From: Wu Zhangjin <[email protected]>
As the Chapter 15: "Errata: Issue of Out-of-order in loongson"[1] shows, to workaround the Issue of Loongson-2F，We need to do:
o When switching from user model to kernel model, you should flush the branch target history such as BTB and RAS.
o Doing some tricks to the indirect branch target to make sure that the indirect branch target can not be in the I/O region.
This patchset applied the above methods and for the binutils patch[3] have been merged into binutils 2.20.1, so, it's time to upstream this patchset now. without this patch, the machines will hang when the instruction sequence hit the Out-of-order Issue of Loongson-2F, therefore, this patchset is very urgent for both 2.6.33 and 2.6.34.
[1] Chinese Version: http://www.loongson.cn/uploadfile/file/20080821113149.pdf
[2] English Version of Chapter 15:
http://groups.google.com.hk/group/loongson-dev/msg/e0d2e220958f10a6?dmode=source
[3] http://sourceware.org/ml/binutils/2009-11/msg00387.html
Regards,
Wu Zhangjin
Wu Zhangjin (3):
Loongson-2F: Flush the branch target history such as BTB and RAS
Loongson-2F: Enable fixups of binutils 2.20.1
Loongson-2F: Fixup of problems introduced by -mfix-loongson2f-jump of
binutils 2.20.1
arch/mips/Makefile | 4 +++- arch/mips/include/asm/stackframe.h | 19 +++++++++++++++++++ arch/mips/loongson/common/reset.c | 12 +++++++++++- 3 files changed, 33 insertions(+), 2 deletions(-)
PATCH 1/3 Loongson-2F: Flush the branch target history such as BTB and RAS
From: Wu Zhangjin <[email protected]>
As the Chapter 15: "Errata: Issue of Out-of-order in loongson"[1] shows, to workaround the Issue of Loongson-2F，We need to do:
"When switching from user model to kernel model, you should flush the branch target history such as BTB and RAS."
This patch did clear BTB(branch target buffer), forbid RAS(row address strobe) via Loongson-2F's 64bit diagnostic register.
[1] Chinese Version: http://www.loongson.cn/uploadfile/file/200808211
[2] English Version of Chapter 15:
http://groups.google.com.hk/group/loongson-dev/msg/e0d2e220958f10a6?dmode=source
--- arch/mips/include/asm/stackframe.h | 19 +++++++++++++++++++ 1 files changed, 19 insertions(+), 0 deletions(-)
arch/mips/include/asm/stackframe.h +19 -0
@@ -121,6 +121,25 @@ .endm #else .macro get_saved_sp /* Uniprocessor variation */ +#ifdef CONFIG_CPU_LOONGSON2F +/* +* Clear BTB(branch target buffer), forbid RAS(row address +* strobe) to workaround the Out-of-oder Issue in Loongson2F +* via it's diagnostic register. +*/ + move k0, ra + jal 1f + nop +1: jal 1f + nop +1: jal 1f + nop +1: jal 1f + nop +1: move ra, k0 + li k0, 3 + mtc0 k0, $22 +#endif /* CONFIG_CPU_LOONGSON2F */ #if defined(CONFIG_32BIT) || defined(KBUILD_64BIT_SYM32) lui k1, %hi(kernelsp) #else
Signature · 1 line
1.7.0.1
Re: [PATCH 1/3] Loongson-2F: Flush the branch target history such as BTB and RAS
Hello.
Quoted 25 lines · Wu Zhangjin
From: Wu Zhangjin <[email protected]>
As the Chapter 15: "Errata: Issue of Out-of-order in loongson"[1] shows, to workaround the Issue of Loongson-2F，We need to do:
"When switching from user model to kernel model, you should flush the branch target history such as BTB and RAS."
This patch did clear BTB(branch target buffer), forbid RAS(row address strobe) via Loongson-2F's 64bit diagnostic register.
[1] Chinese Version: http://www.loongson.cn/uploadfile/file/200808211
[2] English Version of Chapter 15:
http://groups.google.com.hk/group/loongson-dev/msg/e0d2e220958f10a6?dmode=sourceSigned-off-by Wu Zhangjin <[email protected]>--- arch/mips/include/asm/stackframe.h | 19 +++++++++++++++++++ 1 files changed, 19 insertions(+), 0 deletions(-)arch/mips/include/asm/stackframe.h +0
-0
arch/mips/include/asm/stackframe.h line 127
Quoted 7 lines
arch/mips/include/asm/stackframe.h +4
-0@@ -121,6 +121,25 @@ .endm #else .macro get_saved_sp /* Uniprocessor variation */ +#ifdef CONFIG_CPU_LOONGSON2F +/* +* Clear BTB(branch target buffer), forbid RAS(row address +* strobe)
No spaces before the left paren...
arch/mips/include/asm/stackframe.h line 130
Quoted 3 lines
arch/mips/include/asm/stackframe.h +1
-0to workaround the Out-of-oder Issue in Loongson2F + * via it's diagnostic register.
Only "its".
arch/mips/include/asm/stackframe.h
Quoted 3 lines
arch/mips/include/asm/stackframe.h +2
-0+*/ +move k0, ra
Operands misalined...
WBR, Sergei
Re: [PATCH 1/3] Loongson-2F: Flush the branch target history such as BTB and RAS
Hi,
Thanks very much for your careful review, will resend a new one later.
Regards,
Wu Zhangjin
On Wed, 2010-03-10 at 13:44 +0300, Sergei Shtylyov wrote:
[...]
Quoted 30 lines
arch/mips/include/asm/stackframe.h line 127
Quoted 12 lines
arch/mips/include/asm/stackframe.h +4
-0@@ -121,6 +121,25 @@ .endm #else .macro get_saved_sp /* Uniprocessor variation */ +#ifdef CONFIG_CPU_LOONGSON2F +/* +* Clear BTB(branch target buffer), forbid RAS(row address +* strobe) No spaces before the left paren...
arch/mips/include/asm/stackframe.h line 130
Quoted 3 lines
arch/mips/include/asm/stackframe.h +1
-0to workaround the Out-of-oder Issue in Loongson2F + * via it's diagnostic register.Only "its".arch/mips/include/asm/stackframe.h
Quoted 3 lines
arch/mips/include/asm/stackframe.h +2
-0+*/ +move k0, raOperands misalined...
WBR, Sergei
Re: [PATCH 1/3] Loongson-2F: Flush the branch target history such as BTB and RAS
In reply to Wu ZhangjinQuoted 7 lines · Wu Zhangjin
From: Wu Zhangjin <[email protected]>
As the Chapter 15: "Errata: Issue of Out-of-order in loongson"[1] shows, to workaround the Issue of Loongson-2F，We need to do:
"When switching from user model to kernel model, you should flush the branch target history such as BTB and RAS."
Just wondered, model or mode?
Quoted 2 lines
This patch did clear BTB(branch target buffer), forbid RAS(row address strobe) via Loongson-2F's 64bit diagnostic register.
Are you sure that RAS represents "Row Address Strobe", not "Return Address Stack?"
By the way, we have a similar local workaround for vr55xx processors when switching from kernel mode to user mode. It's not necessarily related to out-of-order issues, but we need to prevent the processor from doing instruction prefetch beyond "eret" instruction.
In the long term, it would be appreciated that the kernel has a set of hooks when switching KUX-modes, so that each machine could have his own, processor-specific treatmens.
Shinya
Quoted 39 lines
[1] Chinese Version: http://www.loongson.cn/uploadfile/file/200808211
[2] English Version of Chapter 15:
http://groups.google.com.hk/group/loongson-dev/msg/e0d2e220958f10a6?dmode=sourceSigned-off-by Wu Zhangjin <[email protected]>--- arch/mips/include/asm/stackframe.h | 19 +++++++++++++++++++ 1 files changed, 19 insertions(+), 0 deletions(-)arch/mips/include/asm/stackframe.h +19
-0@@ -121,6 +121,25 @@ .endm #else .macro get_saved_sp /* Uniprocessor variation */ +#ifdef CONFIG_CPU_LOONGSON2F +/* +* Clear BTB(branch target buffer), forbid RAS(row address +* strobe) to workaround the Out-of-oder Issue in Loongson2F +* via it's diagnostic register. +*/ + move k0, ra + jal 1f + nop +1: jal 1f + nop +1: jal 1f + nop +1: jal 1f + nop +1: move ra, k0 + li k0, 3 + mtc0 k0, $22 +#endif /* CONFIG_CPU_LOONGSON2F */ #if defined(CONFIG_32BIT) || defined(KBUILD_64BIT_SYM32) lui k1, %hi(kernelsp) #else
Re: [PATCH 1/3] Loongson-2F: Flush the branch target history such as BTB and RAS
Quoted 11 lines · Shinya Kuribayashi
Quoted 7 lines · Wu Zhangjin
From: Wu Zhangjin <[email protected]>
As the Chapter 15: "Errata: Issue of Out-of-order in loongson"[1] shows, to workaround the Issue of Loongson-2F，We need to do:
"When switching from user model to kernel model, you should flush the branch target history such as BTB and RAS."
Just wondered, model or mode?
Hmm, should be mode.
Quoted 6 lines
Quoted 2 lines
This patch did clear BTB(branch target buffer), forbid RAS(row address strobe) via Loongson-2F's 64bit diagnostic register.
Are you sure that RAS represents "Row Address Strobe", not "Return Address Stack?"
Hi, Yanhua(from Lemote), can you help to clear this part?
Quoted 9 lines
By the way, we have a similar local workaround for vr55xx processors when switching from kernel mode to user mode. It's not necessarily related to out-of-order issues, but we need to prevent the processor from doing instruction prefetch beyond "eret" instruction.
In the long term, it would be appreciated that the kernel has a set of hooks when switching KUX-modes, so that each machine could have his own, processor-specific treatmens.
Good idea.
Thanks!
Regards,
Wu Zhangjin
Re: [PATCH 1/3] Loongson-2F: Flush the branch target history such as BTB and RAS
In reply to Shinya KuribayashiQuoted 7 lines · Shinya Kuribayashi
Are you sure that RAS represents "Row Address Strobe", not "Return Address Stack?"
By the way, we have a similar local workaround for vr55xx processors when switching from kernel mode to user mode. It's not necessarily related to out-of-order issues, but we need to prevent the processor from doing instruction prefetch beyond "eret" instruction.
Some R4000 revisions may do silly things in case of an NMI where c0_errorepc is pointing is pointing to an ERET instruction.
There are various processors which want to save and restore core-specific registers, for example the Cavium cnMIPS core.
Quoted 3 lines
In the long term, it would be appreciated that the kernel has a set of hooks when switching KUX-modes, so that each machine could have his own, processor-specific treatmens.
It seems that uasm is the tool of choice.
Ralf
Re: [PATCH 1/3] Loongson-2F: Flush the branch target history such as BTB and RAS
Quoted 4 lines · Ralf Baechle
Quoted 2 lines · Shinya Kuribayashi
Are you sure that RAS represents "Row Address Strobe", not "Return Address Stack?"
This should be Return Address Stack.
Will resend this patch later.
Thanks & Regards,
Wu Zhangjin
Re: [PATCH 1/3] Loongson-2F: Flush the branch target history such as BTB and RAS
Quoted 9 lines · Wu Zhangjin
Quoted 4 lines · Ralf Baechle
Quoted 2 lines · Shinya Kuribayashi
Are you sure that RAS represents "Row Address Strobe", not "Return Address Stack?"
This should be Return Address Stack.
Will resend this patch later.
Thanks for the head's up.
Signature · 2 lines
Shinya Kuribayashi NEC Electronics
PATCH 2/3 Loongson-2F: Enable fixups of binutils 2.20.1
In reply to Wu ZhangjinFrom: Wu Zhangjin <[email protected]>
As the "Fixups of Loongson2F" patch[1] to binutils have been applied into binutils 2.20.1. It's time to enable the options provided by the patch to compile the kernel.
Without these fixups, the system will hang unexpectedly for the bug of processor.
To learn more about these fixups, please refer to the following references.
[1] "Fixups of Loongson2F" patch for binutils(actually for gas)
http://sourceware.org/ml/binutils/2009-11/msg00387.html
[2] Chapter 15 of "Loongson2F User Manual"(Chinese Version)
http://www.loongson.cn/uploadfile/file/200808211
[3] English Version of the above chapter 15
http://groups.google.com.hk/group/loongson-dev/msg/e0d2e220958f10a6?dmode=source
--- arch/mips/Makefile | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-)
arch/mips/Makefile +3 -1
@@ -135,7 +135,9 @@ cflags-$(CONFIG_CPU_LOONGSON2) += -Wa,--trap cflags-$(CONFIG_CPU_LOONGSON2E) += \ $(call cc-option,-march=loongson2e,-march=r4600) cflags-$(CONFIG_CPU_LOONGSON2F) += \ - $(call cc-option,-march=loongson2f,-march=r4600) + $(call cc-option,-march=loongson2f,-march=r4600) \ + $(call as-option,-Wa$(comma)-mfix-loongson2f-nop,) \ + $(call as-option,-Wa$(comma)-mfix-loongson2f-jump,) cflags-$(CONFIG_CPU_MIPS32_R1) += $(call cc-option,-march=mips32,-mips32 -U_MIPS_ISA -D_MIPS_ISA=_MIPS_ISA_MIPS32) \ -Wa,-mips32 -Wa,--trap
Signature · 1 line
1.7.0.1
PATCH 3/3 Loongson-2F: Fixup of problems introduced by -mfix-loongson2f-jump of binutils 2.20.1
In reply to Wu ZhangjinFrom: Wu Zhangjin <[email protected]>
The -mfix-loongson2f-jump option provided by the binutils 2.20.1 have fixed the Out-of-order Issue of Loongson-2F described in Chapter 15 of "Loongson2F User Manual"[1,2], but introduced some problems.
The option changes all of the jumping target to "addr & 0xcfffffff" through the at($1) register, but for the REBOOT address of loongson-2F: 0xbfc00000, this is totally wrong, so, this patch try to avoid the problem via telling the assembler not to use at($1) register.
[1] Loongson2F User Manual(Chinese Version)
http://www.loongson.cn/uploadfile/file/200808211
[2] English Version of Chapter 15:
http://groups.google.com.hk/group/loongson-dev/msg/e0d2e220958f10a6?dmode=source
--- arch/mips/loongson/common/reset.c | 12 +++++++++++- 1 files changed, 11 insertions(+), 1 deletions(-)
arch/mips/loongson/common/reset.c +11 -1
@@ -21,8 +21,18 @@ static void loongson_restart(char *command)/* do preparation for reboot */ mach_prepare_reboot (); -/* reboot via jumping to boot base address */ +/* reboot via jumping to boot base address +* +* ".set noat" and ".set at" are used to ensure the address not broken +* by the -mfix-loongson2f-jump option provided by binutils 2.20.1 and +* higher which try to change the jumping address to "addr & +* 0xcfffffff" via the at($1) register, this is totally wrong for +* 0xbfc00000(LOONGSON_BOOT_BASE). +*/ + + __asm__ __volatile__(".set noat\n" ); ((void (*)(void ))ioremap_nocache (LOONGSON_BOOT_BASE ,4 )) (); + __asm__ __volatile__(".set at\n" ); }static void loongson_poweroff (void )
Signature · 1 line
1.7.0.1